A vector is an ordered list of numbers. In machine learning, every data point — a customer, an image, a sentence — becomes a vector of feature values before a model can touch it.
What a Vector Actually Represents
A house with 1200 sq ft, 3 bedrooms and 2 bathrooms becomes the vector \([1200, 3, 2]\) — a point in 3-dimensional space. A dataset of 500 houses becomes 500 such vectors, stacked into a matrix. This is true for every ML input: text becomes vectors of word/token weights, images become vectors of pixel intensities.
Vector Addition — Geometrically
Tip-to-tail rule: slide v2 to start where v1 ends — the vector from the origin to that new tip is v1 + v2.
Formula — Addition, Scalar Multiplication, Magnitude
\(\vec{v}, \vec{w}\) are vectors, \(v_i\) is the \(i\)-th component, \(k\) is a scalar (plain number), and \(\lVert \vec{v} \rVert\) (magnitude, or "norm") is the vector's length — computed the same way as the Pythagorean theorem, extended to \(n\) dimensions.
Numerical Example
For \(\vec{v} = [3, 4]\): \(\lVert \vec{v} \rVert = \sqrt{3^2 + 4^2} = \sqrt{9+16} = \sqrt{25} = 5\).
import numpy as np
v = np.array([3, 4])
w = np.array([1, 2])
print(v + w) # [4 6]
print(2 * v) # [6 8]
print(np.linalg.norm(v)) # 5.0Why This Matters for ML
- Magnitude (\(\lVert \vec{v} \rVert\)) is the basis of distance calculations in KNN and clustering
- A model's learned parameters (weights) are literally a vector — linear regression's coefficients form a weight vector
- Feature vectors are what every scikit-learn model's
Xactually is, row by row
Common Mistakes
- Adding vectors of different lengths (different numbers of features) — vector addition is only defined when dimensions match.
- Confusing a vector's magnitude (a single number, its length) with the vector itself (a list of numbers).
Interview Relevance
Q: "Why does scaling matter for distance-based algorithms like KNN?" Because distance is computed from vector magnitude — a feature on a 0–100,000 scale dominates the magnitude calculation over a feature on a 0–1 scale, even if both are equally important. See Feature Scaling.
Practice Question
Given \(\vec{a} = [6, 8]\), compute \(\lVert \vec{a} \rVert\) by hand, then verify it with NumPy's np.linalg.norm().
