The dot product multiplies two vectors' matching components and sums the results — turning two vectors into a single number that measures how much they point in the same direction. It's the single most-used operation in classical ML.
Formula
\(a_i, b_i\) are the components of vectors \(\vec{a}\) and \(\vec{b}\), and \(\theta\) is the angle between them. The second form is what makes the dot product geometrically meaningful: it directly encodes how aligned two vectors are.
Geometric Intuition
The dot product equals \(\lVert a \rVert\) times the length of b's shadow cast onto a — the projection.
When \(\theta = 0°\) (vectors point the same way), \(\cos\theta = 1\) — the dot product is maximized. When \(\theta = 90°\) (perpendicular), \(\cos\theta = 0\) — the dot product is exactly zero. When vectors point in opposite directions, the dot product is negative.
Numerical Example
For \(\vec{a} = [3, 4]\) and \(\vec{b} = [4, 3]\):
import numpy as np
a = np.array([3, 4])
b = np.array([4, 3])
dot = np.dot(a, b) # 24
cos_theta = dot / (np.linalg.norm(a) * np.linalg.norm(b))
print(dot, cos_theta) # 24 0.96Where the Dot Product Shows Up in ML
- Linear/logistic regression: a prediction is a dot product of the weight vector and feature vector: \(z = \vec{w}\cdot\vec{x} + b\)
- SVM: the decision boundary equation \(\vec{w}^T\vec{x}+b=0\) is a dot product
- Cosine similarity (built from the dot product) measures how similar two documents or embedding vectors are — core to search and recommendation systems
- Neural networks: every neuron computes a dot product of its inputs and weights before applying an activation function
Common Mistakes
- Assuming a large dot product always means "similar" — it also grows with vector magnitude, not just direction. Cosine similarity (dividing by both magnitudes) isolates the direction/angle effect.
- Trying to take a dot product of vectors with mismatched lengths — undefined, just like vector addition.
Interview Relevance
Q: "What does it mean when the dot product of two feature vectors is zero?" The vectors are orthogonal (perpendicular) — geometrically, they share no directional overlap; in ML this often signals the underlying features they represent are unrelated/uncorrelated in that vector space.
Practice Question
Compute the dot product of \(\vec{a}=[1,0]\) and \(\vec{b}=[0,1]\) by hand. What does the result tell you about the angle between them?
