Triplet Loss

Triplet Loss extends Contrastive Loss's pairwise idea to three examples at once — an anchor, a positive (similar to the anchor), and a negative (dissimilar to the anchor) — directly optimizing that the anchor be closer to the positive than to the negative, by a margin.

Formula

\(a\) is the anchor's embedding, \(p\) is a positive example's embedding (same identity/class as the anchor), \(n\) is a negative example's embedding (different identity/class). \(d(\cdot,\cdot)\) is typically Euclidean distance. \(m\) is the margin.

Reading the Formula

The loss is zero exactly when \(d(a,n) - d(a,p) \ge m\) — the negative is already farther from the anchor than the positive, by at least the margin. If the negative is too close (or even closer than the positive), the loss is positive and pushes the network to increase \(d(a,n)\) and/or decrease \(d(a,p)\) until the margin condition holds.

Diagram — What the Loss Is Optimizing For

Numerical Example

With \(m=0.5\): \(d(a,p)=0.3\), \(d(a,n)=0.6\). \(L=\max(0, 0.3-0.6+0.5)=\max(0,0.2)=0.2\) — even though the negative is already farther than the positive, the gap (0.3) is smaller than the required margin (0.5), so there's still a penalty. If instead \(d(a,n)=1.0\): \(L=\max(0,0.3-1.0+0.5)=\max(0,-0.2)=0\) — the margin condition is satisfied, zero loss.

Code

import torch
import torch.nn.functional as F

def triplet_loss(anchor, positive, negative, margin=0.5):
    d_ap = F.pairwise_distance(anchor, positive)
    d_an = F.pairwise_distance(anchor, negative)
    return torch.clamp(d_ap - d_an + margin, min=0).mean()

anchor = torch.tensor([[1.0, 1.0]])
positive = torch.tensor([[1.1, 1.1]])
negative = torch.tensor([[5.0, 5.0]])
print(triplet_loss(anchor, positive, negative))

# PyTorch also provides this directly:
triplet_fn = torch.nn.TripletMarginLoss(margin=0.5)
print(triplet_fn(anchor, positive, negative))

Triplet Mining — The Practical Challenge

Randomly sampled triplets are often "easy" (already satisfying the margin, contributing zero gradient) once training progresses — wasting most of a batch's compute on triplets that teach the network nothing new. Hard negative mining — deliberately selecting negatives that are currently close to the anchor (or positives that are currently far) — is a critical, non-trivial part of making triplet loss train effectively in practice, and is a well-known engineering challenge in systems like FaceNet.

Common Mistakes

• Sampling triplets uniformly at random throughout training — as the network improves, most random triplets become "easy" and stop contributing useful gradient, stalling further learning without deliberate hard-negative mining.
• Confusing triplet loss with contrastive loss in an interview — triplet loss compares relative distances (positive vs. negative, both relative to the same anchor) in one loss term, while contrastive loss evaluates pairs independently against an absolute margin.

Interview Relevance

Q: "Why is triplet loss often preferred over contrastive loss for face recognition systems like FaceNet?" Triplet loss directly optimizes a relative ranking — that the positive be closer than the negative by a margin — which matches the actual goal of face verification (correctly ranking "is this the same person") more directly than contrastive loss's independent, absolute per-pair distance targets.

Practice Question

With margin \(m=1.0\), \(d(a,p)=0.2\) and \(d(a,n)=0.9\), compute the triplet loss. Is this triplet "easy" (near-zero gradient) or "hard" (still contributing meaningful gradient) at this point in training?

Related DL Notes

• Contrastive Loss
• Face Recognition
• Vector Norms