A vector database is a specialized data store built to hold large numbers of embedding vectors and efficiently find the most similar ones to a given query vector — the infrastructure that makes embedding-based retrieval practical at real scale.
Why a Specialized Database Is Needed
Finding the nearest vectors to a query among millions or billions of stored embeddings by brute-force comparison (computing similarity against every single one) becomes computationally infeasible at scale. Vector databases use specialized indexing structures for approximate nearest neighbor (ANN) search — trading a small amount of accuracy for dramatic speed improvements, making similarity search over huge collections practical in milliseconds.
Common ANN Indexing Approaches
| Approach | Core Idea |
|---|---|
| HNSW (Hierarchical Navigable Small World) | Builds a multi-layer graph structure enabling fast, approximate traversal toward nearest neighbors |
| IVF (Inverted File Index) | Clusters vectors into groups, then searches only within the most relevant clusters for a query |
| Product quantization | Compresses vectors into compact codes, trading some precision for significantly reduced memory usage |
Code — Using a Vector Database (Conceptual, FAISS-Style)
import faiss
import numpy as np
dimension = 384 # matches the embedding model's output dimension
index = faiss.IndexHNSWFlat(dimension, 32) # HNSW index
# Add embeddings to the index
document_embeddings = np.random.randn(10000, dimension).astype('float32')
index.add(document_embeddings)
# Query: find the 5 most similar documents to a query embedding
query_embedding = np.random.randn(1, dimension).astype('float32')
distances, indices = index.search(query_embedding, k=5)
print(f"Top 5 most similar document indices: {indices[0]}")Popular Vector Database Options
| Option | Notes |
|---|---|
| FAISS | A library (not a full database) for efficient similarity search — widely used as a building block |
| Pinecone, Weaviate, Milvus, Qdrant | Full managed or self-hosted vector database systems, adding persistence, filtering, and scaling on top of ANN search |
| pgvector (PostgreSQL extension) | Adds vector similarity search directly to a standard relational database — convenient when vector search needs to coexist with regular structured queries |
Common Mistakes
- Using brute-force exact nearest-neighbor search at a scale where it becomes a genuine performance bottleneck, rather than adopting an ANN-based vector database once collection size grows significantly.
- Ignoring the accuracy-speed tradeoff inherent to approximate search — ANN methods don't guarantee finding the exact true nearest neighbors, and this small approximation error is usually an acceptable, worthwhile tradeoff for the large speed gain, but it should be a conscious choice, not an unexamined assumption.
Interview Relevance
Q: "Why do vector databases use approximate nearest neighbor search instead of exact search, and what's the tradeoff?" Exact nearest neighbor search requires comparing a query against every single stored vector, which becomes computationally infeasible at the scale of millions or billions of vectors that real production systems often need to search. Approximate methods (like HNSW or IVF) use specialized indexing structures to dramatically speed up search, at the cost of occasionally missing the exact true nearest neighbor in favor of a very close, "good enough" one — a tradeoff that's almost always worthwhile in practice, since the tiny accuracy loss is far outweighed by the enormous speed gain needed for real-time applications.
Practice Question
Why might a team choose pgvector over a dedicated vector database like Pinecone for a system that also relies heavily on standard relational queries?
