An embedding model is a separate, specialized model from a general-purpose LLM — trained specifically to produce vectors where distance reflects semantic similarity, not to generate fluent text.
Embedding Models vs LLMs — Different Jobs
| LLM (e.g. a chat model) | Embedding Model | |
|---|---|---|
| Input | A prompt | A piece of text |
| Output | Generated text, token by token | One fixed-size vector, in a single pass |
| Training objective | Predict the next token accurately | Produce vectors where similar-meaning texts are close together |
| Typical cost | Higher, scales with output length | Lower, fixed per input |
Some providers offer embedding models as a completely separate product/API from their chat/generation models — worth checking whether your provider's embedding model is a distinct, purpose-built model rather than assuming it's "the same model" used for chat.
Embedding Models Differ From Each Other Too
- Dimensionality — different models output vectors of different sizes (see Embedding Dimensions)
- Training domain — some are general-purpose; others are tuned for specific domains (code, multilingual text, specific industries)
- Max input length — embedding models have their own input length limits, separate from any LLM's context window
Important: Vectors From Different Models Aren't Interchangeable
# WRONG — comparing vectors from two different embedding models
vector_a = model_A.embed("some text")
vector_b = model_B.embed("other text")
similarity = cosine_similarity(vector_a, vector_b) # meaningless —
# different
# models produce
# incompatible
# vector spacesAll vectors compared to each other in a search/retrieval system must come from the same embedding model — mixing models (or switching models without re-embedding existing content) silently breaks similarity comparisons.
Practical Use Case
Choosing an embedding model is a real decision — general-purpose models work well for most applications, but a domain-specific model (e.g. one trained on code, or on a specific language) can meaningfully outperform a general model for that specific use case.
Common Mistakes
- Switching embedding models without re-embedding all previously stored content — old and new vectors become incompatible, silently corrupting search quality
- Assuming a general-purpose embedding model is automatically the best choice for a specialized domain (like source code) without testing
Interview Relevance
"If you switch your embedding model, what has to happen to your existing vector database?" — every previously stored vector must be regenerated with the new model; old and new vectors aren't comparable, a commonly overlooked migration cost.
Practice Question
Explain why a support-search system that switched embedding models but forgot to re-embed its existing 50,000 stored documents would start returning poor search results.