Choosing a model isn't just "pick the biggest one" — larger models tend to be more capable on complex tasks, but cost more, run slower, and are often unnecessary for narrow, well-defined tasks that a smaller model handles just as well.
The Core Tradeoff
| Larger Model | Smaller Model | |
|---|---|---|
| Capability on complex/ambiguous tasks | Generally stronger | Generally weaker on hard reasoning, but often sufficient for narrow tasks |
| Cost per request | Higher | Lower — often significantly |
| Latency | Slower per token generated | Faster |
| Good fit for | Complex reasoning, nuanced writing, ambiguous instructions | Classification, simple extraction, well-defined narrow tasks, high-volume low-complexity requests |
A Practical Decision Pattern
Task: "Classify this support ticket as billing / technical / general"
Question: does this task genuinely require complex reasoning?
→ No — it's a narrow classification task with clear categories.
Decision: use a smaller, cheaper, faster model. A large flagship
model would work too, but at unnecessary cost and latency for
a task this well-defined.Many production systems use a mix — a smaller/cheaper model for high-volume, well-defined subtasks, and a larger model reserved for genuinely complex steps. This is often called model selection or "model routing."
Quantization: Shrinking a Model After Training
Beyond choosing a different model entirely, an already-trained model can be quantized — its weights stored at lower numeric precision (see LLM Parameters) — to reduce memory footprint and increase inference speed, typically with a small, measurable quality tradeoff that should be evaluated for your specific use case rather than assumed to be negligible.
Common Mistakes
- Defaulting to the largest/most expensive available model for every task "to be safe" — this is a common, avoidable cost driver in real production systems
- Assuming a smaller model will underperform without actually testing it on your specific task — for narrow, well-defined tasks the gap is often smaller than expected
- Switching to a much smaller model purely for cost savings without re-evaluating output quality on your actual use case
Interview Relevance
"How would you decide which model size to use for a given feature?" is a strong practical-engineering question — a good answer discusses task complexity, cost sensitivity, latency requirements, and testing rather than defaulting to "the newest/biggest model."
Practice Question
You're building a feature that tags 100,000 support tickets per day by category, and a separate feature that drafts nuanced legal contract summaries for 20 documents per day. Which task should use a larger model, and why?