A baseline is a reference point — typically a simpler or previously-established method — against which a new approach's results are compared, and choosing fair, strong baselines is one of the most important, and most commonly done poorly, aspects of credible research.
Why Baselines Are Essential, Not Optional
A reported metric value in isolation is nearly meaningless — "our model achieves 85% accuracy" says little without knowing what a reasonable alternative achieves. Baselines provide this essential context, transforming an isolated number into a meaningful claim about relative improvement, directly echoing the same principle already covered from a practical-project angle in Model Evaluation.
Types of Baselines
| Baseline Type | Purpose |
|---|---|
| Trivial/naive baseline | The simplest possible approach (e.g. always predicting the majority class) — establishes an absolute floor |
| Simple/classical baseline | A well-established simpler method (e.g. a linear model, a basic classical algorithm) — tests whether added complexity is genuinely justified |
| Prior state-of-the-art (SOTA) | The strongest previously published result on the same task/dataset — the most important comparison for claiming genuine progress |
| Ablated versions of the proposed method | The new method with specific components removed — isolates exactly which part of the contribution actually matters (see Ablation Studies) |
What Makes a Baseline Comparison "Fair"
- Equal tuning effort — a new method that received extensive hyperparameter tuning compared against a baseline run with only default settings produces a misleading, unfair comparison.
- Same data and evaluation protocol — comparing results computed under different data splits, preprocessing, or evaluation metrics isn't a valid apples-to-apples comparison.
- Reporting the baseline's best-known results, not a weakened or outdated version, when a stronger reported baseline result already exists in the literature.
A Concrete Illustration of Why Baselines Matter
# Without baselines, this is a nearly meaningless claim:
print("Our model achieves 85% accuracy!")
# With baselines, it becomes a genuinely meaningful, evaluable claim:
results = {
"majority class baseline": "62%",
"logistic regression baseline": "78%",
"prior SOTA (Smith et al. 2024)": "83%",
"our method": "85%",
}
# Now the 85% figure has real context: a modest but genuine 2-point
# improvement over the prior best published resultCommon Mistakes
- Comparing a heavily-tuned new method against an under-tuned or default-configuration baseline — this produces an unfairly inflated apparent improvement that doesn't reflect a genuine methodological advance.
- Omitting the strongest known prior baseline in favor of a weaker one that makes the new method's improvement look more impressive — a form of selective, misleading reporting that undermines a paper's credibility.
Interview Relevance
Q: "Why is comparing a new method only against a weak or under-tuned baseline considered poor research practice?" A reported improvement is only meaningful relative to a fair, genuinely competitive comparison — a new method that receives significant tuning and engineering effort, compared against a baseline evaluated with only default or minimal-effort settings, will show an inflated apparent improvement that doesn't reflect a genuine methodological advance. Rigorous research requires fair comparison: equal tuning effort, the same data and evaluation protocol, and comparison against the strongest known prior results, not a conveniently weaker alternative.
Practice Question
Why is an ablated version of a new method (the method with one specific component removed) itself a valuable type of baseline?
