This note previews the transition from a trained, evaluated model to a live, running system — the complete deployment lifecycle, covered in full technical depth in the dedicated Deployment category right after this one.
The Deployment Stages, Previewed
| Stage | What Happens | Covered In |
|---|---|---|
| Serialization | Converting the trained model into a portable, deployment-ready format | Model Serialization, TorchScript, ONNX |
| Serving | Wrapping the model in an API that other systems can actually call | FastAPI Model Serving, REST API Deployment |
| Packaging | Bundling the model, code, and dependencies into a reproducible, deployable unit | Docker Deployment |
| Infrastructure | Deciding where and how the model runs — GPU vs CPU, cloud provider, scaling strategy | GPU Deployment, Cloud Deployment |
| Optimization | Making inference fast and efficient enough for real production traffic | Model Optimization (Deployment) |
Deployment Isn't the End of the Project
A common, costly misconception is treating deployment as the final step of a project — in reality, a deployed model needs ongoing monitoring (the very next note, and the entire Production DL & MLOps category), since real-world data can shift over time in ways that quietly degrade a model's performance long after its initial evaluation looked strong.
Batch vs Real-Time Inference — An Early Decision
| Batch Inference | Real-Time Inference | |
|---|---|---|
| When predictions are made | Periodically, on accumulated data (e.g. nightly) | Immediately, on each individual request as it arrives |
| Latency requirement | Relaxed — minutes to hours is often fine | Strict — often milliseconds |
| Example use case | Nightly recommendation refresh for all users | A live chatbot response, fraud detection at transaction time |
This distinction, covered fully in Batch Inference and Real-Time Inference, shapes many downstream infrastructure and optimization decisions, and is worth deciding early, since it affects the entire deployment architecture.
Common Mistakes
- Treating model deployment as a one-time "ship it and move on" event, rather than the beginning of an ongoing operational responsibility requiring monitoring and maintenance.
- Deciding on batch vs real-time inference as an afterthought, late in the project, after infrastructure decisions have already been made that don't actually support the chosen approach well.
Interview Relevance
Q: "Why is deployment often described as 'the beginning of a model's real lifecycle,' not the end of a project?" A model's real-world performance can degrade over time even without any code changes, as the actual data it encounters in production shifts away from the training data's distribution — a phenomenon covered fully as data/concept drift in the Production DL & MLOps category. Ongoing monitoring, and often periodic retraining, is required after deployment to keep a model performing well, making deployment the start of an ongoing operational responsibility rather than a final, one-time milestone.
Practice Question
Would a model that classifies whether a submitted document is fraudulent likely need batch or real-time inference? Justify your answer.
