Docker packages a model-serving application, together with all of its exact dependencies, into a single, portable, reproducible container — solving the classic "it works on my machine" problem for deployment.
Why Containerization Matters for Model Deployment
A deep learning serving application typically depends on a specific Python version, specific library versions (PyTorch, CUDA drivers, and more), and sometimes system-level packages — any mismatch between the development environment and the production environment can cause subtle, hard-to-diagnose failures. A Docker container bundles the exact environment together with the code, guaranteeing it runs identically wherever the container is deployed.
Code — A Minimal Dockerfile for Model Serving
# Dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY model_weights.pt .
COPY main.py .
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]Building and Running the Container
# Build the image
docker build -t my-model-api:v1 .
# Run it, mapping container port 8000 to host port 8000
docker run -p 8000:8000 my-model-api:v1
# The API is now callable at http://localhost:8000/predict
# -- identically, regardless of what's installed on the host machineGPU Support in Docker
For models requiring GPU acceleration, a CUDA-enabled base image plus the NVIDIA Container Toolkit on the host allows a container to access the host's GPU (docker run --gpus all ...) — covered in more depth in GPU Deployment.
Keeping Images Small and Efficient
| Practice | Benefit |
|---|---|
| Use a slim/minimal base image | Smaller images build, transfer, and start faster |
Use --no-cache-dir with pip, multi-stage builds | Avoids bloating the image with unnecessary build artifacts and caches |
| Copy dependency files before application code | Lets Docker's layer caching skip re-installing dependencies when only application code changes, speeding up rebuilds |
Common Mistakes
- Copying the entire project directory (including large datasets, notebooks, or unnecessary files) into the image, producing a bloated, slow-to-transfer container image.
- Placing the
COPY . .instruction before the dependency installation step in the Dockerfile — this breaks Docker's layer caching, forcing a full dependency reinstall on every single code change, dramatically slowing down iteration.
Interview Relevance
Q: "Why does the order of instructions in a Dockerfile matter for build speed, specifically around copying dependency files versus application code?" Docker caches each build instruction's result as a layer, and reuses cached layers for instructions that haven't changed since the last build. Copying and installing dependencies (which change infrequently) before copying application code (which changes frequently) means routine code changes only invalidate the fast, final layers — the slow dependency installation step is skipped and reused from cache — dramatically speeding up iterative rebuilds during development.
Practice Question
Why might a Docker image built with a full, non-slim base image and no attention to layer ordering be problematic for a team that deploys frequent, small code updates?
