Supervised learning trains a model on input-output pairs — each training example comes with the "correct answer" — so the model learns a mapping it can apply to new, unlabeled inputs.
Regression vs Classification
Supervised learning splits further based on what kind of output you're predicting:
| Regression | Classification | |
|---|---|---|
| Output type | Continuous number | Discrete category |
| Example | Predict a house's price | Predict spam / not spam |
| Common algorithms | Linear Regression, Random Forest Regressor | Logistic Regression, Decision Tree, SVM |
| Typical metric | RMSE, R² | Accuracy, F1-score |
Minimal End-to-End Example
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_breast_cancer
from sklearn.metrics import accuracy_score
data = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(
data.data, data.target, test_size=0.2, random_state=42
)
model = LogisticRegression(max_iter=5000)
model.fit(X_train, y_train) # learns from labeled training data
preds = model.predict(X_test) # predicts on unseen data
print(accuracy_score(y_test, preds)) # compares predictions to true labelsExpected output: roughly 0.95–0.97 accuracy on this well-behaved, well-separated dataset — real-world data is rarely this clean, so treat unusually high accuracy as a signal to double-check for data leakage, not a reason to celebrate immediately.
Practical Use Cases
- Credit risk scoring, churn prediction, price forecasting
- Medical diagnosis support (given labeled historical cases)
- Email spam detection
Advantages
- Performance is directly measurable against ground truth
- Well understood, mature tooling, easy to evaluate and compare models
Limitations
- Requires labeled data, which is often expensive or slow to collect
- A model only learns patterns present in its training labels — biased or incomplete labels produce a biased model
Common Mistakes
- Evaluating on the same data used for training, which hides overfitting — always hold out a test set (train-test split).
- Treating a classification target as regression (or vice versa) without checking whether the output is really continuous or categorical.
Interview Relevance
Q: "How do you decide between regression and classification for a problem?" Look at the target variable: if it's a continuous number (price, temperature), it's regression; if it's a category (yes/no, class A/B/C), it's classification — even if the categories are numerically coded (0/1).
Practice Question
You're predicting whether a customer will renew a subscription (yes/no) and separately, how many days until they might cancel. Which problem is regression and which is classification?