By the end of this lesson, you will be able to build, train, and evaluate a basic machine learning model using Scikit-learn to predict outcomes from structured data.
What it is
Scikit-learn is a powerful Python library for classical machine learning. It provides simple and efficient tools for predictive data analysis, built on NumPy, SciPy, and matplotlib. The core mental model revolves around three main objects: Estimators (which learn patterns), Transformers (which preprocess data), and Predictors (which make guesses). Key related terms include fitting (training the model) and predicting (applying the trained model).
Why it matters
- Accessibility: It offers a consistent API that makes switching between algorithms (like Linear Regression and Random Forests) trivial.
- Efficiency: It includes optimized implementations for common tasks like scaling, encoding, and cross-validation.
- Integration: It works seamlessly with Pandas DataFrames and NumPy arrays, fitting naturally into existing data workflows.
- Reliability: It is battle-tested in industry and academia, ensuring stable performance for standard ML problems.
Syntax or steps
The standard workflow follows four distinct steps:
- Import: Load necessary modules from
sklearn. - Prepare: Split your dataset into training and testing sets using
train_test_split. - Train: Create an estimator instance and call its
.fit()method on the training data. - Evaluate: Use the
.score()method ormean_squared_errorto check performance on unseen test data.
Example
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
# 1. Load data
data = load_iris()
X, y = data.data, data.target
# 2. Split data
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)
# 3. Train model
model = LogisticRegression(max_iter=200)
model.fit(X_train, y_train)
# 4. Evaluate
predictions = model.predict(X_test)
print(f"Accuracy: {accuracy_score(y_test, predictions):.2f}")This code loads the famous Iris dataset, splits it so 80% is used for learning and 20% for checking. A LogisticRegression model learns the relationship between flower measurements and species. Finally, it predicts species for the test set and calculates how often it was right.
Common mistakes
- Data Leakage: Scaling or imputing data before splitting. Always split first, then transform only the training set, and apply those same transformations to the test set.
- Ignoring Convergence: Some models, like Logistic Regression, may fail to converge within default iterations. Increase
max_iterif warnings appear. - Mismatched Shapes: Ensuring
Xis always 2D (n_samples, n_features) even for single features. Usereshape(-1, 1)if needed. - Overfitting Blindly: Achieving 100% training accuracy but poor test accuracy indicates the model memorized noise rather than learning patterns.
When to use it
| Scenario | Use Scikit-learn | Use Deep Learning (e.g., PyTorch/TensorFlow) |
|---|---|---|
| Structured Tabular Data | Yes (Best choice) | No (Often overkill) |
| Small/Medium Datasets | Yes | No (Harder to train) |
| Unstructured Data (Images/Audio) | Limited | Yes |
| Need Interpretability | Yes | No (Black box) |
Practice
Guided Exercise: Modify the example above to use a RandomForestClassifier instead of LogisticRegression. Import it from sklearn.ensemble. Does the accuracy change?
Challenge: Add a preprocessing step using StandardScaler. Fit the scaler on X_train, transform both X_train and X_test, then retrain the model. Why is this important for distance-based algorithms like KNN?
Quick check
Question: What happens if you call model.fit() on the entire dataset including the test set?
Answer: You introduce data leakage. The model sees the answers during training, leading to artificially high evaluation scores that do not reflect real-world performance.
Summary
Scikit-learn simplifies classical machine learning through a consistent fit-predict interface. By strictly separating training and testing data and following the prepare-train-evaluate loop, you can build reliable predictive models for structured data efficiently.
