A complete, production-shaped Random Forest workflow — training, tuning the key hyperparameters via cross-validation, and using the OOB score as a fast sanity check along the way.
The Full Workflow
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report
from sklearn.datasets import load_breast_cancer
# 1. Dataset
data = load_breast_cancer()
X, y = data.data, data.target
# 2. Train/test split
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
# 3. Quick OOB check before formal tuning -- a fast sanity signal, not a substitute for CV
quick_model = RandomForestClassifier(n_estimators=300, oob_score=True, random_state=42)
quick_model.fit(X_train, y_train)
print("OOB score:", quick_model.oob_score_)
# 4. Tune the key hyperparameters via cross-validation
param_grid = {
"n_estimators": [100, 300],
"max_depth": [None, 6, 10],
"max_features": ["sqrt", "log2"],
"min_samples_leaf": [1, 3, 5],
}
grid_search = GridSearchCV(
RandomForestClassifier(random_state=42), param_grid, cv=5, scoring="accuracy", n_jobs=-1
)
grid_search.fit(X_train, y_train)
print("Best params:", grid_search.best_params_)
# 5. Evaluate the tuned model
best_model = grid_search.best_estimator_
predictions = best_model.predict(X_test)
print(classification_report(y_test, predictions))The Hyperparameters That Actually Matter Most
| Hyperparameter | Effect |
|---|---|
n_estimators | More trees generally helps up to a point, then plateaus while training time keeps growing |
max_depth | Limits individual tree complexity — deeper trees fit more, but each overfits more too |
max_features | Controls how many features are considered per split — smaller values decorrelate trees more |
min_samples_leaf | Prevents leaves from becoming too small/specific, a form of built-in regularization |
Parallelizing Training
# n_jobs=-1 uses all available CPU cores -- trees are independent, so this parallelizes cleanly
model = RandomForestClassifier(n_estimators=300, n_jobs=-1, random_state=42)
model.fit(X_train, y_train)Because every tree in a Random Forest trains completely independently (unlike boosting, where trees depend sequentially on previous ones), training parallelizes almost perfectly across CPU cores — a genuine practical advantage over sequential ensemble methods.
Regression — The Same Pattern
from sklearn.ensemble import RandomForestRegressor
reg_model = RandomForestRegressor(n_estimators=300, max_features="sqrt", n_jobs=-1, random_state=42)
reg_model.fit(X_train_reg, y_train_reg)Common Mistakes
- Running a massive grid search over every hyperparameter simultaneously — start with
n_estimatorsandmax_depth, which usually matter most, before expanding the search. - Forgetting
random_state, making results hard to reproduce across runs given the algorithm's inherent randomness. - Not using
n_jobs=-1on a multi-core machine, leaving significant training speed on the table for no reason.
Interview Relevance
Q: "Why does Random Forest training parallelize better than gradient boosting?" Random Forest's trees are trained completely independently of each other (each on its own bootstrap sample), so they can all be built simultaneously across CPU cores; gradient boosting trains trees sequentially, where each new tree depends on the errors of all previous ones, making that parallelization impossible.
Practice Question
Modify the grid search above to also include criterion (Gini vs entropy) as a tuned hyperparameter.
