Continual learning tackles training a model on a sequence of tasks arriving over time — without access to earlier tasks' data — while avoiding the exact catastrophic forgetting failure flagged back in Fine-Tuning.
The Problem, Precisely
Train a model on Task A, then Task B, without revisiting Task A's data at all. Standard gradient descent, optimizing purely for Task B's loss, has no built-in incentive to preserve whatever the model learned for Task A — it can (and often does) drift significantly, degrading or entirely erasing Task A performance, a specific instance of catastrophic forgetting arising here from a sequence of tasks rather than aggressive single-task fine-tuning.
Three Families of Techniques
| Family | Core Idea |
|---|---|
| Regularization-based | Add a penalty discouraging changes to weights identified as important for earlier tasks |
| Replay-based | Keep (or generate) a small sample of earlier tasks' data, mixing it into later training to keep reinforcing old knowledge |
| Architecture-based | Allocate new, dedicated capacity (new parameters/modules) for each new task, protecting earlier tasks' dedicated parameters from being touched at all |
EWC — Elastic Weight Consolidation (Regularization-Based)
\(F_i\) is the (approximate) "importance" of parameter \(i\) for the previous task (estimated via the Fisher information, related to how sensitive the old task's loss was to that specific parameter), and \(\theta_{i,\text{old}}^*\) is that parameter's value after training on the old task. This adds a penalty — similar in spirit to L2 regularization from L2 Regularization, but anchored to the previous task's learned values rather than zero, and weighted by each parameter's estimated importance — discouraging important parameters from drifting far from their old-task values, while allowing less-important parameters more freedom to adapt to the new task.
Code — A Simplified EWC Penalty
import torch
def ewc_penalty(model, old_params, fisher_importance, lambda_reg=1000):
penalty = 0
for name, param in model.named_parameters():
penalty += (fisher_importance[name] * (param - old_params[name]) ** 2).sum()
return lambda_reg * penalty
# total_loss = new_task_loss + ewc_penalty(model, old_params, fisher_importance)
# Parameters with HIGH fisher_importance (deemed critical for the old task) resist
# changing much; parameters with low importance can adapt more freely to the new taskCommon Mistakes
- Assuming continual learning is simply "sequential fine-tuning done carefully" — without a dedicated technique (regularization, replay, or architectural), sequential fine-tuning on new tasks reliably suffers catastrophic forgetting; this category exists specifically because naive sequential training doesn't work well.
- Assuming replay-based approaches require storing the entirety of earlier tasks' data — even a small, carefully selected subset of old examples, mixed into new training, often substantially mitigates forgetting.
Interview Relevance
Q: "Why does naively fine-tuning a model sequentially on Task A, then Task B, typically cause it to forget Task A?" Standard gradient descent on Task B's loss alone has no mechanism preserving knowledge relevant only to Task A — nothing in the training objective penalizes drifting away from good Task-A solutions. This is exactly catastrophic forgetting, and it's what continual learning techniques (regularization anchoring important weights, replaying old data, or dedicating separate architecture per task) are specifically designed to counteract.
Practice Question
How does EWC's penalty differ from standard L2 regularization, both in what it's anchored to and in how uniformly it's applied across parameters?
