Continual Learning and Catastrophic Forgetting
Continual learning (or lifelong learning) trains models on tasks presented sequentially, with the goal of improving on new tasks without forgetting previously learned knowledge. This contrasts with standard supervised learning, where all data is available at once.
Catastrophic forgetting is the core challenge: when learning task $B$ after task $A$, neural networks often forget task $A$ entirely — parameters are rewritten to optimize for task $B$.
This is critical for real-world AI: systems encounter new data and tasks throughout their lifetime. Unlike humans, deep networks don’t naturally retain old knowledge.
The Stability-Plasticity Dilemma
Stability: Retain knowledge on old tasks.
Plasticity: Learn new tasks quickly.
These are in tension:
- Too stable: Can’t learn new things (network becomes rigid).
- Too plastic: Forget old things (catastrophic forgetting).
Continual learning seeks to balance both.
Catastrophic Forgetting
Why It Happens
Neural networks use distributed representations. When trained on task $B$, parameters shift to optimize task $B$‘s loss, overwriting parameters used for task $A$.
$$\theta^*B = \arg \min\theta \mathcal{L}_B(\theta)$$
Optimal parameters for task $B$ differ from task $A$; they occupy different regions of parameter space.
Demonstration
Train a network on MNIST (handwritten digits), then fine-tune on CIFAR-10 (natural images). Performance on MNIST drops from 98% to 5% — catastrophic.
Continual Learning Approaches
Replay-Based Methods
Experience replay: Store exemplars (representative samples) from old tasks. When learning new tasks, interleave old exemplars:
$$\mathcal{L} = \mathcal{L}{\text{new task}} + \beta \mathcal{L}{\text{replayed old tasks}}$$
Advantages:
- Simple; directly combats forgetting.
- Effective at balancing stability and plasticity.
Limitation: Storing exemplars requires memory; in a continual setting with hundreds of tasks, storage becomes prohibitive.
Regularization-Based Methods
Elastic Weight Consolidation (EWC):
After learning task $A$, compute the Fisher Information Matrix (FIM) — which parameters are important for task $A$?
$$\mathcal{L}_{\text{new}} = \mathcal{L}_B(\theta) + \frac{\lambda}{2} \sum_i F_i (\theta_i - \theta^*_A)^2$$
where $F_i$ is the FIM and $\theta^*_A$ are optimal parameters after task $A$.
Penalizes changing important parameters (high Fisher information). Plasticity on unimportant parameters; stability on important ones.
Advantages: No exemplar storage; principled (information-theoretic).
Limitation: FIM computation is expensive; assumes task boundaries are known.
Synaptic Importance
Synaptic Importance Weighted Experience Replay (SI-ER):
Track how much each parameter affects training loss (synaptic importance). Regularize important parameters:
$$\omega_i = \sum_b \left| \frac{\partial \mathcal{L}_b}{\partial \theta_i} \right|$$
Higher importance $\omega_i$ means parameter is critical; penalize its change more strongly.
Combines replay with importance weighting for efficient continual learning.
Architecture-Based Methods
Dynamically Expandable Networks (DER):
Add new capacity (neurons, layers) for each new task. Old task parameters are frozen; new task learns on new capacity.
$$\theta_{\text{task 1}}: \text{frozen}$$ $$\theta_{\text{task 2}}: \text{new parameters}$$
Advantages: No forgetting by design (old parameters aren’t updated).
Limitation: Unbounded growth; network becomes huge after many tasks. Inefficient.
Progressive Neural Networks:
New task learns on top of old task features, with gating mechanisms:
$$h_{\text{new}} = f_{\text{new}}(x) + g_{\text{new}}(h_{\text{old}})$$
where $g$ is a learned gating function. Old features are protected; new task can leverage or ignore them.
Meta-Learning Approaches
MAML for continual learning:
Optimize for a learning rate that works well across multiple tasks. Apply this meta-learned learning rate to new tasks, enabling quick adaptation without forgetting.
$$\text{loss}{\text{meta}} = \sum{\text{tasks}} \mathcal{L}(\theta - \alpha \nabla \mathcal{L}{\text{task}}, D{\text{task}})$$
Task Boundaries and Online Continual Learning
Assumption
Most continual learning assumes task boundaries are known: you know when you switch from one task to another.
In reality, task boundaries are often unknown. Online continual learning removes this assumption: stream of data with unknown task changes.
Challenges
Without task boundaries, distinguishing between:
- Forgetting (parameters drifting from old task optimum).
- Distribution shift (same task, different data).
is difficult.
Evaluation
Metrics
Backward transfer: How much does learning task $B$ hurt performance on task $A$?
$$BT = \text{Acc}_A^{\text{initial}} - \text{Acc}_A^{\text{after task B}}$$
Lower (more negative) BT indicates more forgetting.
Forward transfer: Does learning task $A$ help on task $B$?
$$FT = \text{Acc}_B^{\text{with A}} - \text{Acc}_B^{\text{without A}}$$
Positive FT indicates transfer.
Average accuracy: Mean accuracy across all tasks after learning all of them.
Benchmarks
Class-Incremental Learning: Classes arrive sequentially. Learn new classes without forgetting old ones.
Domain-Incremental: Same classes, but data distribution shifts (e.g., rotated MNIST, then sketch MNIST).
Task-Incremental: Different tasks with task boundaries known.
Challenges
Open-Ended Learning
How many tasks can a continual learner handle? After 1000 tasks, can it still learn a new one?
Capacity limit: Finite networks have limited capacity. Architectural methods (expanding networks) grow unbounded.
Generalization
Does continual learning improve generalization to unseen tasks? Or does it just avoid forgetting?
Online Learning
Real-time learning with single-pass data (no replay possible) remains difficult.
Applications
Robotics
Robots learn multiple skills sequentially: grasp-and-place, navigation, manipulation. Each new skill shouldn’t degrade previous skills.
Recommendation Systems
User preferences evolve. Models must adapt to new user interests without forgetting old preferences.
Autonomous Vehicles
Encounter new scenarios (weather, pedestrians, traffic patterns) throughout deployment. Models should improve without catastrophic failure on previously learned scenarios.
Edge Devices
Mobile phones or IoT devices learn from user data over time. Continual learning enables personalization without centralized training.
Research Directions
- Long-horizon continual learning: Learning from thousands of tasks while maintaining performance.
- Generalization in continual learning: Do continually learned models generalize better or worse than standard models?
- Continual meta-learning: Meta-learn learning strategies optimized for continual settings.
- Neuro-plastic systems: Brain-inspired approaches that dynamically reorganize for continual learning.
Continual learning is essential for AI systems that must adapt and improve throughout their lifetime, bridging the gap between static pre-trained models and truly lifelong learning agents.