Continual Learning
How LLMs can learn continuously from experience, adapting their behavior over time without full retraining. The bridge between static model weights and dynamic agent memory.
The Problem
Pre-trained LLMs are frozen after training. Current “memory” approaches (structured memory + in-context learning) face scaling limits:
- VRAM is expensive
- Context length requires rare long-range dependency data
- Self-attention has quadratic complexity
Approaches (2x2 Framework)
From Predictions on Continual Learning:
| Low Coverage | High Coverage | |
|---|---|---|
| Narrow | In-context learning | LoRA fine-tuning |
| Wide | Structured memory | Full fine-tuning |
The “missing piece” is generic test-time LoRA fine-tuning for narrow, high-coverage cases (e.g., personalizing for a specific user or task).
Current State
- In-context learning: Agents (Claude Code) and memory frameworks (Mem0) handle well
- Full fine-tuning: Cursor’s continuous RL releases new checkpoints every 5 hours
- LoRA at test-time: No mature technique yet — predicted to emerge from big labs by end of 2026
- Feature-guided LoRA: Using SAE features to make test-time adaptation efficient and interpretable
Related Approaches
- Doc-to-LoRA — documents → LoRA adapters in <1s
- Titans paper — smart optimizer on fixed-size memory module
- On-policy distillation with LoRA matches full SFT at 9x lower cost
Open Questions
- How to weight experiences? (garbage in, garbage out)
- Self-generated token datasets are small and noisy
- How to prevent catastrophic forgetting during continual adaptation?
- Can SAE features provide stable, interpretable directions for adaptation?
Connections
- Agent Memory — continual learning as a form of memory
- Mechanistic Interpretability — features as basis for adaptation
- LLM Personalization — alternative personalization approach