Predictions on Continual Learning
Blog post by ngeo.dev predicting how continual learning for LLMs will be solved. Claims the missing piece is test-time LoRA fine-tuning guided by interpretable features.
Key Claims
- Current continual learning approach (structured memory + in-context learning) has fundamental scaling problems: VRAM cost, context length limits, quadratic attention
- The 2x2 framework for teaching LLMs new behaviors:
- Low coverage + narrow → in-context learning
- Low coverage + wide → structured memory
- High coverage + narrow → LoRA fine-tuning (the missing piece)
- High coverage + wide → full fine-tuning (e.g., Cursor’s continuous RL)
- The missing piece: generic test-time LoRA fine-tuning for narrow applications (specific user or task)
- Proposes using SAE features or model features to guide efficient LoRA optimization at test time
- Features provide: (a) denoising via projection, (b) beneficial biases for useful learning
- On-policy distillation with rank-32 LoRA matches full SFT at 9x lower cost (Thinking Machines result)
Takeaways
- Prediction: big labs will implement feature-guided test-time LoRA by end of 2026
- Practical applications: coding agent adapting to architect/fixer/lead role, home assistant personalizing communication style
- The Titans paper’s approach (fixed-size memory module) is a step but not the solution
- This connects interpretability (finding features) to practical personalization
Connections
- Online RL for Cursor Tab — continuous RL as the “wide + high coverage” case
- Doc-to-LoRA — documents → LoRA adapters in <1s
- LLM Personalization via Reward Factorization — alternative personalization approach
- Mechanistic Interpretability — SAE features as basis for personalization
- Agent Memory — continual learning as a form of agent memory