Improving Cursor Tab with Online RL
Source: https://www.cursor.com/blog/cursor-tab-online-rl Author: Jacob Jackson Date: 2025-09-12
Summary
Cursor’s engineering post on using online reinforcement learning to improve their Tab (code completion) model. Key insight: they couldn’t get better data from humans, so they generated it from the model itself via RL with user acceptance as reward.
Key Claims
- Problem: SFT on human-accepted completions hits a ceiling — the data distribution is biased toward easy cases.
- Solution: online RL where the reward is whether a user accepts the suggestion (implicit feedback, no labeling).
- Key challenge: reward signal is noisy — users accept bad suggestions and reject good ones. Need denoising.
- Results: 15% improvement in acceptance rate vs. pure SFT baseline with the same model size.
- Generalization surprise: RL model improved on types of completions not seen in the reward data — suggesting genuine capability generalization.
- Lesson: implicit user feedback at scale can replace explicit human labeling for alignment.
Connection to Other Sources
Practical complement to GRPO++ theory. Confirms Agent Labs Thesis — operational data creates compounding advantage.
Entities
- Cursor — AI code editor; Jacob Jackson is an ML researcher there
Concepts
- RL Infrastructure — online RL from implicit feedback
- Coding Agents — code completion as first step toward agentic coding
- Synthetic Data — rollouts from the model itself as implicit synthetic data