CursorBench: Cursor’s Internal Evaluation Suite
Source: https://cursor.com/blog/cursorbench Author: Cursor team Date: 2025-2026
Summary
CursorBench is Cursor’s proprietary benchmark based on real sessions from their engineering team rather than public repositories. It uses “Cursor Blame” to trace committed code back to original agent requests, creating natural query-solution pairings. It shows more model separation at frontier levels than public benchmarks.
Key Claims
- Data sourced from actual Cursor engineering sessions (not synthetic repos) — reduces contamination
- Cursor Blame: traces committed code → original agent request (natural ground truth)
- Problem scope doubled from v1 to v3 — involves monorepos, production log investigation, long experiments
- Agentic graders score intentionally underspecified tasks (mirrors real dev communication)
- Key finding: public benchmarks saturate at frontier; CursorBench shows more separation
- Rankings correlate better with real developer experience than public benchmarks
Connection to Other Sources
Complements OpenAI’s real-world task evaluation — both motivated by benchmark saturation at the frontier. Both conclude: real-task performance and benchmark performance diverge.
Entities
- Cursor (coding agent company)
Concepts
- Coding Agents — the evaluation of coding agent performance
- Reward Hacking — benchmark saturation motivates real-task evals