Coding Agents
Definition
AI systems that autonomously write, execute, test, and iterate on code — going beyond code completion to full agentic loops: read problem → write code → run → observe result → revise.
Current State of the Art (2026)
- Claude Code (Opus 4.6): the leading general coding agent as of early 2026
- Codex (OpenAI): used by OpenAI internally via harness engineering
- SWE-bench / merge rate as evaluation (see Are LLMs Not Getting Better?)
What Works
Autoresearch and optimization: With access to GPU clusters, coding agents can run 910+ experiments in 8 hours, discovering interaction effects that sequential search misses (Scaling Autoresearch). They discover emergent strategies (heterogeneous hardware routing) without explicit instruction.
Algorithm optimization: Claude Code matched published SOTA on algorithmic problems without evolutionary scaffolding, given only problem + solution + evaluator (AutoEvolver).
Architecture analysis: A coding agent reverse-engineered Claude Code’s architecture in one day using multi-agent orchestration (Claude Code Analysis).
What Doesn’t Work (Yet)
Merge rate stagnation: Statistical analysis shows LLM merge rates (code maintainers would approve) haven’t improved since early 2025 (Are LLMs Not Getting Better?). Test passage ≠ real quality.
Cognitive debt: Agents generate code faster than humans can understand it. “Tiny booboos compound at a rate that’s unsustainable” (Slowing Down). Speed removed the human cognitive bottleneck but didn’t replace it.
Aspiration satisficing: Agents declare victory at local optima. Need explicit “aspiration prompting” (tell the agent the SOTA is X) to break through (AutoEvolver).
Design Patterns
- Harness engineering: Encode quality discipline into the environment (tests, linters, automated checks) rather than relying on agent judgment (Harness Engineering)
- Parallel factorial search: More GPUs → factorial grids → catch parameter interaction effects (Scaling Autoresearch)
- Tool categorization by side effects: Read-only tools parallel, write operations serialized (Claude Code architecture)
- Agent Interaction Guidelines: Transparency, disclose agent identity, keep humans accountable (Linear AIG)
Open Questions
- Is the merge rate plateau a ceiling, or did it step up with Claude 4 / Gemini 2.5?
- What’s the right human-in-the-loop cadence for different task types?
- Can aspiration prompting scale to complex multi-month engineering projects?