Contra DSPy and GEPA
Source: https://benanderson.work/contra-dspy-gepa Author: Ben Anderson Date: 2025-12-18
Summary
Skeptical critique of DSPy (automated prompt optimization framework) and GEPA (gradient-based prompt optimization). Argues that automatic prompt optimization frameworks solve the wrong problem and create new problems.
Key Claims
- DSPy’s premise: automate the process of optimizing prompts through search and evaluation. Let the framework write prompts.
- The problem with auto-optimization: optimized prompts are often uninterpretable and brittle — they work in the evaluation setup but fail in production.
- GEPA’s more fundamental issue: gradient-based prompt optimization can only find prompts in a local optimum near the starting point.
- What’s missing: human judgment about what the prompt should accomplish. Automation doesn’t replace this.
- Counter-argument: DSPy with good human-defined metrics and diverse test cases can be genuinely useful. The critique is against blind automation, not optimization assistance.
Concepts
- Context Engineering — prompt optimization as context engineering automation
- Reward Hacking — auto-optimized prompts can hack their own evaluation metrics