Thoughts on Evals

Source: https://www.raindrop.ai/blog/thoughts-on-evals Author: Raindrop AI Date: 2026-01-27

Summary

Practitioner perspective on building evaluations for AI agents. Raindrop AI builds customer service AI agents; their eval challenges are real-world rather than academic.

Key Claims

• Evals for customer service agents: you need to measure not just correctness but tone, escalation decisions, and policy compliance.
• The “can’t test everything” problem: agent behavior space is effectively infinite — you need strategic sampling, not comprehensive coverage.
• Eval failure modes: (1) testing the wrong thing (proxy metrics that don’t track real quality), (2) evaluating too rarely (problems accumulate), (3) over-indexing on aggregate metrics (missing systematic failures for specific user groups).
• Recommendation: combine automated evals (for coverage) with human spot-checks (for validity) — neither alone is sufficient.
• Adversarial evals: intentionally try to break your agent. If you can’t break it, your evals are probably too easy.

Concepts

• Coding Agents — eval design for agents
• Reward Hacking — proxy metrics that don’t track real quality