Building AI Products In The Probabilistic Era
Source: https://giansegato.com/essays/probabilistic-era Author: Gian Segato Date: 2026-01-27
Summary
Essay arguing that AI products require fundamentally different engineering approaches because they’re probabilistic rather than deterministic. The shift from “it always does X” to “it usually does X” breaks many product and engineering assumptions.
Key Claims
- Deterministic software: given input X, always returns Y. Engineers can reason about and test this precisely.
- Probabilistic AI: given input X, returns Y with probability p. This changes testing, debugging, and reliability guarantees.
- Key shifts required: (1) statistical testing instead of unit testing, (2) confidence intervals instead of exact outputs, (3) graceful degradation instead of error handling.
- UX implication: users must be taught that AI outputs are probabilistic — “wrong sometimes” is a feature, not a bug, if managed well.
- The hardest part: deciding when probabilistic output is “good enough” for a given use case.
Connection to Other Sources
Connects to Anthropic evals and Thoughts on Evals — the probabilistic nature of AI is why evals must be statistical.
Concepts
- Coding Agents — probabilistic reliability is the key challenge for agents
- Reward Hacking — probabilistic outputs make proxy rewards easier to game