Reinforcement Fine-Tuning for Voice Agents
Source: https://polyai.com/blog/reinforcement-fine-tuning-voice Author: TomHaynes (via PolyAI) Date: 2025-07-12
Summary
PolyAI’s approach to training voice agents using RL from full conversational outcomes. Key insight: instead of training on individual turn quality, they train on whether the full conversation achieved its goal.
Key Claims
- Voice agent RL challenge: reward assignment is hard — which turns in a conversation caused success or failure?
- PolyAI’s approach: outcome-based RL, where the reward is the full conversation outcome (did the customer solve their problem?).
- Credit assignment solution: use a “value function” trained separately to predict conversation outcome at each turn.
- Result: voice agents trained this way learn naturally conversational behaviors that SFT cannot produce.
- The generalization: this approach works for any task where the outcome is binary but the path has many steps.
Concepts
- RL Infrastructure — full-conversation outcome RL
- Reward Hacking — outcome-based RL avoids turn-level metric gaming