FP16 vs BF16 for RL Training
Source: https://x.com (tweet) Author: unknown (ML researcher) Date: 2025-2026
Summary
Surprising finding: FP16 has smaller training-inference gap than BFloat16 for RL, meaning it fits better for RL training. Even differences between RL algorithms vanish once FP16 is adopted. Implication: most RL+LLM papers using BF16 may have methodology issues.
Key Claims
- FP16 outperforms BF16 for RL training — smaller training-inference distribution gap
- When FP16 is adopted, differences between RL algorithms become negligible
- Shocking implication: ~5,000 RL+LLM papers using BF16 in 2025 may have methodology issues
- This is a systematic confound in the RL+LLM literature
Concepts
- RL Infrastructure — numerical precision as critical implementation detail
- Reward Hacking — confounded baselines may explain some performance gaps