POLARIS: A Post-Training Recipe for Scaling Reinforcement Learning on Advanced Reasoning
Source: https://notion.honerable-payment.ai/polaris (inferred) Author: POLARIS team Date: 2025-09-17
Summary
POLARIS is a post-training recipe that significantly boosts RL-based reasoning by combining several techniques: curriculum learning, reward shaping, and careful hyperparameter tuning. Claims state-of-the-art results on reasoning benchmarks.
Key Claims
- Key components: (1) curriculum learning — start with easy problems, progressively increase difficulty; (2) reward shaping — partial credit for near-correct reasoning; (3) KL constraint tuning — balance exploration vs. staying close to reference policy.
- Curriculum learning insight: training on hard problems from the start causes reward collapse. Curriculum prevents this.
- Partial credit reward: binary rewards (correct/incorrect) waste signal. Partial credit for correct reasoning with wrong answer improves sample efficiency 3x.
- Combined effect: POLARIS achieves significant improvements over baselines on math reasoning benchmarks.
Concepts
- RL Infrastructure — full post-training recipe combining multiple techniques
- Test-Time Compute — reasoning model trained with POLARIS recipe
- Reward Hacking — partial credit rewards reduce gaming incentives