RL 是新的 Fine-Tuning
Source: https://mp.weixin.qq.com/s/… Author: Ashley, Haozhen Date: 2025-10-24
Summary
Chinese-language post arguing that RL is replacing supervised fine-tuning as the primary post-training technique. Surveys the Chinese AI community’s perspective on the shift from SFT to RL-dominated post-training pipelines.
Key Claims
- The shift: in 2023-2024, SFT was the primary alignment technique. In 2025, RL (GRPO, DPO, PPO variants) is the primary lever.
- Why RL won: SFT teaches models to imitate; RL teaches models to improve. For reasoning and agent tasks, improvement > imitation.
- Chinese lab perspective: DeepSeek’s RL work demonstrated the approach at scale; Chinese labs moved fast to adopt.
- Practical observation: RL-trained models are less “safe-sounding” but more genuinely capable — they optimize for correctness, not appearance.
- The counter-argument: RL is data-hungry and compute-intensive; SFT remains cost-effective for most practitioners.
Concepts
- RL Infrastructure — RL as the new SFT; transition in post-training practices
- Synthetic Data — RL requires rollout data, which is synthetic
- Test-Time Compute — RL produces models that benefit from test-time thinking