openrubrics-reward-modeling

OpenRubrics: Scalable Synthetic Rubric Generation for Reward Modeling

Authors: Tianci Liu, Ran Xu, Tony Yu, Ilgee Hong, Carl Yang, Tuo Zhao, Haoyu Wang
Date: 2026-04-22

Summary

Addresses the scalability challenge of rubric-based reward modeling by generating rubrics synthetically. Combines with LLM alignment approaches.

Key Claims

• Synthetic rubric generation can scale reward modeling without human annotation
• Rubrics provide more structured feedback than binary preference pairs
• Bridges DPO-style alignment with rubric-based RL

Concepts

• Reward Hacking
• Synthetic Data
• AI Alignment

See Also

• rubrics-as-rewards-paper — companion paper
• dpo-reward-model — DPO approach to alignment