AI Alignment
Overview
The challenge of making AI systems behave in accordance with human values and intentions. Encompasses safety, character training, constitutional AI, RLHF, and value alignment.
Key Insight from Sources
Anthropic frames character training as a core alignment intervention — an AI with good character traits (curiosity, honesty, open-mindedness) handles novel situations better than one trained purely on harm avoidance rules. Source