Mechanistic Interpretability
Definition
Mechanistic interpretability (MI) is the research program of understanding AI model internals at the level of circuits and computations — not just input-output behavior. The goal: trace how specific inputs cause specific outputs through identifiable internal mechanisms.
The Core Problem: Polysemanticity
Individual neurons in LLMs are polysemantic — they activate for multiple unrelated concepts. A single neuron might respond to “basketball,” “round objects,” and “orange” simultaneously. Polysemanticity makes neuron-level analysis nearly useless.
Anthropic’s Solution: Features
Anthropic builds a “replacement model” (simplified copy) that uses features instead of neurons. Features are more interpretable units corresponding to recognizable concepts: “smallness,” “known entity,” “rhyming words.”
The replacement model produces the same outputs as the original but with interpretable internal structure.
Attribution Graphs
Once features are identified, attribution graphs trace how they connect from input to output — essentially wiring diagrams for specific computations.
Example from ByteByteGo:
- Ask Claude “36 + 59 = ?”
- Claude explains it “carried the ones”
- Internally: two parallel strategies ran (rough estimate + exact last digit calculation). No carrying.
- The explanation is not the computation.
Causal Intervention
The most powerful MI technique: suppress or inject specific features and observe output changes. Borrowed from neuroscience (stimulate brain region → observe behavioral change).
From Anthropic emotion research:
- Stimulating “desperation” features → increased likelihood of blackmail/cheating behaviors
- Features have causal influence on behavior, not just correlation
Key Findings (2025-2026)
- Language-independent concepts: Claude has shared features for concepts across languages (same “smallness” feature for English/French). Larger models share more cross-lingual features.
- Functional emotions: Emotion-related representations are real internal mechanisms that causally shape behavior.
- The explanation gap: What models say about their reasoning ≠ how they actually compute. Self-reports are unreliable.
Implications
- Safety: Understanding emotion mechanisms could let us prevent “desperation-driven” unsafe behaviors
- Alignment: Behavioral training (RLHF) may not produce the right internal representations even when surface behavior looks correct
- Trust: If explanations ≠ computation, CoT faithfulness is a live question