Representation Engineering Mistral-7B: An Acid Trip
Source: https://vgel.me/posts/representation-engineering/ Author: vgel.me Date: 2025-09-24
Summary
Hands-on exploration of representation engineering — using activation steering to manipulate what a model “thinks about” during inference. The author adds “acid trip”-style noise to Mistral-7B’s activations and observes the effects.
Key Claims
- Representation engineering: instead of prompting a model to behave differently, directly modify activations (vectors inside the model) to shift behavior.
- The technique: identify the “representation” of a concept (e.g., “danger”), create a steering vector, add it to activations at inference time.
- Striking result: adding certain activation patterns causes coherent behavior changes — models talk about unrelated things in ways consistent with the steered concept.
- The language-independent finding: steering vectors often transfer across languages — suggesting truly abstract representations.
- Practical implications for alignment: if you can steer away from “harm” representations directly, this could be a faster alignment technique than RLHF.
Connection to Other Sources
Practical demonstration of concepts in Anthropic’s emotion vectors paper. Both show that internal representations are causally connected to behavior.
Concepts
- Mechanistic Interpretability — representation engineering as interpretation-via-intervention
- AI Alignment — direct activation steering as potential alignment technique
- AI Character & Personality — personality traits as steerable representations