How Attention Residuals Work

Author: Jia-Bin Huang
Date: 2026-03-24

Summary

Visual explanation of attention residual connections in transformers — how information flows through the residual stream.

Concepts

• Mechanistic Interpretability

See Also

• tweet-transformer-circuits-mental-model — transformer circuits
• attention-wrong-norm — attention normalization issues