Context Rot: How Increasing Input Tokens Impacts LLM Performance
Source: https://trychroma.com/blog/context-rot Author: Chroma Date: 2025-07-15
Summary
Chroma’s empirical study on how model performance degrades as context window length increases. Introduces the term “context rot” for this degradation. Quantifies the effect across different models and task types.
Key Claims
- Context rot is real: performance drops non-linearly as context length increases, even when the relevant information is present.
- “Lost in the middle” effect: models attend best to the beginning and end of context; middle content is systematically underweighted.
- Magnitude: 10-30% performance drop on retrieval tasks when context extends from 4K to 64K tokens, depending on the model.
- Mitigation: (1) put important content at the start/end, (2) use structured formats (XML/JSON) to help models locate key info, (3) chunk and summarize aggressively.
- Not a solved problem: even frontier models with 128K+ context windows show context rot.
- Implication for RAG: larger context is not always better — precision matters more than recall.
Connection to Other Sources
Foundational empirical evidence for the context engineering advice in Anthropic’s context engineering post and Manus’s lessons.
Concepts
- Context Engineering — context rot is the core failure mode this discipline addresses
- Agent Memory — long-term memory is needed precisely because context rot makes long contexts unreliable