SubQ: First Fully Sub-Quadratic LLM
Author: @subquadratic + @alex_whedon | Date: 2026-05-05
SubQ is announced as the first fully sub-quadratic LLM achieving 12M-token context windows at 150 tokens per second. This represents a major breakthrough in efficient long-context processing, bypassing the quadratic attention bottleneck entirely.
Key Claims
- First LLM to achieve fully sub-quadratic complexity (not just approximate)
- Supports 12 million token context length
- Generates at 150 tokens/second despite massive context
- Eliminates the fundamental O(n²) attention scaling barrier
Takeaways
- Long-context architectures are moving beyond approximations to fundamentally different compute patterns
- 12M tokens opens up repository-scale code understanding, book-length reasoning, and multi-document synthesis
- Speed remains practical despite extreme context lengths