ThunderKittens 2.0: Even Faster Kernels for Your GPUs
Source: https://hazyresearch.stanford.edu/blog/2026-02-19-tk-2 Author: Stuart Sul, Chris Ré (HazyResearch) Date: 2026-02-24
Summary
HazyResearch’s release of ThunderKittens 2.0 — an improved GPU kernel framework that makes writing fast attention kernels significantly more accessible. Key advance: better performance primitives and improved hardware support.
Key Claims
- ThunderKittens: a framework for writing fast CUDA kernels in a higher-level language, targeted specifically at attention operations.
- TK 2.0 improvements: new tile abstractions, better warp-level primitives, H100-specific optimizations.
- Target audience: researchers who need fast attention but don’t want to write raw CUDA.
- Performance: achieves within 10-15% of hand-tuned FlashAttention in most attention patterns.
- New attention patterns supported: linear attention, sliding window attention, state-space model attention.
Concepts
- RL Infrastructure — fast attention enables larger context and more efficient RL rollouts
- Scaling & Compute — hardware efficiency extends effective compute