Data Efficiency Through GPU Abuse
Author: @flappyairplanes | Date: 2026-05-06
Thread from a Sequoia event. The thesis: one path to data-efficiency is to “abuse GPUs like they’ve never been abused before” — pushing hardware utilization to extremes enables extracting more learning per token, reducing total data requirements.
Key Claims
- Data efficiency and compute efficiency are linked: better hardware utilization → more learning per sample
- Current training pipelines leave significant GPU utilization on the table
- Extreme kernel optimization can reduce the data needed to reach a given capability level
- Presented at Sequoia, suggesting VC interest in this direction
Takeaways
- The data wall can be attacked from the compute side, not just the data side
- Hardware-software co-design for training is becoming a key differentiator
- “Abusing GPUs” = maximizing FLOP utilization per training step through custom kernels and scheduling