Karpathy: 1B Parameters on Clean Data = Today’s 1.5T Models
Author: @aakashgupta (relaying Karpathy on Dwarkesh podcast)
Date: 2026-04-22
Summary
Karpathy claims a 1B parameter model trained on sufficiently clean data could match the intelligence of today’s 1.5T parameter models. Data quality >> parameter count.
Key Insight
Extreme data quality can compensate for 1000x fewer parameters. The scaling law isn’t just about compute — it’s about signal-to-noise in training data.
Concepts
See Also
- karpathy-recipe-training-neural-networks — Karpathy’s training recipe
- chinchilla-compute-optimal-training — compute-optimal training