Ilya Sutskever’s Top 30 Reading List
Source: https://aman.ai/primers/ai/top-30-papers/ Author: aman.ai (compiled) Date: 2025-06-23
Summary
Compiled list of papers that Ilya Sutskever (OpenAI co-founder) has cited as foundational for understanding modern deep learning. Includes papers on transformers, scaling, attention mechanisms, and training techniques.
Key Claims
- The papers on the list include: Attention is All You Need, Neural Turing Machines, Pointer Networks, AlexNet, WaveNet, GPT-2, AlphaGo, and several RL papers.
- Ilya reportedly said: “If you really understand these papers, you’ll understand 90% of what’s important in deep learning.”
- Common theme: the papers are all about general mechanisms, not domain-specific solutions — consistent with the bitter lesson.
- The list is weighted toward: sequence modeling, attention, generative models, and RL — not discriminative classification.
Entities
- Ilya Sutskever — OpenAI co-founder; SSI co-founder
Concepts
- Scaling & Compute — several papers are about scaling
- Test-Time Compute — attention and reasoning papers are represented
- RL Infrastructure — RL papers prominent on the list