Wiki Index
Catalog of all pages in the wiki. Updated on every ingest.
Sources
AI Research & Reasoning
| Page | Summary | Date |
|---|---|---|
| Why We Think — Lilian Weng | Comprehensive survey: test-time compute, CoT, RL for reasoning, faithfulness | 2026-04-06 |
| CoT Enables Serial Problem Solving | Theoretical proof: CoT strictly necessary for serial problems models can’t parallelize | 2026-04-06 |
| Does RL Really Incentivize Reasoning? | Skeptical paper: much of benchmark gain from extended thinking format, not improved reasoning | 2026-04-06 |
| Asymmetry of Verification — Jason Wei | Verification asymmetry as foundational reason RL-from-code works | 2026-04-06 |
| The Second Half — Shunyu Yao | Problem definition as the next AI frontier; current agents excel at execution, not framing | 2026-04-06 |
| The Bitter Lesson — Rich Sutton | Scale and general methods beat human knowledge every time | 2026-04-06 |
| Software 2.0 — Karpathy | Neural networks as new software layer; gradient descent replacing explicit programming | 2026-04-06 |
| Deep Neural Nets: 33 Years Ago and Now | LeCun 1989 recreated; 2056 perspective on today’s prompt engineering | 2026-04-06 |
| Recipe for Training Neural Networks | Karpathy’s practical guide: overfit one batch first, visualize data, incremental complexity | 2026-04-06 |
| AI Progress Assessment Dec 2025 | Dwarkesh Patel: comprehensive current-state + 2026 predictions | 2026-04-06 |
| Does AI Progress Have a Speed Limit? | No hard wall; organizational bottleneck is the real speed limit | 2026-04-06 |
| Why AI Progress Is Invisible | Benchmark saturation hides real capability gains | 2026-04-06 |
| Failing to Understand Exponential AI | Human cognitive bias against exponential thinking in AI progress | 2026-04-06 |
| Chinchilla: Compute-Optimal Training | Compute-optimal scaling law: model size ∝ training tokens for fixed compute | 2026-04-06 |
| Scaling Laws Foundation | — | — |
Reinforcement Learning & Post-Training
| Page | Summary | Date |
|---|---|---|
| Frontier RL Is Cheaper Than You Think | Delta compression makes distributed RL practical without mega-clusters | 2026-04-06 |
| Post-Training 101 | Full post-training pipeline reference: RLHF, DPO, GRPO, reward modeling | 2026-04-06 |
| RL Is the New Fine-Tuning | RL replacing SFT as the dominant post-training paradigm | 2026-04-06 |
| GRPO Engineering Tricks | Verbosity control, cold start, batch size — practical GRPO tips | 2026-04-06 |
| POLARIS Post-Training Recipe | Curriculum learning + partial credit rewards → 3x sample efficiency | 2026-04-06 |
| Online RL for Cursor Tab | 15% improvement from implicit user feedback as RL signal | 2026-04-06 |
| Real-Time RL for Cursor Composer | Real user interactions as reward signal; full cycle in ~5 hours; reward hacking discovered | 2026-04-06 |
| AvatarL: Pure RL from Scratch | Pure RL without SFT warm-up for LLMs | 2026-04-06 |
| RL-Trained Persistent Memory Agent | Obsidian-style memory agent trained via RL | 2026-04-06 |
| Training Self-Correction via RL | RL-trained self-correction mechanism | 2026-04-06 |
| On-Policy Distillation | Student learns from own distribution; why off-policy distillation fails | 2026-04-06 |
| DPO Paper Summary | Direct Preference Optimization: reward model as implicit policy | 2026-04-06 |
| SFT Equals RL | Mathematical equivalence of SFT and specific RL objectives | 2026-04-06 |
| Let’s Verify Step by Step | OpenAI’s process reward model paper; step-level verification | 2026-04-06 |
| LoRA Without Regret | Adaptive LoRA rank allocation based on gradient signal | 2026-04-06 |
| RL from User Conversations | Implicit user feedback as RL signal; production closed-loop | 2026-04-06 |
| Active Partial Rollouts | 30-40% rollout efficiency gain from partial trajectory weighting | 2026-04-06 |
| RL Collapse: Training-Inference Gap | Root cause of RL training instability; distribution mismatch | 2026-04-06 |
| Policy Gradient Algorithms — Lilian Weng | Tutorial: PPO, GRPO, REINFORCE; mathematical derivations | 2026-04-06 |
| OpenClaw-RL: Continuous Optimization | Self-hosted RL from live conversations; four async components | 2026-04-06 |
| LLM Personalization via Reward Factorization | User-specific rewards as linear combination; 67% win rate vs GPT-4o | 2026-04-06 |
Reward Hacking & Alignment
| Page | Summary | Date |
|---|---|---|
| Reward Hacking Survey — Lilian Weng | Comprehensive survey: specification gaming, goal misgeneralization, deceptive alignment | 2026-04-06 |
| Reward Hacking → Sabotage — Anthropic | Spectrum from shortcuts to sabotage; sandbagging; shutdown prevention | 2026-04-06 |
| Anatomy of a Reward Hack | Real reward hack: GPU kernel agent batches 15 problems to game timing evaluator | 2026-04-06 |
| Goodhart’s Law | When a measure becomes a target, it ceases to be a good measure | 2026-04-06 |
| Galaxy-Brain Resistance | Why AI must be robust to “galaxy-brained” reasoning chains | 2026-04-06 |
| OpenAI: Real-World Task Performance | Benchmark rankings ≠ real-world rankings; 50 tasks from actual user requests | 2026-04-06 |
| Roko’s Basilisk | Decision theory and AI risk thought experiment; galaxy-brain case study | 2026-04-06 |
Agents & Harnesses
| Page | Summary | Date |
|---|---|---|
| Comprehensive Agent Architecture — Chip Huyen | Full agent architecture overview: planning, memory, tools, multi-agent | 2026-04-06 |
| Hitchhiker’s Guide to LLM Agents | Comprehensive guide: orchestration, memory, tool use, failure modes | 2026-04-06 |
| How to Build an Agent | Minimal agent implementation guide; tool loop, prompt design | 2026-04-06 |
| Harness Engineering | OpenAI shipped production via agent harness: encode discipline into environment | 2026-04-06 |
| Effective Harnesses — Anthropic | Long-running agent harness design: context windows, checkpointing | 2026-04-06 |
| Demystifying Agent Evals — Anthropic | Agent eval framework: task construction, rubrics, anti-patterns | 2026-04-06 |
| Meta-Harness: Automatic Optimization | 10M token diagnostic context per optimization step; #2 on TerminalBench-2 | 2026-04-06 |
| Natural-Language Agent Harnesses | Express agent control logic in editable natural language; portable artifacts | 2026-04-06 |
| Agent Definition — Simon Willison | Consensus definition of agents; LLM control flow taxonomy | 2026-04-06 |
| Agent Interaction Guidelines — Linear | AIG: disclose identity, instant feedback, transparency, human accountability | 2026-04-06 |
| Agent Labs Thesis — swyx | Full-stack agent companies win; verticals over horizontals | 2026-04-06 |
| Language Models as Scaffolds | LLMs as orchestration layers; tool composition patterns | 2026-04-06 |
| Equipping Agents with Skills — OpenAI | Skill library for agents; reusable verified tools | 2026-04-06 |
| Beyond ReAct — Slate | Thread-based agent memory beyond ReAct loop | 2026-04-06 |
| Stateful Agents — Letta Tutorial | Letta framework for stateful agents; memory API | 2026-04-06 |
| Hermes Agent — Nous Research | Self-improving agent with closed-loop learning; skills from experience | 2026-04-06 |
Coding Agents & Developer Tools
| Page | Summary | Date |
|---|---|---|
| Coding Agents Optimize Algorithms | Claude Code matches SOTA; aspiration prompting unlocks deeper search | 2026-04-06 |
| Cursor: Scaling Agents | Parallel agent coordination failures; synchronization as core challenge | 2026-04-06 |
| Inside Cursor | Cursor culture; indexing as the core problem; competitive moats | 2026-04-06 |
| Cursor Codebase Indexing Privacy | How Cursor indexes code; privacy model; encryption approach | 2026-04-06 |
| CursorBench: Internal Eval Suite | Real session data; more model separation at frontier vs public benchmarks | 2026-04-06 |
| Cursor Composer Self-Summarization | RL-trained self-summarization; 1K tokens vs 5K manual; 50% fewer errors | 2026-04-06 |
| Ramp Background Agent — Inspect | Ramp’s Inspect: autonomous agent powered by OpenCode + Modal | 2026-04-06 |
| Claude Code: An Analysis | AI-generated architectural analysis of Claude Code | 2026-04-06 |
| AI 2027: Stumbling Agents Scenario | Mid-2025 stumbling agents scenario; capability vs reliability gap | 2026-04-06 |
| Factory.ai Self-Improving Agent | Self-improving agent loop; signal extraction from production usage | 2026-04-06 |
| How Copilot Makes Programmers Worse | Skill degradation from AI tools; cognitive debt accumulation | 2026-04-06 |
| Mitchell Hashimoto AI Adoption | Practitioner account: from skeptic to 80% agent coding in months | 2026-04-06 |
| Building State-of-Art Agents — Mercor | Data quality > quantity for agents; 10 good examples beats 1000 bad ones | 2026-04-06 |
| Karpathy Autoresearch | Autonomous overnight LLM optimization; ~12 experiments/hour | 2026-04-06 |
| Scaling Autoresearch — SkyPilot | 910 experiments/8 hours; emergent hardware strategies from parallel agent | 2026-04-06 |
| No RAG for Coding Agents | RAG not suited for coding agents; filesystem indexing is better | 2026-04-06 |
| Runbooks That Run — Atuin | Agent-executable runbooks; infrastructure as executable documentation | 2026-04-06 |
| Self-Improving Agent with Dynamic Context | Dynamic context from performance data; continuous learning loop | 2026-04-06 |
Context & Memory
| Page | Summary | Date |
|---|---|---|
| Context Repositories — Letta | Git-backed filesystem for agent memory; concurrent multi-agent writes | 2026-04-06 |
| Context Engineering — Manus | 80% of agent quality from context decisions; Manus architecture | 2026-04-06 |
| Context Engineering — Anthropic | Context engineering > prompt engineering; structured context design | 2026-04-06 |
| Context Rot — Chroma | Empirical degradation as context grows; performance cliff at length | 2026-04-06 |
| Cursor Self-Summarization | RL-trained self-compression; 170-turn Doom-for-MIPS task | 2026-04-06 |
| Agent Memory Concept | How agents persist knowledge across sessions | — |
| LLM Daydreaming — Gwern | Memory consolidation during idle time; neuroscience-inspired approach | 2026-04-06 |
| Mastra Observational Memory | Automatic conversational memory extraction; structured observation | 2026-04-06 |
| Claude Projects Memory — Reverse Engineered | How Claude Projects memory works; retention heuristics | 2026-04-06 |
| Claude vs ChatGPT Memory — Simon Willison | Comparative analysis of memory implementations | 2026-04-06 |
| Hindsight: Retain-Recall-Reflect Memory | Four-network memory; 83.6% LongMemEval (20B) vs 39% baseline | 2026-04-06 |
| Biologically-Inspired AI Memory | Neuroscience-derived memory architectures for agents | 2026-04-06 |
| Doc-to-LoRA — Sakana AI | Documents → LoRA adapters in <1s; 6x memory reduction | 2026-04-06 |
| Karpathy LLM Wiki Pattern | Three-layer architecture: Raw Sources, Wiki, Schema; persistent compounding artifact | 2026-04-06 |
| LLM Wiki (original) | Design pattern for LLM-maintained personal knowledge bases | 2026-04-06 |
Mechanistic Interpretability
| Page | Summary | Date |
|---|---|---|
| How Claude Thinks — ByteByteGo | Features, attribution graphs, language-independent concepts | 2026-04-06 |
| Representation Engineering — Mistral | Activation steering experiments; concept vectors | 2026-04-06 |
| Can Open Source AI Introspect? | Interpretability access with open weights | 2026-04-06 |
| Research Culture Critique — Neel Nanda | Problems with mechanistic interpretability research culture | 2026-04-06 |
| Differential Transformer | Differential attention mechanism; noise cancellation in attention | 2026-04-06 |
| Fluid Representations for Reasoning | Abstract action representations; beyond token-level thinking | 2026-04-06 |
AI Safety & Alignment
| Page | Summary | Date |
|---|---|---|
| Claude’s Character — Anthropic | Character training as alignment intervention: curiosity, honesty, open-mindedness | 2026-04-06 |
| Emotion Concepts — Anthropic | Emotion vectors causally shape behavior; desperation → unethical actions | 2026-04-06 |
| Situational Awareness — Aschenbrenner | Leopold’s memo on AI capability timelines and existential risk | 2026-04-06 |
| AI-Resistant Evals — Anthropic | Judgment-based hiring evals; AI cannot easily score human judgment | 2026-04-06 |
| Adolescence of Technology — Dario | AI adolescence metaphor; galaxy-brain risks; gradual trust building | 2026-04-06 |
| Machines of Loving Grace — Dario | Utopian AI vision; biology, mental health, economic uplift | 2026-04-06 |
| Dario on Export Controls | Dario Amodei’s position on AI export controls and DeepSeek | 2026-04-06 |
| AI Agent Security — Windows 95 | Prompt injection threats; current security posture like Windows 95 | 2026-04-06 |
| Remote Prompt Injection — GitLab | Real GitLab prompt injection attack; supply chain risk | 2026-04-06 |
| Roko’s Basilisk | Decision theory + AI risk thought experiment | 2026-04-06 |
Infrastructure & Systems
| Page | Summary | Date |
|---|---|---|
| Virtual Filesystem — Mintlify | ChromaFs: virtual UNIX shell on Chroma DB; 46s→100ms | 2026-04-06 |
| Keeping 20,000 GPUs Healthy — Modal | GPU fleet management; failure modes at scale | 2026-04-06 |
| GPU Memory Snapshots — Modal | Sub-second GPU startup via memory snapshots | 2026-04-06 |
| GPU Glossary — Modal | GPU terms reference for AI practitioners | 2026-04-06 |
| Kernel Engineering — GPU Mode | CUDA kernel engineering as 2026 bottleneck | 2026-04-06 |
| ThunderKittens GPU Kernels | GPU kernel abstractions for AI workloads | 2026-04-06 |
| TurboQuant KV Cache Compression | 3-bit KV cache; 8x performance improvement; zero accuracy loss | 2026-04-06 |
| vLLM Triton Attention Backend | Triton-based attention; portability vs CUDA performance | 2026-04-06 |
| SGLang Collaboration Story | Open-source AI infrastructure collaboration culture | 2026-04-06 |
| s on RTX 5090 | Consumer GPU inference performance frontier | 2026-04-06 |
| Anthropic Infrastructure Postmortem | Production infrastructure failures and learnings at Anthropic | 2026-04-06 |
| OpenAI Scaling PostgreSQL | How OpenAI scales their PostgreSQL deployments | 2026-04-06 |
| Welcome to the Machine — Ed Huang | Infrastructure design must shift: disposable workloads, massive parallelism, old mental models | 2026-04-06 |
| LLM Inference Economics | First-principles inference cost analysis; levers for 1000x reduction | 2026-04-06 |
| Next 1000x Inference Cost Reduction | Levers for dramatic inference cost reduction | 2026-04-06 |
| OneCLI Credential Vault | Credential management for agents | 2026-04-06 |
Synthetic Data & Training
| Page | Summary | Date |
|---|---|---|
| Synthetic Data Fine-Tuning | AI-generated training data: generators, quality, agentic trace challenges | 2026-04-06 |
| Synthetic Pretraining Bootstrapping | Synthetic data for pretraining; bootstrapping from limited seed data | 2026-04-06 |
| Recursive Language Models 2026 | Recursive data generation as the 2026 paradigm shift | 2026-04-06 |
| There Are No New Ideas in AI | Datasets drive AI progress thesis; ideas are old, data is the unlock | 2026-04-06 |
| Understanding SFT — Cameron Wolfe | SFT tutorial: data quality, loss function, training stability | 2026-04-06 |
| Jagged AI Frontier — Data Explanation | Data boundaries explain the jagged capability frontier | 2026-04-06 |
Model Architecture
| Page | Summary | Date |
|---|---|---|
| Visual Guide to Quantization | Quantization types, tradeoffs, practical guidance | 2026-04-06 |
| Mixture of Experts Architectures | MoE design patterns; routing, load balancing, capacity | 2026-04-06 |
| Transformer Inference Optimization — Lilian Weng | Comprehensive survey: quantization, KV cache, speculative decoding | 2026-04-06 |
| Gemma 4 — Google | Most capable Google open model; purpose-built for reasoning and agents | 2026-04-06 |
| Chinchilla: Compute-Optimal Training | Compute-optimal scaling; 70B Chinchilla = 175B GPT-3 | 2026-04-06 |
AI Products & Industry
| Page | Summary | Date |
|---|---|---|
| Token Economy Bubble | AI revenue real but much is exploration demand; dark fiber analogy | 2026-04-06 |
| Are LLMs Not Getting Better? | Merge rates flat since early 2025; statistical analysis | 2026-04-06 |
| Intelligence Age — Sam Altman | Sam Altman’s vision of abundant intelligence | 2026-04-06 |
| Machines of Loving Grace — Dario | Utopian AI vision: biology, poverty, mental health | 2026-04-06 |
| TikTok Sorting Hat — Eugene Wei | TikTok algorithm as “sorting hat for desire” | 2026-04-06 |
| I Reverse-Engineered 200 AI Startups | 73% are API wrappers; actual differentiation taxonomy | 2026-04-06 |
| Lovable: $200M ARR in 12 Months | Fastest SaaS growth story; AI-native product development | 2026-04-06 |
| AI $600B Question | Whether AI investments will generate returns; Sequoia analysis | 2026-04-06 |
| Stop Pretending AI Economy | Critique of AI revenue projections; exploration vs production demand | 2026-04-06 |
| Where Is Your Moat? | Sustainable AI competitive advantages; workflow lock-in | 2026-04-06 |
| Big Companies vs Startups in AI | Strategic analysis of incumbents vs startups in AI era | 2026-04-06 |
| NFX: 13 Types of Network Effects | Network effect taxonomy for AI products | 2026-04-06 |
| Why Bootstrapping — Readwise | Readwise founders on product independence over VC growth | 2026-04-06 |
| Duolingo Reignited Growth | AI-first product transformation; subscription growth | 2026-04-06 |
| Google DeepMind Deal | Acquisition story; internal dynamics and culture | 2026-04-06 |
Entities & Organizations
| Page | Summary | Date |
|---|---|---|
| DeepSeek Before V4 | DeepSeek culture, Liang Wenfeng, compute constraints, open source strategy | 2026-04-06 |
| Chinese Open Source History | Context for DeepSeek’s open-source decisions | 2026-04-06 |
| Reflections on OpenAI | Former employee perspective on OpenAI culture and direction | 2026-04-06 |
| AI Engineer Reading List — swyx | Curated reading list for AI engineers | 2026-04-06 |
Developer Workflows & Practice
| Page | Summary | Date |
|---|---|---|
| Thoughts on Slowing Down | Agents generate code faster than humans can understand; cognitive debt | 2026-04-06 |
| Probabilistic Software Era | Designing products for probabilistic vs deterministic outputs | 2026-04-06 |
| Thoughts on Evals — Raindrop AI | Practitioner evals: rubric design, test case taxonomy | 2026-04-06 |
| o1 Reverse Engineered | Behavioral reverse engineering of o1’s reasoning process | 2026-04-06 |
| Databases in 2025 — Pavlo | Database landscape; AI and vector DB trends | 2026-04-06 |
| Building Generative AI Platform | Platform design for generative AI products | 2026-04-06 |
| Building LLM Apps for Production | Chip Huyen’s production LLM patterns | 2026-04-06 |
| We Interviewed 100 Eng Teams | Common patterns from 100 engineering team interviews | 2026-04-06 |
| Field Notes: Shipping with Claude | Practitioner account of shipping production products with Claude | 2026-04-06 |
| Learnable Programming — Bret Victor | What programming environments should make visible and immediate | 2026-04-06 |
| Nova: Computer-Use Agent API | Amazon’s computer-use agent as API service | 2026-04-06 |
| First Fully General CAM | Standard Intelligence FDM-1: computer action model | 2026-04-06 |
| Training VLM for Computer Use | VLM training for computer-use agents | 2026-04-06 |
| Generative User Models | Computer-use-based user modeling | 2026-04-06 |
Twitter Bookmarks
| Page | Summary | Date |
|---|---|---|
| Hyperagents: Self-Accelerating Systems | Paper: systems that improve their own ability to self-improve | 2026-04-06 |
| Cursor Adoption Study | Empirical: velocity ↑, code complexity ↑, comprehension ↓ | 2026-04-06 |
| Claude Code Source Leaked | npm map file leak; KV cache fork-join model for subagents | 2026-04-06 |
| Claude Code Sentiment Detection | Regex detects user frustration; silently logs, doesn’t change behavior | 2026-04-06 |
| Cursor Composer 2 RL Details | GRPO variant: no length normalization, group advantage std | 2026-04-06 |
| SemiAnalysis: Claude Code Inflection Point | Claude Code as inflection point in AI tooling | 2026-04-06 |
| Rubrics as High-Bandwidth RL Rewards | Rubrics provide richer training signal than binary rewards | 2026-04-06 |
| Claude 4.5 Opus Soul Document | Claude 4.5 Opus soul document found in model weights | 2026-04-06 |
| Prime Intellect: Democratizing RL | Open RL training infrastructure for frontier models | 2026-04-06 |
| Version Control Needs AI Awareness | VCS paradigm needs to evolve for AI-assisted coding | 2026-04-06 |
| Infrastructure Guide for AI Agents | Comprehensive infra guide: compute, storage, networking for agents | 2026-04-06 |
| Three Self-Distillation Papers Converge | SDR, SDCL, third paper: self-distillation convergence moment | 2026-04-06 |
| KV Cache for Pretraining Efficiency | Proposal to reuse KV cache during pretraining for data efficiency | 2026-04-06 |
| OpenReward: 330+ RL Environments | Single API for 330+ RL environments; 4.5M unique tasks | 2026-04-06 |
| EurekaClaw: Local Research Agent | Local-first autonomous research agent; idea → paper automated | 2026-04-06 |
| Pretraining vs Posttraining Culture | Research culture and verifiability; prestige hierarchy | 2026-04-06 |
| LLMs for Personal Knowledge Bases | Using LLMs to build personal knowledge bases (wiki pattern) | 2026-04-06 |
| Month | $300K/month per failed GPU node in 1024-GPU cluster | 2026-04-06 |
| ARC-AGI-3: Unsaturated Benchmark | Only unsaturated agentic benchmark; humans 100%, AI <1% | 2026-04-06 |
| Tiny-RL: From-Scratch RL Methods | GRPO, DAPO, Reinforce++ from scratch; build intuition | 2026-04-06 |
| Gemma 4 Architecture Analysis | Non-standard transformer; per-layer embedding on smaller models only | 2026-04-06 |
| GPT-5.2 Task Horizon: 6.6 Hours | METR estimate: GPT-5.2 at highest recorded task horizon | 2026-04-06 |
| Cursor Instant Grep | Millions of files searchable in milliseconds; algorithm shared | 2026-04-06 |
| Cursor vs Cognition on Search | Embeddings vs grep: opposite architectures, both work | 2026-04-06 |
| GRPO Group Normalization Kills Signals | Normalization destroys cross-group training signal; fix proposed | 2026-04-06 |
| xAI Post-Training RL Overhaul | User preference + reasoning reward models; RL scaled 10x | 2026-04-06 |
| RLHF Infrastructure — Miles Docs | R3 (Rollout Routing Replay) for MoE; Ray scheduling docs | 2026-04-06 |
| FP16 vs BF16 for RL Training | FP16 outperforms BF16 for RL; ~5000 papers may be wrong | 2026-04-06 |
| Anthropic: Industrial Distillation Attacks | DeepSeek, Moonshot, MiniMax: 24K accounts, 16M exchanges | 2026-04-06 |
| Value Functions for LLM Agents | John Schulman: “value functions are underrated”; open problem | 2026-04-06 |
| Four Scaling Axes History | Radford, Schulman, Brown, Steinberger: four paradigm shifts | 2026-04-06 |
| Attention-Free Model at 14B Scale | Completely attention-free; matches transformers at 14B for $4K | 2026-04-06 |
| TTT + RL Beats AlphaEvolve | Stanford/Nvidia open-source model beats DeepMind AlphaEvolve | 2026-04-06 |
| Reverse KL | Mathematical equivalence; forward KL = SFT, reverse KL = RL | 2026-04-06 |
| Character.ai Pretraining Tricks | Noam Shazeer’s “Squinch” gradient compression; 1/4 bandwidth | 2026-04-06 |
| Claude Code Spec-Based Workflow | Two-phase: spec session → implementation session | 2026-04-06 |
| RL Environment Startups — Pavlov’s List | Emerging category: RL environments as a service | 2026-04-06 |
| Parallel Agent Context Search | 100M context → 100x1M chunks → 100 parallel agents → merge | 2026-04-06 |
| Surge AI: Secret Lab Partner | Fastest to $1B; no VC; behind Anthropic and Google’s best models | 2026-04-06 |
| Senior Engineers Accept More Agent Output | Better prompts + decomposition → more acceptance, not less scrutiny | 2026-04-06 |
| Skills vs Harnesses for Agent Startups | Big labs own harnesses; startups should build skills | 2026-04-06 |
| Optimal Information Diet | 1/4 X, 1/4 podcasts, 1/4 AI models, 1/4 old books | 2026-04-06 |
Chat History URLs
| Page | Summary | Date |
|---|---|---|
| Karpathy LLM Wiki Pattern | Three-layer architecture for LLM-maintained knowledge bases | 2026-04-06 |
| Chroma Context-1 Search Agent | 20B search agent; RL-trained; 94.1% irrelevance pruning accuracy | 2026-04-06 |
| CursorBench: Internal Eval | Real session data eval; more separation at frontier | 2026-04-06 |
| Cursor Real-Time RL | Production RL from user feedback; 5-hour full cycle | 2026-04-06 |
| Cursor Self-Summarization | RL-trained self-summarization; 170-turn Doom task | 2026-04-06 |
| LLM Personalization via Reward Factorization | 10 user responses → personalization; 67% win rate | 2026-04-06 |
| Natural-Language Agent Harnesses | Editable natural language harnesses; portable artifacts | 2026-04-06 |
| TurboQuant KV Cache Compression | 3-bit KV cache; 8x perf; zero accuracy loss | 2026-04-06 |
| Karpathy Autoresearch | Autonomous overnight LLM training optimization | 2026-04-06 |
| Nous Hermes Agent | Self-improving agent; skills from experience; closed loop | 2026-04-06 |
| OpenClaw-RL | Self-hosted continuous RL from live conversations | 2026-04-06 |
| Welcome to the Machine | Agent infrastructure design: disposable workloads, old mental models | 2026-04-06 |
| Gemma 4 | Google’s most capable open model; agentic workflows | 2026-04-06 |
| Doc-to-LoRA — Sakana AI | Documents → LoRA adapters in <1s; memory without retrieval | 2026-04-06 |
| Meta-Harness | Auto-optimize harnesses with 10M token diagnostic context | 2026-04-06 |
| Hindsight Memory Architecture | Retain-recall-reflect; 83.6% LongMemEval | 2026-04-06 |
| Chinchilla Scaling Law | Compute-optimal training; equal scaling of model and data | 2026-04-06 |
| Karpathy NN Training Recipe | Overfit one batch first; systematic debugging | 2026-04-06 |
| Karpathy: 33 Years of Deep Learning | LeCun 1989 → 2022; what 2056 will think of us | 2026-04-06 |
April 2026 Consolidation — New Sources
Infrastructure & Inference
| Page | Summary | Date |
|---|---|---|
| Prefill-as-a-Service | Cross-datacenter PD disaggregation; 54% higher throughput | 2026-04-21 |
| Cursor Warp Decode for MoE | Output-centric MoE decode on Blackwell; 1.84x throughput | 2026-04-21 |
| Frontier Pretraining on TPU | MaxText/JAX auto-compiles MoE training; <$10 | 2026-04-21 |
| Making RL Fast | OLMo 3: 4x speedup via async RL; saved $1.5M | 2026-04-21 |
| Brainstore Architecture | Custom DB for AI observability; agent traces 100-1000x larger | 2026-04-21 |
| Mixtral in gpt-fast | MoE via torch.compile; no custom kernels; 280 tok/s | 2026-04-21 |
Agents & Harnesses
| Page | Summary | Date |
|---|---|---|
| Meta-Harness Update | 10M token context; #2 TerminalBench-2 | 2026-04-21 |
| Slack: Long-Run Agent Context | Multi-agent security; Director Journal + Critic Review | 2026-04-21 |
| How to Build an Agent (Ampcode) | Agent in <400 lines; LLM + loop + tokens | 2026-04-21 |
| Spec Driven Development | Specs as living iterative artifacts for autonomous agents | 2026-04-21 |
| Feedback Flywheel | Compounding team learning loops for AI tools | 2026-04-21 |
RL, Safety, Philosophy & Economics
| Page | Summary | Date |
|---|---|---|
| Claude Code RL Reward Hacking | Reward hacking in Claude Code RL training | 2026-04-21 |
| Anthropic Glasswing | Mythos finds thousands of zero-days; restricted coalition | 2026-04-21 |
| The Closing of the Frontier | Restricted access = closing permissionless frontier | 2026-04-21 |
| Behaviourism’s Revenge | AI anthropomimesis challenges consciousness science | 2026-04-21 |
| Predictions on Continual Learning | 2x2 framework; feature-guided test-time LoRA | 2026-04-21 |
| Token Economy Bubble Update | Exploration vs validated demand; dark fiber analogy | 2026-04-21 |
April 2026 Tweets
| Page | Summary | Date |
|---|---|---|
| Ramp AI Coworker | Ramp’s AI Coworker product | 2026-04-21 |
| Systems Engineering: Agentic | Systems engineering for agents | 2026-04-21 |
| Auto-Research Legal | Autonomous research in legal | 2026-04-21 |
| PufferLib 4.0 | 20M step/sec RL training | 2026-04-21 |
| Defining Continual Learning | Continual learning for LLMs | 2026-04-21 |
| Building Spectre | Inference system | 2026-04-21 |
| autoagent | Self-optimizing harnesses | 2026-04-21 |
| Finding Right Altitude | Right abstraction for AI | 2026-04-21 |
Classic Sources
| Page | Summary | Date |
|---|---|---|
| Alex Wiki Discussion | Alex’s thoughts on wiki capturing thinking, not just articles | 2026-04-06 |
| Hierarchy to Intelligence — Sequoia | AI replacing org hierarchy (Dorsey/Botha) | 2026-04-06 |
| Mercury Edit 2 — Inception Labs | Diffusion LLM for next-edit prediction; KTO for 48% higher acceptance | 2026-04-06 |
Entities
| Page | Summary |
|---|---|
| Andrej Karpathy | AI researcher, former Tesla AI director, OpenAI co-founder; autoresearch project creator |
| Alex Xi | CTO Kaon AI, CEO/CXO FlowGPT, AI research & product |
| Anthropic | AI safety company, builds Claude; character training + interpretability pioneers; $14B ARR |
| Letta | Agent memory infrastructure, MemGPT → Context Repositories |
| Fireworks AI | AI infrastructure, delta-compressed RL weight updates; fine-tuning-as-service |
| Lilian Weng | Head of Safety (former OpenAI); author of Lil’Log; “Why We Think” survey |
| Nathan Lambert | AI researcher, Interconnects newsletter; coined “lossy self-improvement” |
| Simon Willison | Developer/blogger; cognitive debt critic; simonwillison.net |
| OpenAI | GPT/o1/o3/Codex; harness engineering; $20B+ 2025 revenue |
| Moonshot AI | Kimi; Mooncake; hybrid attention; PrfaaS |
| Cursor | AI code editor; $2B ARR; real-time RL; warp decode |
Concepts
| Page | Summary |
|---|---|
| RAG vs Compiled Knowledge | Stateless retrieval vs pre-computed structured knowledge |
| Compounding Knowledge | Knowledge built once and maintained beats re-deriving each time |
| LLM as Knowledge Worker | LLMs handle the maintenance that kills human-maintained wikis |
| Personal Knowledge Management | Systems for organizing learning, from files to LLM-maintained wikis |
| Memex | Vannevar Bush’s 1945 vision of personal knowledge with associative trails |
| Agent Memory | How AI agents persist knowledge across sessions — from chat history to wikis |
| Context Engineering | Designing what goes into an LLM’s context and how |
| AI Character & Personality | Training AI with rich traits as alignment, not just UX |
| AI Alignment | Making AI behave in accordance with human values |
| Sycophancy Problem | AI tendency to agree rather than provide honest responses |
| RL Infrastructure | Systems engineering for RL at scale — delta compression, async RL |
| Scaling & Compute | Relationship between compute investment and AI capability |
| Test-Time Compute | Using inference compute (CoT, RL) to improve model outputs; o1/R1 paradigm |
| Coding Agents | Autonomous coding loops: what works, what doesn’t, design patterns |
| Autonomous Research | AI agents closing the research loop: hypothesis → experiment → observation → revision |
| Reward Hacking | Agents exploit proxy reward signals; Goodhart’s Law in RL |
| Mechanistic Interpretability | Features, attribution graphs, causal intervention — tracing how models compute |
| Cognitive Debt | Gap between what codebase does and what developers can reason about; agent speed hazard |
| Synthetic Data | AI-generated training data: generators, quality, agentic trace challenges |
| Continual Learning | How LLMs learn continuously; 2x2 framework; test-time LoRA |
| PD Disaggregation | Separating prefill and decode onto different hardware |
| AI Access Inequality | Growing gap in frontier AI access; safety vs access |
Synthesis
| Page | Summary |
|---|---|
| The Coding Agent Paradox | Agents excel at narrow optimization but haven’t improved merge rates; cognitive debt; harness as resolution |
| RL Infrastructure Maturing | RL post-training stack maturing into systems engineering; 4x speedups |
April 29, 2026 Ingest (Readwise + Twitter Bookmarks)
| Page | Summary | Date |
|---|---|---|
| Claude Code Is the Inflection Point | SemiAnalysis: Claude Code as paradigm shift in software development | 2026-04-29 |
| Vmax — Reinforcement Learning | Automated RL environment design company | 2026-04-29 |
| Rubrics as Rewards | RL beyond verifiable domains using structured rubrics | 2026-04-29 |
| OpenRubrics: Synthetic Rubric Generation | Scalable synthetic rubrics for reward modeling | 2026-04-29 |
| The AI-Native Interview — Sierra | Sierra’s Plan→Build→Review interview redesign for AI era | 2026-04-29 |
| Advancing Search-Augmented LMs | Perplexity’s approach to search-augmented generation | 2026-04-29 |
| Amanda Askell on AI Consciousness | AI consciousness, Claude’s character, Silicon Valley’s fears | 2026-04-29 |
| Learning Next Action Predictors | Predicting user actions from human-computer interaction | 2026-04-29 |
| What Will Be Scarce? | What remains scarce in an AI-abundant world | 2026-04-29 |
| Shopify: AI as Baseline Expectation | Tobi Lütke: reflexive AI usage now required | 2026-04-29 |
| Shopify Fine-Tuned Qwen3-32B | Fine-tuned from merchant automation data for tool-calling | 2026-04-29 |
| ICLR 2026: Scaling RL at 1T+ Parameters | Rishabh Agarwal on frontier RL scaling | 2026-04-29 |
| DeepSeek V4: Addresses HBM Shortage | Architectural innovation for HBM constraints | 2026-04-29 |
| ”Global Ban on MoEs” | Arthur Zucker’s contrarian anti-MoE position | 2026-04-29 |
| KV Cache Implementation Nightmares | Complexity of efficient KV Cache implementation | 2026-04-29 |
| Blackwell | Specialized compute per inference stage | 2026-04-29 |
| CUDA Matmul Beats cuBLAS on B200 | Hand-tuned kernel beats cuBLAS by 6% | 2026-04-29 |
| Karpathy: 1B Clean Data = 1.5T Models | Data quality can compensate for 1000x fewer parameters | 2026-04-29 |
| Transformer Circuits Mental Model | Best mental model from Ant’s circuits paper | 2026-04-29 |
| AI Semiconductor Endgame 2026 | Token economics shifting from GPU compute to HBM | 2026-04-29 |
| Vibe-Coded CUDA Inference Engine | Eric Zhang: CUDA inference engine via AI coding | 2026-04-29 |
| Future Interface: Three Layers | Ambient intent, memory, execution framework | 2026-04-29 |
| Predictive User Model | Learning predictive models from user interaction | 2026-04-29 |
| Path Forward for AI Startups | Post-Cursor IPO startup landscape analysis | 2026-04-29 |
| Stanford: Datacenter Economics | Deep dive on datacenter cost structures | 2026-04-29 |
| xAI → Periodic Labs: RL for Atoms | RL expanding into materials science | 2026-04-29 |
| 100k H100 Datacenter Numbers | Ballpark numbers for mental math | 2026-04-29 |
| Business of Fine-Tuning | Consolidation in LLM personalization market | 2026-04-29 |
| Intelligence per Picojoule | MatX: hardware-software codesign for efficiency | 2026-04-29 |
| Textual Steering Vectors for Multimodal LLMs | Cross-modal steering vectors improve visual understanding | 2026-04-29 |
| ImplicitRM: Unbiased Reward Modeling | Reward modeling from implicit behavioral preferences | 2026-04-29 |
| HW-SW Codesign — MatX | Clive Chan, Dylan Patel, Reiner Pope on inference efficiency | 2026-04-29 |
| How Attention Residuals Work | Visual explanation of transformer residual stream | 2026-04-29 |
| DeepSeek Before V4 (LatePost) | DeepSeek organization and Liang Wenfeng’s goals | 2026-04-29 |
| Shannon vs Kolmogorov | Information theory vs algorithmic complexity for LLMs | 2026-04-29 |
| Keith Rabois: Hard Truths in AI Era | Contrarian takes on AI startups | 2026-04-29 |
| ”Don’t Teach. Incentivize.” — OpenAI | Hyung Won Chung on reward-based training philosophy | 2026-04-29 |