RAG vs Compiled Knowledge

The Distinction

RAG (Retrieval-Augmented Generation) — retrieve relevant chunks from raw documents at query time, generate answer from scratch. Stateless. No accumulation.

Compiled Knowledge — LLM pre-processes sources into structured, interlinked pages. Cross-references, contradictions, and synthesis are computed once and maintained. Query-time work is minimal.

Arguments for Compiled Knowledge

1. Synthesis across 5+ documents doesn’t need to be re-derived every time (Karpathy)
2. Cross-references are pre-built — the LLM doesn’t need to discover connections at query time
3. Contradictions are flagged proactively, not missed during retrieval
4. Knowledge compounds — each new source enriches the existing structure
5. The maintenance cost is near-zero with LLMs doing the bookkeeping

Arguments for RAG

1. Simpler to implement — no wiki maintenance logic needed
2. Works well for factual lookup (find the right chunk, return it)
3. No risk of wiki going stale or accumulating errors
4. Scales to very large corpora without manual structure

Open Questions

• At what scale does compiled knowledge break down? (Karpathy suggests ~100 sources / hundreds of pages is fine with just an index file)
• Can the two approaches be combined? (Compiled wiki for core knowledge, RAG fallback for long tail)
• How do you handle confidence/provenance when the wiki is many layers removed from raw sources?

Connections

• Compounding Knowledge
• LLM as Knowledge Worker
• Andrej Karpathy