The Wall Between Shannon and Kolmogorov
Author: Vishal Misra
Date: 2026-03-24
Summary
Explores the fundamental divide between Shannon information theory (statistical, compressible) and Kolmogorov complexity (algorithmic, incompressible). Implications for understanding what LLMs actually learn.