A Visual Guide to Quantization

Source: https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-quantization Author: Maarten Grootendorst Date: 2024-08-10

Summary

Excellent visual explainer on quantization — the technique of reducing model weight precision from float16/32 to int8/4. Key reference for understanding how models are made smaller and faster.

Key Claims

  • Quantization basics: reduce the bit-width of model weights (FP32 → FP16 → INT8 → INT4). Each halving approximately halves memory.
  • Quality vs. compression tradeoff: INT8 loses ~1-2% quality; INT4 loses 3-5%; below INT4 quality degrades rapidly.
  • Post-training quantization (PTQ): quantize after training. Simple but suboptimal.
  • Quantization-aware training (QAT): train with quantization noise, then quantize. Better quality at same compression.
  • Why it matters: a 70B model in FP16 needs 140GB VRAM; in INT4, it needs ~40GB — fits on 2 A100s instead of requiring 4.

Concepts