Character.ai Pretraining Tricks: Squinch Algorithm
Source: https://x.com (thread) Author: unknown (referencing Character.ai / Noam Shazeer) Date: 2025-2026
Summary
Five pretraining tricks from Character.ai before Google acquisition. Key invention: Noam Shazeer’s gradient compression algorithm “Squinch” — maintains SOTA MFU despite running on GCP H100-TCPX (1/4 of InfiniBand bandwidth).
Key Claims
- Character.ai ran pretraining on GCP H100-TCPX (1/4 bandwidth of InfiniBand)
- Noam Shazeer invented “Squinch” — gradient compression algorithm for low-bandwidth training
- Squinch maintains state-of-the-art MFU (Model FLOP Utilization) despite bandwidth constraint
- 4 other tricks mentioned but not fully detailed in thread
- Before Google acquisition: company had to solve bandwidth constraints rather than throw money at hardware
Entities
- Anthropic (context: competitor)
Concepts
- Scaling & Compute — gradient compression as scaling technique
- RL Infrastructure — infrastructure optimization tricks