Agent Memory
Overview
How AI agents persist and manage knowledge across sessions. The fundamental challenge: LLMs are stateless, but useful agents need continuity.
Approaches (ordered by sophistication)
- Chat history — append everything, hope the context window is big enough. Doesn’t scale.
- Tool-based memory — MemGPT-style: agent calls
save_memory()/recall_memory()tools. Simple but limited. - File-based memory — agent reads/writes files (MEMORY.md, daily notes). Our current approach. Works but no cross-referencing.
- Git-backed filesystem — Letta’s Context Repositories. Files + versioning + concurrent multi-agent writes. Source
- LLM-maintained wiki — Karpathy’s pattern. Structured, interlinked pages with ingestion/query/lint workflows. Our current project. Source
Key Design Questions
- What gets remembered? Everything vs curated. Trade-off between completeness and noise.
- Who maintains it? Human, agent, or hybrid. Karpathy: agent writes, human curates.
- How is it organized? Flat files vs hierarchy vs interlinked wiki vs database.
- How does it scale? Index files work to ~100s of pages. Beyond that, need search (embeddings/BM25).
Open Questions
- Can agents reliably self-manage context (Letta’s bet) or do they need human-designed structure?
- What’s the right granularity? Per-conversation? Per-concept? Per-entity?
- How to handle contradictions in memory over time?
Connections
Vault Notes
- Potaris Knowledge Management Agent — Alex’s personal project: agent mounted on a directory of notes
- Potaris PRD — Zero-maintenance knowledge system; AI agents auto-organize saved content