OpenClaw-RL: Continuous RL Optimization from Live Conversations
Source: https://github.com/Gen-Verse/OpenClaw-RL Author: Gen-Verse Date: 2025-2026
Summary
RL framework that wraps a self-hosted model as an OpenAI-compatible API, intercepts live multi-turn conversations, and continuously optimizes the policy in the background. Fully asynchronous four-component architecture: agent serving, rollout collection, reward evaluation, policy training. All models and data remain on user infrastructure.
Key Claims
- Self-hosted, private — zero API dependency, all data on user infrastructure
- OpenAI-compatible API wrapper — drop-in replacement
- Fully asynchronous: four independent components don’t block each other
- Three optimization methods: Binary RL, On-Policy Distillation (OPD), combined
- Results: “significant improvement” with just 36 problem-solving + 24 grading interactions
- No manual labeling — conversation feedback converted automatically to training signals
- Scales from personal agents to: terminal agents, GUI agents, software engineering agents
Connection to Other Sources
Practical implementation of RL from User Conversations and Cursor Real-Time RL. Self-hosted version of what labs do in production.
Concepts
- RL Infrastructure — continuous online RL from conversation feedback
- Agent Memory — personalization through continuous learning