Reinforcement-learning

A long-running agent has to keep learning without forgetting what made it useful. This write-up covers catastrophic forgetting and how it is measured, the four families of fixes, memory in token space, continual RLVR, the production data flywheel, its guardrails, the multi-LoRA serving shape, and how to run the same loop with Claude.

A world model is a learned simulator. Give it a state and an action and it predicts what happens next. This write-up follows the idea from Ha and Schmidhuber's V+M+C agent through Dreamer, MuZero and TD-MPC, to Genie, Cosmos and V-JEPA, the drift that comes with imagining too far ahead, LeCun's case for predicting representations instead of pixels, and why robots need all of this.

Post-training has moved from imitating human labels to learning from verifiable outcomes. This write-up covers the RLVR objective, GRPO and the choice of KL penalty, advantage collapse and the methods that recover the lost gradient, reward hacking and noisy verifiers, outcome versus process reward models, the production data flywheel that turns failures into weight updates, long chain-of-thought and latent reasoning, and the OpenRLHF stack that runs it all.