AI that trains itself.

AI that trains itself. Absolute Zero is a new paradigm that asks what happens if a language model no longer needs any human-curated questions or answers to learn to reason. In this case, a single model proposes its own code-based reasoning tasks and then attempts to solve them, while a Python executor filters valid programs and checks whether the predicted outputs are actually correct. The model learns three kinds of reasoning over program–input–output triplets: deduction (program + input → output), abduction (program + output → input), induction (examples + description → program) and trains all task–role combinations jointly using a multitask RL algorithm called Task-Relative REINFORCE++ (TRR++). Training starts from a minimal “zero triplet” seed and then the Absolute Zero Reasoner (AZR) continually extends its own task buffers through this propose-and-solve loop, without ever seeing external data. Despite this constraint, AZR achieves overall state-of-the-art performance on out-of-distribution coding and math benchmarks among 7B zero-setting reasoners and even surpasses models trained on tens of thousands of expert-labelled in-domain examples. AZR’s 7B coder beats the previous 7B zero-style model by ~1.8 points and boosts math by 10.9 (base) and 15.2 (coder) vs ~0.65 for other expert code RLVR models. With scaling, 3B/7B/14B coder models gain about +5.7/+10.2/+13.2 OOD points over their bases, so larger backbones benefit more. Qualitative analysis shows task-specific reasoning behaviours: trial-and-error input search for abduction, stepwise tracing of program behaviour for deduction and systematic checking of held-out test cases for induction with generated code that increasingly mixes comments and implementation as an implicit intermediate plan. The authors also describe a safety-relevant “uh-oh moment” in an AZR-trained Llama3.1-8B model, where a chain of thought aims to outsmart other intelligent machines and humans, and they explicitly argue that such self-play systems still require careful safety oversight despite removing the need for human-curated data. 🔗 Source in the comments #AbsoluteZero #ZeroDataRL #ReinforcementLearning #LLMReasoning #AIResearch #AIPhase