The Karpathy Loop — AI Agents Running Autonomous Training Experiments

Narrative

In March 2026, Andrej Karpathy released autoresearch, an open-source system implementing autonomous research loops where AI agents iteratively propose, test, and evaluate changes to training code within fixed compute budgets (typically 5-minute GPU slots). The minimal 630-line Python implementation encodes a feedback pattern: agent reads source code, forms hypotheses for improvement (e.g., learning rates, model depth, optimizer settings), modifies code, runs a timed experiment, and evaluates results using fitness signals (validation bits-per-byte). Karpathy's own framing positions this within 'agentic engineering'—a shift where humans direct and supervise while AI agents handle code iteration. The New Stack and Fortune coverage documents the mechanics and significance; NextBigFuture captures Karpathy's broader vision of this 'self-improvement loopy era.' Real-world adoption demonstrates measurable impact: Karpathy's published results show 700 experiments over two days discovering 20 optimizations improving training speed by 11% on small language models. Shopify CEO Tobi Lutke independently adopted autoresearch on an internal 0.8B parameter model, achieving 19% validation improvement from 37 overnight experiments. SkyPilot's research extends the pattern to GPU clusters, indicating clear production scaling pathways. DataCamp's coverage emphasizes accessibility to practitioners. Collectively, practitioner and industry publications frame autoresearch as significant methodology for engineering effectiveness—automating expensive hyperparameter and architecture search, reducing reliance on large research teams, and enabling smaller teams to run continuous autonomous optimization cycles.