DORA Metrics in the AI Era

Narrative

DORA's own account of its metric history shows two structural revisions relevant to the AI period. The 2023 report redefined the recovery metric to isolate failures caused by a production change from failures caused by external factors such as a data-centre outage, tightening it statistically against the other three keys. The bigger shift came in 2024, when researchers treated change failure rate as a proxy for rework and introduced a fifth metric, deployment rework rate, to capture unplanned fixes to user-facing bugs directly. The 2024 report also formally split the four keys into a throughput pair (deployment frequency, lead time) and a stability pair (change failure rate, failed deployment recovery time), a distinction subsequent DORA and vendor commentary treats as load-bearing rather than cosmetic.

DORA's 2024 survey produced the counterintuitive headline that anchors most subsequent AI-and-delivery commentary: modelling a 25% increase in AI adoption predicted a 1.5% decrease in delivery throughput and a 7.2% decrease in delivery stability, alongside gains in individual flow and productivity. Google's own DORA team framed the same wave of data as showing AI adoption "significantly increases individual productivity, flow, and job satisfaction" while "negatively impacting software delivery stability and throughput," attributing the tension to fundamentals such as batch size and testing discipline being bypassed rather than reinforced by AI-assisted coding. The 2025 "State of AI-assisted Software Development" report, drawing on more than 100 hours of qualitative data and survey responses from nearly 5,000 technology professionals, reframed this as an amplifier effect rather than a simple trade-off: AI "doesn't fix a team; it amplifies what's already there," with strong teams becoming more efficient and struggling teams having existing dysfunction intensified. That report introduced the DORA AI Capabilities Model, describing seven foundational practices proposed to amplify AI's positive effects, and classified organisations into seven team profiles running from "harmonious high-achievers" to a "legacy bottleneck" cluster, alongside a finding that 90% of respondents now use AI daily, a 14-point rise on 2024.

METR's randomized controlled trial is the one experimental data point in a field otherwise dominated by correlational survey and telemetry work. Sixteen experienced open-source developers completed 246 real tasks in repositories they knew well, alternating between AI-allowed (primarily Cursor Pro with Claude 3.5/3.7 Sonnet) and AI-disallowed conditions; the measured result was that developers took 19% longer to complete tasks with AI, with a 95% confidence interval of roughly -40% to -2%, even though they had forecast a 24% speedup and self-reported a 20% speedup afterward. METR's own follow-up work traced part of the gap to a mismatch between algorithmic benchmark scoring and holistic code-quality review, since AI systems often produce code that "scores well but isn't production-ready due to issues with test coverage, formatting, and code quality." A February 2026 update reported that METR's second productivity study had to be redesigned because an increasing share of developers refused to participate in AI-disallowed conditions and because a pay-rate cut to $50/hour introduced new selection effects, with the team stating it now believes developers are more sped up by AI in early 2026 than the original study measured in early 2025.

GitClear's commit-level telemetry corroborates the instability side of DORA's finding rather than the productivity side. Its second annual AI Copilot Code Quality report, covering 211 million changed lines from 2020-2024, found duplicated code blocks rose roughly eightfold in 2024 and linked cloned code to "anywhere from 15% to 50% more" defects. A 2026 follow-up tracking 2023-2026 found block duplication climbing 81% from its 2023 baseline and, separately, that heavy AI users out-produce non-users by 4-10x but that most of that gap predates AI adoption, with heavy users showing a more modest 25% velocity gain against their own past output. On the model-capability side, METR's pre-deployment evaluations tied to specific frontier releases (GPT-4.5, GPT-5, and the GPT-5.6 preview) show the autonomous coding "time horizon" climbing steadily, with GPT-5 measured at a 50%-time-horizon of roughly 2h15m against o3's 1h30m, providing the capability trendline against which DORA's throughput and stability findings should be read: models are getting measurably more capable on long-horizon coding tasks even as team-level delivery telemetry shows instability rising in step with adoption.