Software Project Outcomes Over Time, and Is AI Improving Them

Narrative

Longitudinal software-project outcome data converges on a stable, uncomfortable baseline that has barely moved across three decades despite methodology change. Standish's CHAOS series, run since 1994, has become the most-cited but most-contested source: its founding figure was 16.2% of projects delivered on-time, on-budget, on-scope, with 31.1% cancelled and large-company projects delivering only 42% of planned features on average. By the 2020 edition, Standish reported 31% "successful," 50% "challenged" and 19% "failed," with small projects succeeding roughly 90% of the time versus under 10% for large ones, and government IT projects performing markedly worse. Independent academics (Eveleens and Verhoef's "Rise and Fall of the Chaos Report Figures") argue these figures are largely artefacts of shifting definitions and non-random sampling, and are "meaningless for benchmarking" as a result, a critique that has stuck through repeated citation cycles.

McKinsey's 2012 study with Oxford's BT Centre for Major Programme Management, based on more than 5,400 IT projects, found large IT projects (over $15 million) run 45% over budget and 7% over time on average while delivering 56% less value than predicted, with 17% classified as "black swan" projects with budget overruns exceeding 200%. Bent Flyvbjerg's Oxford Global Projects database extends this into fat-tail territory: IT projects show a power-law rather than normal distribution of cost overruns, a mean overrun near 73%, and, in the worst 18% of cases exceeding 50% overrun, a conditional tail expectation around 447%, worse than nuclear storage construction. PMI's Pulse of the Profession has moved toward a "value delivered" definition of success rather than the iron triangle, reporting a 73.8% average project performance rate in 2024 and finding no statistically significant difference in performance between agile, hybrid, predictive or remote/in-person delivery models, directly undercutting vendor claims that methodology choice alone drives outcomes.

On low-code/no-code, Gartner's Magic Quadrant for Enterprise Low-Code Application Platforms places OutSystems, Mendix, Microsoft Power Apps, ServiceNow, Appian and Salesforce as Leaders through the 2025 and 2026 cycles, and Gartner projects that by 2026 roughly 75% of new applications will be built on low-code platforms, with the market reaching an estimated $44.5 billion. Independent commentary increasingly flags a governance gap: citizen developers lack testing and security expertise, and analysts warn of a "2027-2028 technical debt reckoning" as ungoverned low-code proliferates, an important corrective to purely vendor-driven adoption-curve framing.

On AI-assisted coding, the strongest independent evidence is genuinely mixed and time-sensitive. METR's July 2025 randomised controlled trial of 16 experienced open-source developers found AI tools made them 19% slower on complex, familiar codebases, even though developers believed AI had sped them up by 20%; METR's own February 2026 update reports the same cohort's estimated effect flipping to an 18% speedup, while cautioning that growing non-participation bias (developers refusing to work without AI) makes the newer estimate unreliable. Google's DORA reports show a similar reversal at industry scale: the 2024 report found AI adoption correlated with lower throughput and stability, while the 2025 "State of AI-assisted Software Development" report (nearly 5,000 respondents) found AI adoption now correlates positively with throughput, product performance and individual effectiveness, but continues to correlate with worse delivery stability, a pattern DORA calls the "amplifier" effect: AI magnifies existing organisational strengths and dysfunctions rather than fixing them. GitClear's code-churn research across roughly 211 million lines (2020-2024) found refactoring collapsed from about 25% to under 10% of changed lines while duplicated code blocks rose eightfold and churn (code rewritten within two weeks) roughly doubled, suggesting throughput gains are partly offset by rework and maintainability costs that vendor-commissioned Forrester Total Economic Impact studies (376% claimed ROI for GitHub Copilot at scale) do not capture.