Research · Academic & arXiv
Back to sweepResearch sweep · deep · 2025 – present
Enterprise AI Transformation Programmes (2025–2026)
Enterprise AI transformation programmes from July 2025 to July 2026: reported success and failure rates and who measures them, delivery frameworks borrowed from and diverging from classic digital transformation, token cost economics and budgeting under consumption pricing, adoption strategy including leadership over-provisioning of access, and AI governance across AI-Ops, financial business cases, security and regulatory controls as they scale with sector risk tolerance, referencing MIT, McKinsey State of AI, DORA, ThoughtWorks Technology Radar, NIST AI RMF, ISO 42001, and the EU AI Act.
- Claude Fable 5
- financial
- tech
- academic
- vc
- blogs
- frontier
Synthesised 2026-07-15
Narrative
The single most cited data point in this space, the claim that 95% of generative AI pilots show no measurable P&L impact, originates from MIT's Project NANDA "State of AI in Business 2025" report rather than a peer-reviewed paper. The report is based on 52 executive interviews, surveys of 153 leaders, and analysis of 300 public AI deployments, and its lead author Aditya Challapally has acknowledged that companies were often hesitant to share failure rates, a methodological caveat that critics have used to question the durability of the headline number. The same report describes a "GenAI Divide" in which over 80% of organizations have piloted tools such as ChatGPT or Copilot and nearly 40% report deployment, yet enterprise-grade custom systems are abandoned at high rates, with 60% of firms evaluating them but only 5% reaching production.
The most methodologically rigorous empirical counterpoint to vendor-driven adoption narratives comes from METR, whose July 2025 randomized controlled trial found that when experienced open-source developers were allowed to use AI tools on real repositories, they took 19% longer to complete tasks, even though they had forecast a 24% speedup and, after the fact, still estimated they had been roughly 20% faster. This "perceived versus actual performance" gap, drawn from 16 developers completing 246 tasks, has become a touchstone finding cited across subsequent productivity literature, though METR itself now labels the result historical given how quickly frontier tools have changed. METR's parallel benchmark work, HCAST (Human-Calibrated Autonomy Software Tasks) and RE-Bench, underpins its "time horizon" metric, which the organization says has grown exponentially with a doubling time of roughly seven months since 2019, a trend it continued tracking into 2026 with an expanded 228-task suite.
Economic literature from NBER working papers offers a more measured empirical middle ground. Bick, Blandin and Deming's survey work found that as of late 2024, nearly 40 percent of the U.S. population aged 18 to 64 uses generative AI, with adoption at work proceeding as fast as the personal computer's early diffusion, while Brynjolfsson and colleagues' firm-level studies of customer-service deployment show productivity gains concentrated among newer, less experienced workers. A large three-experiment field study spanning nearly 5,000 developers at Microsoft, Accenture and a Fortune 100 firm found AI-assisted developers increased weekly task completion by 26.08%, with junior developers showing larger gains than seniors, complicating any single "success rate" narrative.
Governance and framework literature on arXiv has grown rapidly but remains dominated by proposed maturity models and architectures rather than longitudinal outcome data. Papers such as the Agentic AI Governance Maturity Model note that industry surveys report only 21% of enterprises have mature governance models for autonomous agents while 40% of agentic AI projects are projected to fail by 2027 due to inadequate governance, and validate their five-level framework, grounded in NIST AI RMF and ISO/IEC 42001, through simulation rather than field deployment. Complementary work maps ISO/IEC 42001, NIST AI RMF and the EU AI Act as complementary rather than substitutive frameworks, with ISO providing certifiable management-system framing, NIST providing risk-function structure, and the EU AI Act providing binding legal requirements within EU scope. Separate empirical work on regulatory operationalization finds that while data governance and cybersecurity practices are relatively mature, significant weaknesses persist in continuous lifecycle governance and oversight of autonomous and agentic AI systems, with explainability requirements inconsistently implemented despite regulatory mandates.
Sources
| ID | Title | Outlet | Date | Significance |
|---|---|---|---|---|
| a1 | MIT report: 95% of generative AI pilots at companies are failing | Fortune | 2025-08 | Primary journalistic account of MIT NANDA's "GenAI Divide" report and its headline 95% failure statistic, including methodology details and lead author caveats about self-reported failure data. |
| a2 | MIT Report Finds 95% of AI Pilots Fail to Deliver ROI, Exposing "GenAI Divide" | Legal.io | 2025-08 | Details the report's sample (52 interviews, 153 leader surveys, 300 public deployments) and the divide between high pilot adoption and low production conversion. |
| a3 | Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity | arXiv / METR | 2025-07 | METR's landmark RCT finding AI tools slowed experienced developers by 19% despite developers perceiving a speedup, the most rigorous empirical challenge to vendor productivity claims. |
| a4 | Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (blog) | METR | 2025-07 | METR's own plain-language write-up of the RCT, including its update noting a February 2026 follow-up on late-2025 AI tools. |
| a5 | HCAST: Human-Calibrated Autonomy Software Tasks | METR | 2025 | Foundational benchmark paper describing 140 human baseliners' 563 attempts across software engineering, ML and cybersecurity tasks, underpinning METR's capability measurement methodology. |
| a6 | Evaluating frontier AI R&D capabilities of language model agents against human experts (RE-Bench) | METR | 2024-11 | Introduces RE-Bench, comparing frontier model agents to 71 human expert attempts on ML research engineering tasks, a core benchmark for measuring AI R&D automation. |
| a7 | Measuring AI Ability to Complete Long Tasks | arXiv / METR | 2025-12 | Introduces the 50%-task-completion time horizon metric and documents a seven-month doubling time in AI capability from 2019-2025, widely used as a capability-trend proxy. |
| a8 | Time Horizon 1.1 | METR | 2026-01 | Updates the time-horizon methodology with an expanded 228-task suite, showing continued exponential capability growth through 2025-2026. |
| a9 | The Rapid Adoption of Generative AI (NBER Working Paper) | NBER / SSRN | 2024-09 | Nationally representative survey finding nearly 40% of the US working-age population uses generative AI, with work adoption as fast as the PC, providing an independent adoption baseline against consultancy figures. |
| a10 | Generative AI at Work (NBER Working Paper) | NBER | 2023 | Micro-level firm study by Brynjolfsson and Li on generative AI deployment in customer service, a foundational field study on productivity heterogeneity by worker experience. |
| a11 | Shifting Work Patterns with Generative AI (NBER Working Paper w33795) | NBER | 2025 | Reviews the empirical literature on generative AI's effect on work patterns and explains why workplace adoption of new AI capabilities takes time even after individual productivity gains are demonstrated. |
| a12 | The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers | SSRN | 2025-08 | Large-scale field experiment across nearly 5,000 developers at Microsoft, Accenture and a Fortune 100 firm finding a 26.08% increase in weekly task completion, with larger gains for junior developers. |
| a13 | The Labor Market Effects of Generative Artificial Intelligence | SSRN | 2026-01 | Comprehensive worker survey finding 35.9% of US workers used generative AI by December 2025 and identifying small positive wage effects with no aggregate employment decline, countering degradation narratives. |
| a14 | An Empirical Study of Measurement Framework Adoption - DORA and SPACE: How Organizational Context Shapes Success and Failure | SSRN | 2025-07 | Examines how organizational culture, size and engineering maturity shape success or failure of DORA and SPACE measurement framework adoption, directly relevant to how classic DevOps metrics are being ported into AI delivery. |
| a15 | A Framework for the Adoption and Integration of Generative AI in Midsize Organizations and Enterprises (FAIGMOE) | arXiv | 2025-10 | Proposes a structured adoption framework for generative AI in mid-size enterprises, synthesising pilot-to-scale considerations and citing prior DORA/SPACE and pilot-scaling literature. |
| a16 | The Unified Control Framework: Establishing a Common Foundation for Enterprise AI Governance, Risk Management and Regulatory Compliance | arXiv | 2025-03 | Widely cited foundational governance paper proposing a unified control architecture spanning enterprise AI governance, risk and regulatory compliance, referenced across subsequent governance maturity papers. |
| a17 | Governance Frameworks for Enterprise AI Systems Operating in Regulated Environments | International Journal of Computer Applications | 2025 | Synthesis finding that data governance and cybersecurity practices are relatively mature while lifecycle governance and oversight of agentic AI systems remain weak despite regulatory mandates. |
| a18 | Engaged AI Governance: Addressing the Last Mile Challenge Through Internal Expert Collaboration | arXiv | 2026-04 | Empirical study of how development teams operationalize EU AI Act-style regulatory requirements through collaborative workshops, addressing the scarce empirical literature on governance implementation versus principle-level frameworks. |
| a19 | AI Agents Under EU Law: A Compliance Architecture for AI Providers | arXiv | 2026-04 | Analyses how agentic AI's non-human identities and privileged system access create compliance challenges under EU law that traditional IAM frameworks were not designed to handle. |
| a20 | AI Governance Frameworks: ISO/IEC 42001, NIST AI RMF, and the EU AI Act | SSRN | 2026-05 | Systematic synthesis of the three principal AI governance frameworks as complementary rather than substitutive, incorporating the EU AI Act's November 2025 Digital Omnibus amendment. |
| a21 | Governing the Agentic Enterprise: A Governance Maturity Model for Managing AI Agent Sprawl in Business Operations | arXiv | 2026-03 | Proposes a five-level Agentic AI Governance Maturity Model grounded in NIST AI RMF and ISO/IEC 42001, citing industry data that only 21% of enterprises have mature agent governance while 40% of agentic AI projects are projected to fail by 2027. |
| a22 | Governing the Agentic Enterprise: A New Operating Model for Autonomous AI at Scale | California Management Review | 2026-03 | Proposes the Agentic Operating Model, arguing agentic AI is an institutional shift requiring new governance layers distinct from earlier copilot-era operating models. |
| a23 | Runtime Governance for AI Agents: Policies on Paths | arXiv | 2026-03 | Cites a 2026 KPMG survey finding 75% of large-enterprise leaders rank security, compliance and auditability as the most critical requirements for agent deployment, framing runtime governance as the key bottleneck. |
| a24 | AI Trust OS: A Continuous Governance Framework for Autonomous AI Observability and Zero-Trust Compliance in Enterprise Environments | arXiv | 2026-04 | Proposes telemetry-first continuous governance to replace manual attestation-based compliance workflows, addressing supply-chain AI governance requirements emerging under the EU AI Act. |
| a25 | AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities | arXiv | 2025-08 | Surveys token-cost and pricing dimensions of agentic coding workflows, quantifying how reasoning strategies and tool-augmented loops multiply token consumption relative to single-turn use. |
| a26 | Measuring AI R&D Automation | arXiv | 2026-03 | Related-work synthesis situating METR's RE-Bench alongside SWE-bench, MLE-bench and PaperBench within the broader AI R&D automation benchmark landscape. |
| a27 | PostTrainBench: Can LLM Agents Automate LLM Post-Training? | arXiv | 2026-03 | Benchmark extending AI R&D automation evaluation beyond METR's RE-Bench, finding newer models substantially outperform Anthropic's Sonnet 4.5 evaluation on end-to-end training-loop automation. |