AI Dark Code — Organisational Accountability and Control

Narrative

The dominant story from VC and analyst coverage across June 2025 to April 2026 is a collision between accelerating enterprise adoption of AI-generated and agent-produced code and a governance apparatus that has not kept pace. a16z's June 2025 CIO survey established the baseline: agentic workflows are now production realities, but 'quality assurance of agents is not super easy' and every prompt is tuned for a specific model, creating deep lock-in and accountability ambiguity. By November 2025, McKinsey's State of AI — drawing on 1,993 respondents — confirmed that only 23% of enterprises were scaling any agentic system, with 51% having experienced AI incidents; governance and human-in-the-loop accountability were the explicit separators between high performers and the rest. Bain's Technology Report 2025 reinforced the accountability distribution problem: while central platform teams own the core infrastructure, domain teams must own agent assembly, testing, and monitoring — yet most organisations lack the observability and lineage infrastructure to make this work. Gartner's December 2025 prediction report introduced the sharpest specific quantitative warning: prompt-to-app approaches will increase software defects by 2,500% by 2028, driven by 'context-deficient' AI code that is syntactically correct but architecturally naive, producing bugs exponentially more expensive to fix than conventional errors. Forrester responded in August 2025 with the AEGIS framework — the analyst community's first purpose-built governance model for agentic AI — introducing 'least agency' (versus least privilege), 'continuous assurance', and 'explainable outcomes' as core principles, and explicitly naming 'obscured causal provenance' and the near-impossibility of post-incident forensics as the defining dark-code problem.

By early 2026, the analyst consensus had crystallised around several hard numbers and structural predictions. Gartner's January 2026 zero-trust data governance prediction (50% of organisations adopting zero-trust postures for AI-generated data by 2028) and its February 2026 AI governance market sizing ($492M in 2026 spend, $1B+ by 2030) framed the commercial opportunity. Microsoft's February 2026 Cyber Pulse report provided the most direct empirical anchor for the dark-code thesis: first-party telemetry confirmed more than 80% of Fortune 500 companies are actively running AI agents built with low-code/no-code tools — agents built substantially outside formal engineering review channels. CB Insights identified AI agent observability and evaluation tooling as the fastest-growing cybersecurity sub-segment and an M&A battleground for 2026, with the agent security and risk management market leading on Mosaic score momentum. KPMG's Q4 2025 AI Pulse survey quantified investment intent: half of enterprise leaders plan to allocate $10–50M specifically for data lineage, model governance, and agentic architecture hardening. The VC prediction landscape (a16z, Bessemer, Sequoia, aggregated by multiple synthesis pieces) converged on 'code clean-up agents' and 'agent operations' as the two organisational responses to the 2025 AI coding boom's technical debt hangover — with Bessemer's Lindsey Li naming the category explicitly and multiple analysts predicting 'agent operations' will become a formal enterprise function analogous to DevOps, complete with audit logs, rollback controls, and human override as non-negotiable production requirements.