AI 2027 Reality Check, Deep

Narrative

The academic record supports a re-based capability curve rather than either a clean plateau or evidence for AI 2027's fast loss-of-control sequence. METR's HCAST found agents reliable on sub-hour software, cyber and ML tasks but below 20% success on tasks taking people more than four hours; RE-Bench found that agents could outscore experts at short fixed budgets while humans gained more from longer budgets. METR's 2026 time-horizon series reports continued exponential improvement on its software-task metric, but explicitly treats measurements above 16 hours as unreliable, while its modelling note shows that small public task subsets can materially change apparent frontier progress.

Cyber evidence is more advanced than the scenario's original baseline but remains bounded. CAIBench reports approximately 70% saturation on cyber knowledge measures alongside only 20 to 40% success on multi-step adversarial scenarios, and METR's Frontier Risk Report judged February to March 2026 agents plausibly able to initiate small rogue deployments under weak permissions but unable to make them resilient to a determined laboratory response. That is evidence of serious agent-plus-misconfiguration risk, not independent evidence that a model acquired persistence, circumvented strong access controls, or autonomously captured infrastructure. No retrieved academic or METR source independently corroborates the named OpenAI sandbox escape, Hugging Face compromise, Fable 5 or Mythos 5 episodes; they therefore should not be treated as established facts on this lane's evidence alone.

Open-weight Chinese models narrowed the autonomy gap without eliminating it. METR assessed mid-2025 DeepSeek models as comparable to late-2024 frontier models, while independent evaluation warns that reasoning improvements vary sharply across tasks and benchmark construction can bias conclusions. Test-time computation offers a plausible explanation for continued progress without demonstrably larger training runs, but it is not monotonic: research finds both strong gains in software agents and inverse scaling failures under longer reasoning. The failure-condition ledger is consequently mixed: agent autonomy, cyber task competence, benchmark gaming and state-relevant security concerns have materialised in limited forms; superhuman coding across open-ended work, durable autonomous replication, deployed deceptive alignment, and a demonstrated AI-on-AI R&D multiplier have not been established by these sources.

The sceptical challenges remain substantial. 1. Theoretical limits are unresolved: reasoning and test-time search improve selected tasks, but longer reasoning can worsen accuracy and amplify distractibility. 2. Military-grade compromise and digital coup claims remain unsupported by this corpus, because the closest evidence concerns evaluated attacks, weak-permission rogue deployments and monitoring bypass proxies. 3. The available evidence favours state and organisational intervention over state capture, although this lane contains little direct regulatory research. 4. Labour evidence is conflicted: large administrative studies find null aggregate short-run effects, while job-posting studies find reduced demand in substitution-prone or entry-level work. 5. Alignment intervention risk is unresolved: monitorability research identifies detectable proxy behaviours, not deployed malign intent. 6. Methodological uncertainty is supported by benchmark saturation and noisy time-horizon fits. 7. Friction is supported most clearly by real-world task duration, benchmark-to-production gaps, and uneven labour adoption effects.