Agentic Engineering And Enterprise Architecture Discipline

Narrative

The academic and benchmark story is that agentic engineering is not being defined by one glamorous coding demo, but by a stack of increasingly hard evaluations that measure real modification work, long-horizon autonomy, and the gap between benchmark success and production acceptance. SWE-bench established the core problem: editing real repositories to resolve issues, where early top models solved only trivial cases. METR then pushed the field toward autonomy metrics with HCAST and RE-Bench, shifting attention from single-task pass rates to human-calibrated time horizons and sustained task completion. Recent METR reports on o3, DeepSeek/Qwen, and GPT-5.1-Codex-Max show that capability is rising, but the meaningful question is how far agents can operate without human intervention across multi-step software tasks.

A second clear theme is that enterprise-quality software properties are now the bottleneck, not just code synthesis. The literature on deprecated APIs, hallucinated code, SWE-bench-passing PRs that maintainers would not merge, and security-focused studies of exploitation and vulnerability repair all point in the same direction: agents can generate plausible artifacts, but maintainability, security, reviewability, and operational fit remain fragile. The most useful current research therefore is benchmark-heavy, empirical, and failure-oriented, with METR especially valuable because it measures both capability growth and the ways benchmark traces can mislead teams about real-world utility.