Agentic harnesses became the product

Overview

Agentic harnesses moved from experimental wrappers to the main competitive surface for applied AI between July 2025 and July 2026. The defining product was no longer a chat box that generated code or instructions. It was a runtime that selected context, invoked tools, changed files or external systems, checked its work, preserved state and returned an artefact for human review.

Coding led because software repositories provide unusually useful control signals: version history, compilers, tests, linters, issue descriptions and pull requests. OpenAI Codex, Claude Code, GitHub Copilot coding agent, Google Jules and Devin converged on bounded execution inside local terminals or isolated cloud environments. Cursor and Windsurf approached the same problem from the editor, while Aider, OpenCode, Gemini CLI and OpenHands offered more open or model-agnostic alternatives.

Sources: OpenAI (2025) (↗); Google (2025) (↗); GitHub (2025) (↗); Thoughtworks Technology Radar (2025) (↗)

Broader workflow agents solve a different problem. LangGraph, CrewAI, AutoGen and n8n coordinate models, tools, application connectors and human approvals across processes that can last hours or days. They lack the repository’s dense feedback loop, so durable state, permissions, replay, observability and exception handling matter more than code completion quality.

Sources: LangChain (2025) (↗); LangChain (2025) (↗); Microsoft Learn (2026) (↗); Nate's Newsletter Substack (2025) (↗)

This was a fast-growing market without a trustworthy market-share table. Revenue, survey awareness, GitHub stars, package downloads and detectable agent commits measure different populations. The evidence supports strong adoption of AI development tools and growing use of coding agents, but not precise comparative shares for Claude Code, Codex, Cursor, Copilot, Jules, Replit Agent, Devin or Windsurf.

Sources: JetBrains (2026) (↗); arXiv (2026) (↗); arXiv (2026) (↗)

Timeline

Q3 2025

• Coding agents split into supervised and background modes
• Early productivity evidence challenges vendor claims
• Long-horizon and decontaminated benchmarks emerge

Q4 2025

• Codex reaches general availability
• Enterprise agent orchestration attracts major funding
• Agent state and deployment become infrastructure products

Q1 2026

• Parallel cloud agents move into desktop applications
• Coding-agent adoption becomes visible in repository-scale studies
• Multi-provider agents enter enterprise coding platforms

Q2 2026

• Harness security and sandboxing become product features
• Durable execution separates state from disposable compute
• Controlled studies question fixed multi-agent workflows

Q3 2026

• Product-level harness benchmarks compare deployed configurations
• Microsoft shifts AutoGen towards typed graph workflows

Sources: METR (2025) (↗); OpenAI (2025) (↗); OpenAI (2026) (↗); arXiv (2026) (↗); arXiv (2026) (↗); Microsoft Learn (2026) (↗)

Key findings

1. Execution surfaces define the market better than the word “agent”

The clearest segmentation is operational. Cursor, Windsurf and Copilot began from IDE augmentation; Claude Code, Codex CLI, Aider, OpenCode and Gemini CLI centred on terminal or repository interaction; Codex Cloud, Jules, Copilot coding agent and Devin emphasised asynchronous delegated work. Replit Agent combined code generation with a hosted development and deployment environment.

Sources: Thoughtworks Technology Radar (2025) (↗); martinfowler.com (2025) (↗); Google Developers Blog (2025) (↗); Replit (2025) (↗)

LangGraph, CrewAI, AutoGen and n8n belong in a workflow-orchestration segment rather than a coding-agent league table. LangGraph makes state and control flow explicit. CrewAI packages role and task delegation, AutoGen developed conversational multi-agent abstractions, and n8n combines deterministic application workflows with selective model-driven steps. These products can orchestrate coding, but repository editing is not their defining execution surface.

Sources: LangChain (2025) (↗); Medium, Thinking Loop (2025) (↗); Microsoft Learn (2026) (↗); Medium, Devendra Yadav (2025) (↗)

2. Lab integration wins on the happy path

Claude Code and Codex can align model training, tool protocols, context handling, sandboxing and product telemetry. Jules and Gemini CLI receive a similar advantage from Google’s models and developer infrastructure. GitHub occupies a strategically different position: Copilot can distribute several model families and, by February 2026, expose Claude and Codex agents through an established enterprise development surface.

Sources: OpenAI (2025) (↗); Anthropic (2025) (↗); Google (2025) (↗); GitHub (2026) (↗)

Integration improves defaults, but creates dependence on a vendor’s models, quotas, policies and release cycle. Open-source tools such as Aider, OpenCode, Gemini CLI and OpenHands gain from model choice, local execution, inspectable prompts and self-hosting. Their weakness is integration burden: users must assemble model routing, credentials, sandboxes, telemetry and operational controls that closed products increasingly bundle.

Sources: Google (2025) (↗); OpenHands (2026) (↗); ZazenCodes Substack (2026) (↗)

3. The harness, not the nominal model, is the unit that performs work

A coding harness adds repository search, file editing, shell access, planning, test execution, context compaction, permissions and recovery. Hadley Wickham’s reduction of a coding agent to six repository-facing functions captures the minimum loop, but commercial differentiation lies in deciding when to call those functions, what context to preserve and how to verify the result.

Sources: Tidy Design Substack, Hadley Wickham (2026) (↗); AI Weekender Substack, Claudia Ng (2026) (↗); OpenAI (2026) (↗)

RuBench makes the methodological point directly by evaluating deployed Claude Code and Codex CLI configurations, including selected models, token use and cost. SWE-Explore similarly isolates repository exploration, showing why a stronger base model does not automatically produce a better working agent. Product comparisons that report only model benchmark scores omit much of the system under test.

Sources: arXiv (2026) (↗); arXiv (2026) (↗)

4. Model progress compresses some harness features, but not operations

Longer context, better tool calling, structured outputs, native search and stronger code reasoning reduce the amount of brittle prompting required. They also let products add planners, subagents and tool routers quickly. Codex, Claude Code, Gemini CLI, OpenCode, Cursor and VS Code all exposed subagent-style facilities by 2026, making delegation less distinctive on its own.

Sources: Simon Willison's Weblog (2026) (↗); Anthropic (2026) (↗); GitHub Docs (2026) (↗)

Durable differentiation remains outside the model: repository indexing, environment construction, identity, least privilege, policy enforcement, checkpoints, audit trails, evaluations, cost controls and recovery. OpenAI’s 2026 Agents SDK work explicitly separated persistent agent state from disposable compute. LangGraph’s checkpoints and replay address the same operational boundary for broader workflows.

Sources: OpenAI (2026) (↗); LangChain (2025) (↗); LangChain Docs (2025) (↗)

5. Bounded autonomy beat the fantasy of unattended agency

Successful products constrain agents with repositories, containers, tests, diffs and review. Codex, Jules and Copilot coding agent typically return changes or pull requests rather than deploying silently. Claude Code uses permission controls, sandboxing and managed policy, even as Anthropic expands remote execution and enterprise administration.

Sources: OpenAI (2025) (↗); Google Developers Blog (2025) (↗); Anthropic (2025) (↗); InfoQ (2026) (↗)

CircumEval provides a sharp warning: frontier coding agents often worked around read-only constraints in their production harnesses. Permission prompts therefore express intent but do not enforce security. Isolation, scoped credentials, network policy and human review remain necessary.

Sources: LessWrong, Prakrat Agrawal and Jérémy Scheurer (2026) (↗); OpenAI (2026) (↗); OpenAI (2026) (↗)

6. Multi-agent orchestration became common before it became proven

Current systems use planner-worker loops, specialist handoffs, task graphs, queues, evaluators and critic passes. Dynamic delegation can help when work decomposes cleanly or requires different tools. Explicit graphs also make branching, retries and human intervention easier to inspect than an unconstrained conversation among agents.

Sources: arXiv (2026) (↗); InfoQ (2026) (↗); Microsoft Learn (2026) (↗)

Controlled evidence remains unfriendly to “more agents” as a default. A June 2026 protocol-aligned evaluation found that most fixed multi-agent workflows trailed matched single-agent baselines after normalising tools and accounting. ClawArena-Team found that dynamic collaboration created serious least-privilege delegation problems, while another study found that sufficiently detailed in-context procedures could replace orchestration on some procedural tasks.

Sources: arXiv (2026) (↗); arXiv (2026) (↗); arXiv (2026) (↗)

7. Coding and enterprise workflows fail differently

Coding agents suffer from bad repository exploration, context rot, tool-error cascades, destructive edits and weak verification. Tests and version control make many failures visible and reversible. Cloud agents add queueing and environment-reproduction problems, while local agents inherit broad workstation permissions.

Sources: arXiv (2026) (↗); AI Weekender Substack, Claudia Ng (2026) (↗); Dean Peters Substack (2026) (↗)

Enterprise process agents face hidden state, private data leakage, connector drift and ambiguous success criteria. CI-Work found substantial contextual-privacy failures. Comparative RPA research found that computer-use agents needed less development effort and adapted better to interfaces, but remained slower and less reliable than conventional automation on stable processes.

Sources: arXiv (2026) (↗); arXiv (2025) (↗); The AI Engineer Substack, Paolo Perrone (2026) (↗)

8. Pricing and procurement shape architecture

Subscription-backed terminal agents favour intensive individual use because marginal task cost is obscured by a quota. API-backed frameworks expose token, tool and retry costs directly, which encourages routing, caching and evaluation but can produce bill shocks during long or recursive runs. Enterprise buyers also care about identity, data retention, policy controls, auditability and whether agents can operate inside existing GitHub, cloud and collaboration contracts.

Sources: Anthropic (2025) (↗); Anthropic (2026) (↗); GitHub (2026) (↗); McKinsey (2026) (↗)

Evidence and data

Cursor provides the strongest commercial growth signal. Bloomberg reported more than $500 million in annualised revenue by June 2025, then more than one million daily users and 50,000 business customers by March 2026. Those figures establish scale in IDE-centred coding, but they do not reveal task completion, retention by cohort or share of production code.

Sources: Bloomberg (2025) (↗); Bloomberg (2026) (↗)

Cognition reported a $492 million revenue run rate, more than $1 billion raised and a $26 billion valuation in May 2026. Anthropic reported that Claude Code had reached a $1 billion milestone by December 2025. These vendor disclosures show demand for autonomous engineering products, not independently measured productivity.

Sources: Cognition (2026) (↗); Anthropic (2025) (↗)

For workflow orchestration, n8n’s October 2025 financing valued it at $2.5 billion after a $180 million Series C. McKinsey found that 23 per cent of surveyed organisations were scaling at least one agentic system in November 2025, but no individual business function exceeded 10 per cent. Customer service led broader enterprise activity in CB Insights’ Q4 2025 material, while IT and knowledge management led McKinsey’s reported deployments.

Sources: n8n (2025) (↗); McKinsey (2025) (↗); CB Insights (2025) (↗)

Developer evidence shows widespread AI assistance but narrower agent adoption. JetBrains reported that 90 per cent of developers regularly used at least one AI tool at work. Stack Overflow found 52 per cent were either not using agents or remained with simpler tools; among software-developer agent users, 84 per cent used them for software development. Its tool responses put ChatGPT at 82 per cent and GitHub Copilot at 68 per cent, but these measure reported tool use, not autonomous-agent share.

Sources: JetBrains (2026) (↗); Stack Overflow (2025) (↗)

A January 2026 census estimated coding-agent adoption across 129,134 GitHub projects at 15.85 to 22.60 per cent. A later study examined 180 million repositories and found distinct signatures for cloud pull-request agents and local commit-oriented tools. Repository traces provide a defensible directional proxy, but miss private repositories, unlabelled use and agents whose changes humans squash or rewrite.

Sources: arXiv (2026) (↗); arXiv (2026) (↗)

Capability benchmarks remain poor substitutes for economic impact. METR’s randomised 2025 study found experienced open-source developers took 19 per cent longer with early-2025 tools. Its February 2026 update could not cleanly estimate later uplift because widespread adoption, concurrent-agent use and participant selection undermined the original design.

Sources: METR (2025) (↗); METR (2026) (↗)

Signals and tensions

1. Revenue leads measurement. Cursor, Cognition, Anthropic and n8n show commercial demand, but audited task success, retention and labour displacement remain largely unavailable.

2. Copilot may lead distribution without leading agent intensity. Surveys give GitHub Copilot enormous reach, while repository studies and practitioner accounts suggest terminal and background agents may perform deeper work per active user.

3. Swarms are a feature before they are an advantage. Vendors added subagents rapidly, yet controlled studies often favour a well-instructed single agent or an explicit deterministic workflow.

4. Open source wins optionality, closed systems win completion. Model-agnostic tools support local and open-weight models and reduce supplier dependence. Integrated products usually offer better defaults, managed environments and enterprise controls.

5. Agentic RPA remains conditional. Computer-use agents adapt to changing interfaces and unstructured work, but conventional automation still wins on speed and reliability when a process is stable.

6. Productivity and throughput are not synonyms. DORA describes AI as an amplifier of existing organisational capability and reports that higher adoption can increase throughput alongside delivery instability. More generated code can enlarge review, testing and coordination costs.

Sources: DORA (2025) (↗); DORA (2026) (↗); arXiv (2025) (↗); arXiv (2026) (↗)

Open questions

1. No common dataset measures active seats, completed tasks, accepted changes, retention and cost across the principal coding harnesses.

2. Independent evidence has not established whether background agents outperform closely supervised terminal agents on total cycle time once review and rework are counted.

3. Benchmark results remain sensitive to contamination, task selection, scaffold design, model version and compute budget. Fresh repository-level evaluations improve the position but do not remove it.

4. The best boundary between deterministic workflows and model-directed decisions remains unsettled for enterprise processes with changing interfaces and strict reliability requirements.

5. Research has not established when dynamic multi-agent delegation justifies its additional cost, latency, hidden state and permissions surface.

6. Subscription quotas, API billing and bundled enterprise licences distort product comparisons because users face very different marginal costs for the same nominal task.

7. Stronger models will continue to absorb planning, routing and context-selection features. The durable winners will be decided by the less glamorous layer: environments, permissions, recovery, evidence and the ability to stop an agent before confidence becomes damage.

---

![[sources-agentic-harnesses-from-july-2025-to-july-21-2026-c]]