The Agent Harness Landscape and the Shift to Markdown-Defined, Event-Driven Agents

How coding and general-purpose agent harnesses evolved between October 2025 and October 2026 (Claude Code, OpenAI Codex CLI, OpenCode, Pi, Antigravity CLI, Cursor, Amp, OpenClaw): adoption and user base, the move to event-driven and multi-agent workflows defined in markdown, the SKILL.md Agent Skills standard and its most popular skills repositories, and whether harness scaffolding stays necessary as models improve

Claude Fable 5.1
financial
frontier
academic
vc
blogs
tech

Synthesised 2026-10-25

Narrative

The academic literature on harnesses is young and mostly dates from February to October 2026. The strongest finding is that the markdown control plane (AGENTS.md, CLAUDE.md, SKILL.md) is now the object of study, and the results are less flattering than vendor guidance. Gloaguen et al. (ETH Zurich, arXiv 2602.11988, also an ICLR 2026 workshop paper) found that repository context files gave no improvement in task success and raised inference cost by over 20%. A July 2026 two-agent ablation (arXiv 2607.27250) reproduced a null correctness effect for Claude Code and Codex CLI, with only narrow process-level efficiency gains. A repository-level study of 2,853 GitHub projects (arXiv 2602.14690) found that AGENTS.md is adopted as a tool-agnostic standard and that Claude Code users employ the broadest range of configuration mechanisms.

Sources

ID	Title	Outlet	Date	Significance
a1	Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?	arXiv (ICLR 2026 Workshop on Memory for LLM-Based Agentic Systems)	2026-02	Gloaguen, Mündler, Müller, Raychev and Vechev (ETH Zurich, 2026) report no gain in task success from context files and a cost rise of over 20%, the central empirical challenge to the markdown-as-control-plane thesis.
a2	Evaluating AGENTS.md (ETH SRI Lab publication page)	ETH Zurich SRI Lab	2026	Confirms the venue and the headline result (no improvement, over 20% extra inference cost) from the authors' own group page.
a3	Do Context Files Help Coding Agents? A Two-Agent Ablation Study on Real Repositories	arXiv	2026-07	Independent July 2026 test on Claude Code (Sonnet 4.6) and Codex CLI (GPT-5.5) finding no detectable correctness effect from context-file strategies, with narrow process-level efficiency effects.
a4	Probe-and-Refine Tuning of Repository Guidance for Coding Agents	arXiv	2026-06	Summarises the split in the literature: Lulla et al. report 28.6% less runtime with curated AGENTS.md while Gloaguen et al. report lower resolve rates, with agents following file instructions too literally.
a5	Harness Engineering for Agentic AI Coding Tools: An Exploratory Study	arXiv	2026-02	First repository-level analysis (2,853 GitHub repos, eight configuration mechanisms, five tools) showing context files dominate and AGENTS.md acts as a cross-tool standard.
a6	Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents, A Source-Code Study of Eleven Systems	arXiv	2026-09	Source-code comparison of eleven harnesses (including Claude Code, Codex, OpenCode, OpenHands) showing convergence on markdown context files and Codex adopting Claude Code's hook and plugin formats.
a7	An Empirical Study of Harness Design for Coding Agents	arXiv	2026-09	Controlled harness ablation showing predefined tools help a weaker model while bash-only helps a stronger one (53% lower cost on SWE-Bench), evidence that scaffolding value depends on model strength.
a8	The Scaffold Effect in Coding Agents: Harness Choice as a Hidden Variable in Coding-Agent Evaluation	arXiv	2026-07	Holding the model fixed across Goose, OpenCode and OpenHands-SDK, harness choice moves tokens per solved task by up to 40x while pass-rate differences stay at 0 to 8 points (preliminary, ICML 2026 workshop submission).
a9	Scanning the Harness: An Empirical Study of Supply-Chain Defects in AI Coding-Agent Configurations	arXiv	2026-09	Scans 3,171 public repositories of harness components (skills, commands, context files, plugin marketplaces) and notes AGENTS.md is stewarded by the Linux Foundation's Agentic AI Foundation.
a10	SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks	arXiv	2026-02	First benchmark of Agent Skills (86 tasks, 11 domains, 7,308 trajectories) with no-skill, curated and self-generated conditions; curated skills lift pass rates but self-generated ones do not.
a11	SkillsBench summary (alphaXiv overview)	alphaXiv	2026-02	Gives the per-condition numbers: fleet-wide pass rate from 33.9% to 50.5% with curated skills, and drops of 8.1 to 11.5 points with self-generated skills (the arXiv version's headline averages differ slightly, 16.2 versus 16.6 points).
a12	SoK: Agentic Skills, Beyond Tool Use in LLM Agents	arXiv	2026-02	Systematisation paper that flags SkillsBench as a single, non-peer-reviewed source of quantitative skills evidence needing independent replication.
a13	An Empirical Study of Agent Skills' Downstream Utility	arXiv	2026-10	October 2026 study over 87 SkillsBench tasks and a 37,596-skill corpus showing the same skills help some model-harness configurations and hurt others on 36.78% of tasks.
a14	From Anatomy to Smells: An Empirical Study of SKILL.md in Agent Skills	arXiv	2026-07	Finds over 99% of SKILL.md files contain at least one authoring smell and that smells persist as skills evolve, quantifying the quality problem of unstructured markdown skills.
a15	An Empirical Study of Downstream Adaptation for Agent Skills	arXiv	2026-07	First study of how public skills are forked and adapted, and the source describing the SKILL.md structure (YAML front matter, Markdown body, bundled scripts and resources).
a16	Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale	arXiv	2026-01	Early-2026 large-scale audit of 31,132 skills from two marketplaces finding 26.1% with at least one vulnerability across 14 patterns.
a17	Malicious Agent Skills in the Wild: A Large-Scale Security Empirical Study	arXiv	2026-02	Behaviourally verified 98,380 community skills, confirming 157 malicious ones (632 vulnerabilities), with one actor behind 54.1% of cases and 93.6% removal within 30 days of disclosure.
a18	Technical Report: Exploring the Emerging Threats of the Agent Skill Ecosystem	arXiv	2026-05	Practitioner-authored report finding 13.4% of skills with critical issues and up to 9% of popular skills.sh skills exhibiting attack-vector patterns, though none confirmed malicious.
a19	SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skills	arXiv	2026-04	Evaluates a detector on 49,592 ClawHub skills and records that OpenClaw's marketplace audits found 13% to 26% of skills vulnerable, linking the OpenClaw skills market to the security literature.
a20	Towards a Risk Assessment of Malicious Skill Files in Coding Agents	arXiv	2026-08	Releases 2,826 adversarial skill files mapped to MITRE ATT&CK; also repeats an unverified claim of 162,000 GitHub stars for a popular skills repository in three months, a figure to trace before reuse.
a21	Metrics of Agent Ability	METR	2026-07	METR note (July 2026) on moving from binary time horizons to continuous, human-grounded metrics as the existing suite saturates.
a22	Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity	METR	2025-07	METR's RCT (16 developers, 246 tasks, mostly Cursor Pro with Claude 3.5/3.7) found a 19% slowdown; METR now marks it out of date and points to an early-2026 continuation, so it is a stale baseline.
a23	The Rise of AI Teammates in Software Engineering (SE 3.0): How Autonomous Coding Agents Are Reshaping Software Engineering	arXiv	2025-07	Introduces AIDev, the baseline measured dataset of agent-authored GitHub PRs (Codex, Devin, Copilot, Cursor, Claude Code); a snapshot to August 2025 and so predates the 2026 harness changes.
a24	Security in the Age of AI Teammates: An Empirical Study of Agentic Pull Requests on GitHub	arXiv	2026-01	Uses the AIDev curated subset of 33,596 PRs and shows the agent mix (Codex 21,799; Claude Code 459), a measured caution that PR-count data skews towards bot-labelled agents.
a25	Cross-Model LLM Code Review: Should you use Claude to review Codex or vice versa?	arXiv	2026-07	Tests writer-reviewer pairs across Claude Code (Opus 4.7) and Codex CLI (GPT-5.5) with cost accounting, direct evidence on verifier-pair multi-agent patterns.