An agent rewrites its own coding harness, and beats the engineers who used to do it by hand

A coding agent is more than its language model. Around the model sits a harness: the system prompt that shapes its work style, the tools that expose the shell and file system, the middleware that manages context and recovery, and the long-term memory it carries between tasks. On long-horizon software tasks this scaffolding moves the score as much as the model does, even when the model is held fixed.

The catch is that harness engineering is hand work. Developers read trajectories, spot recurring failure patterns, and hand-craft edits across prompts, tools, and middleware. Worse, the best harness is model-specific, so every new base model needs a fresh round of manual tuning. As models ship faster, that manual loop falls behind.

The authors argue the bottleneck is not the cleverness of an automating agent but observability: give an evolution agent a clear action space and structured evidence, and it can converge on better designs on its own. Agentic Harness Engineering (AHE) is their attempt to prove it.

AHE is a closed loop in which one agent rewrites another agent's harness while the base model stays frozen. It rests on three matched forms of observability, each turning a messy part of the problem into something an agent can actually read and act on.

1. Component observabilityThe harness is decoupled into seven editable component types exposed as files at fixed mount points (system prompt, tool description, tool implementation, middleware, skill, sub-agent, long-term memory). Each failure pattern maps to one component class, and each logical edit is one git commit, so changes are localized and revertible.
2. Experience observabilityAn Agent Debugger distills roughly 10 million raw trajectory tokens into a layered, drill-down evidence corpus of about 10 thousand tokens. The evolver reads structured root causes and a benchmark-level overview rather than raw logs, with original traces available on demand.
3. Decision observabilityEvery edit ships with a self-declared prediction in a change manifest: the failure evidence, root cause, targeted fix, expected fixes, and at-risk regressions. The next round intersects those predictions with observed task-level deltas, so each edit becomes a falsifiable contract that is rolled back at file granularity if it fails.

A single ten-iteration campaign from a deliberately minimal bash-only seed (NexAU0) finished in about 32 hours and topped every baseline on the panel: three human-designed harnesses and two self-evolving loops, ACE and Training-Free GRPO, all starting from the same seed.

The gap to the prompt-only methods is a layer mismatch. ACE distills natural-language playbooks and TF-GRPO reinforces tool sequences, but neither opens the surrounding scaffolding to edits. AHE jointly evolves prompt, tools, middleware, and memory, and the gain concentrates in exactly the layers the others leave untouched. The one soft spot is the Hard tier, where AHE marginally trails Codex because its own components interfere on the longest tasks.

If the harness only encoded Terminal-Bench tricks, it would not survive a move. It does. Dropped unchanged onto SWE-bench-verified, AHE posts the highest aggregate success while spending 12% fewer tokens than the seed, 21% fewer than TF-GRPO, and 32% fewer than ACE. The two prompt-only baselines actually regress below the seed here, because the text they inject rides every model call and adds cost without reshaping behavior.

Cross-model transfer tells the same story. Re-evaluated on five alternate base models with no further evolution, the harness lifts pass@1 everywhere, from +2.3 pp on GPT-5.4 to +10.1 pp on deepseek-v4-flash. The cross-family gains are the largest, which the authors read as weaker bases leaning more heavily on coordination patterns AHE has fixed inside tools, middleware, and memory.

A component ablation localizes the value. Swap in memory, tools, or middleware alone and each beats the seed on its own. Swap in only the system prompt and it regresses by 2.3 pp. Factual harness structure transfers; prose-level strategy does not. The three positive single-component gains sum to +11.1 pp against full AHE's +7.3 pp, so the components interact non-additively rather than stacking cleanly.

The loop's self-attribution is sharp in one direction and dull in the other. Its fix predictions land roughly five times above chance, so each edit targets a real, anticipated failure rather than an arbitrary task. But its regression predictions barely clear the random baseline. Across nine rounds the agent issued 43 regression predictions and only 5 landed, while 40 regressions it never foresaw actually happened.

That blindness is exactly what produces the non-monotone dips in the evolution curve: the agent can justify why an edit should help, but cannot reliably name what the same edit is about to break. The authors flag regression foresight as the clearest direction for future self-evolution loops, and are candid that AHE is a controlled research prototype, with operating-point coupling and incomplete guardrails still on the table.