Future gains in long-horizon agents depend as much on the system built around the model as on the model itself

The paper's central claim is that agentic AI should be studied as a system-scaling problem rather than only a model-scaling problem. Once a foundation model is wired into tools, terminals, repositories and memory stores, its behaviour depends on how context is constructed, how memory is retrieved, how tools and subagents are invoked, and how actions are verified. The author calls this surrounding layer the harness, and calls the agenda of improving it scaling the harness.

The framework decomposes an agent into six interacting components and writes performance over a horizon as a function of all six: a reasoning substrate (the foundation model), a memory store, a context constructor, a skill-routing layer, an orchestration loop and a verification-and-governance layer. Model scaling improves the first; system scaling improves the other five. The author is explicit that this is a conceptual organisation, not a quantitative model. The function has no closed form and the factors are not strictly independent. The claim is only that each factor names a distinct point of intervention that can be changed, switched off or measured on its own while keeping the same model.

To make the levers concrete, the paper compares three harnesses built around comparable frontier models: Claude Code (v2.1.88), OpenClaw (v2026.4.6) and the author's own Python reference harness, CheetahClaws (v3.05.79). The point is not to rank them. All three address context governance, memory management and skill routing, which the author reads as evidence that these are intrinsic design problems rather than quirks of any one implementation. Their differences follow deployment priorities rather than model choice: reliable use for a vendor coding agent, multi-channel access for a personal assistant, and transparency for a research tool. One concrete divergence is trust representation. CheetahClaws stores per-entry confidence and recency as fields used directly in retrieval ranking, while the other two derive trust implicitly from access patterns.

The paper singles out three of the six components where bigger models alone have been least sufficient: context construction, memory and skill routing, each tightly coupled to verification and governance. For each it names four sub-axes, a characteristic failure mode and the system-level move that addresses it. The memory and skill failures are described as symmetric: both let the agent act on a claim whose truth was never re-established.

A worked example from the memory section shows the asymmetry the author worries about. A note saying the data loader lives in utils/loader.py can become flatly wrong after a refactor without any change to its wording. Semantic search still ranks it highly, but acting on it now means calling a deleted symbol or reintroducing a fixed regression. Stale memory rarely prevents retrieval, but regularly leads the agent to act confidently on an invalidated assumption. The proposed fix is to make trust a runtime decision: weight a staleness penalty and a confidence-gated risk term alongside relevance, and treat retrieved content as a hypothesis until re-checked against the live environment. Claude Code is cited as realising this coupling by pairing a persistent CLAUDE.md file with just-in-time file reads, glob and grep over the live repository.

The three bottlenecks, each with its four sub-axes, the failure it guards against and the system move the paper proposes:

• Context governanceSub-axes: relevance, compactness, traceability, refresh. Failure mode: exposure without access, where the model sees more tokens but does not attend to the right ones because relevant evidence competes with padding. System move: treat each turn's context as the output of a selection policy rather than a fixed buffer, with persistent priors plus just-in-time refresh and recorded provenance.
• Trustworthy memorySub-axes: precision, durability, retrievability, verifiability. Failure mode: stale-but-confident, where a once-correct note has silently drifted from its target. System move: re-establish trust at retrieval and periodically verify memory against the environment. Durable memory without verification accumulates drift, while environment-only search without distilled priors discards every prior verification.
• Dynamic skill routingSub-axes: specificity, selectivity, composability, verifiability. Failure mode: confident-but-unchecked, where a specialised subagent returns plausible output that no downstream layer validates. System move: treat routing as a learned, adaptive policy rather than a fixed rule set, and make post-condition checking a first-class part of every skill specification. The author likens routing to scheduling in an operating system: capacity exists, but useful work depends on allocating it to the right pathway at the right time.

Alongside the six-component view, the paper reads prompt, skill and memory as three temporal layers of system scaling. A prompt is the short-horizon control interface: flexible and cheap, but brittle, and unable by itself to create persistence or transfer. A skill is a reusable execution pattern, which may be a workflow template, a tool-use routine, a specialised subagent or a versioned bundle of instructions and scripts. Skills make behaviour reusable but introduce the routing problem of which skill to invoke, when to switch and how to compose several in one trajectory.

Memory is the longitudinal layer, storing project conventions, user preferences, stable facts about the environment and prior failures. It fails along the three trust axes above: drift (loss of durability), over-generalisation (loss of precision) and pollution (loss of verifiability). The author's summary is that prompt controls what to do now, skill controls how to do this class of things, and memory controls what should survive over time. A well-built agent needs all three rather than any one.

The paper credits current benchmarks such as SWE-bench, AgentBench, WebArena and Terminal-Bench with moving evaluation toward multi-step execution against real artefacts, but argues they remain insufficient for system-scaled agents. A single score mixes model capability with harness design, so a reported model score is often a model-plus-harness score. The author cites SWE-agent, where redesigning the agent-computer interface alone improved SWE-bench accuracy with a fixed model, and a field-level analysis finding that many agent results stop being cost-optimal once prompting strategy and demonstrations are controlled. Endpoint metrics also hide cost and risk: two agents may both solve a task yet differ substantially in tokens, tool calls, retries, failed edits and human interventions.

Multi-agent systems are the paper's illustration of why single-episode success is not enough. It cites Anthropic's report that a multi-agent system with a Claude Opus 4 lead and Claude Sonnet 4 subagents outperformed single-agent Claude Opus 4 by 90.2% on an internal research evaluation, and that in a BrowseComp-based analysis token usage alone accounted for 80% of performance variance, rising to 95% once tool-call count and model choice were added. The author reads these figures as evidence that performance is strongly shaped by how a system allocates computation. The same section stresses that decomposition is easier than collaboration, and cites failure analyses attributing many multi-agent failures to system design, inter-agent misalignment and inadequate verification rather than to model limits.

The broader gap is that benchmarks reset the agent between tasks while real agents accumulate state across sessions. The same accumulation that enables improvement can produce contamination, staleness, over-generalisation and privacy leakage, none of which a one-shot evaluation can reveal. The paper proposes reporting outcome metrics jointly with process metrics, and lists the repeated-use properties that the next generation of benchmarks should add. It also proposes an agent evolution standard built around four questions: what persists, what updates, what is measured and what is auditable. Without such a standard, the author warns, learning agents risk becoming opaque accumulations of prompts, notes and heuristics.

The author engages three objections directly and concedes part of each. The paper does not claim model scaling has stopped mattering. Its stated position is that once models pass a capability threshold, additional long-horizon gains increasingly depend on system design. The framework is offered as a conceptual organisation with no closed-form equation, the three harnesses are compared qualitatively rather than benchmarked, and the manuscript describes itself as under active development.

The three objections and the paper's responses:

• Stronger models will solve system problemsThe author agrees model scaling will keep improving agents, but argues that stale memory, over-broad tool permissions, missing provenance, unverified retrieval and unsafe action execution are system failures. A stronger model may reduce their frequency but does not remove the need for mechanisms that decide what is exposed, what is authorised and how failures are traced.
• End-to-end training will replace modular systemsEnd-to-end training may improve coordination between components, but deployed agents operate over private files, credentials, repositories and external services, where auditability, permission control, rollback and provenance are required. An end-to-end policy still has to act through the same permission, verification and audit surfaces.
• System-level evaluation is too expensive or environment-specificThe author accepts that trace-level metrics are harder to standardise than endpoint accuracy, but argues this is precisely why they are needed, since cost, latency, tool risk, memory drift and verification overhead determine whether a deployed system is usable. The goal is to complement simple benchmarks, not replace them.