Only EvoAgent beats the single-controller baseline across the tested task suite, while other workflows offer narrower gains.

BenchAgent compares complete agent workflows under shared evaluation conditions. It does not isolate the effect of adding another agent.

The main experiment evaluates six multi-agent configurations against BenchAgent Core across ten benchmarks covering reasoning, coding, reading and instruction following. Each configuration uses GPT-4.1 and completes three runs on the same 3,302 tasks.

Core maintains one task-level controller, although it can delegate web research to a search helper. The alternatives organise work through predefined roles, debate, feedback or evolving prompts. EvoAgent generates and selects alternative agent instructions for each task, then combines candidate answers.

The framework shares task inputs, available tools, scoring rules and limits on each Core execution. Workflows can invoke Core different numbers of times, so total token budgets are not matched. These are normalised implementations within BenchAgent, rather than evaluations of every original system in its native environment.

EvoAgent reaches 79.70% pooled accuracy against Core’s 76.65%, a gain of 3.05 percentage points. It beats Core in all three runs while using 1.27 times as many recorded tokens.

Every other multi-agent configuration trails Core in each run. Their resource costs differ substantially: CAMEL, AutoGen and Jarvis use fewer tokens, whereas LLM-Debate and ChatEval use more.

LLM-Debate consumes 1.39 times Core’s tokens while scoring 1.96 percentage points lower. ChatEval consumes 4.34 times as many tokens while also scoring lower.

Pooled accuracy counts correct answers across all tasks, giving larger benchmark subsets more weight. Token totals include recorded input and output usage across workflow calls. The reported accuracy variation measures differences between repeated runs on fixed tasks, rather than uncertainty across new task samples.

LLM-Debate performs best on HumanEval, where generated Python code must pass supplied tests. Its 94.66% accuracy exceeds Core’s 84.99% by 9.67 percentage points.

ChatEval leads on IFEval, which requires a response to satisfy every instruction attached to a prompt. It reaches 85.33% against Core’s 63.08%, a gain of 22.25 percentage points. Yet it scores only 4.44% on AIME mathematics problems, against Core’s 48.89%.

EvoAgent and LLM-Debate each beat Core on five of the ten benchmarks, but only EvoAgent improves pooled accuracy. The size of each gain and the number of tasks behind it matter.

The ranking also changes on GAIA, which tests retrieval, file inspection and tool use. In its separate GPT-4.1 run, Jarvis leads BenchAgent configurations at 46.67%, against Core’s 37.58% and EvoAgent’s 36.36%. With Qwen3-32B, every tested multi-agent configuration trails Core on GAIA; both backend comparisons use only one run.

More discussion sometimes ends with an incorrect answer even when an earlier role answered correctly. The authors examine 371 tasks from one ChatEval GSM8K arithmetic run with directly parseable initial numeric answers.

Between the first Math Expert response and the final output, 36 answers change from correct to incorrect. Only six change from incorrect to correct.

In one fruit-count example, the first Math Expert and later role responses give the correct answer, 12. The final prediction nevertheless becomes 10. This identifies a concrete need to check whether correct answers survive final aggregation and extraction.

The analysis excludes tasks without parseable initial answers and covers one workflow run. It does not establish a general failure rate or show that debate itself caused these errors.

A separate Claude Code workflow records higher GAIA accuracy with fewer tokens, but its execution conditions differ from BenchAgent’s. This configuration also uses GPT-4.1, routed through a translation layer, rather than a Claude model.

It answers 110 of 165 tasks correctly, reaching 66.67% accuracy with 52,984.69 recorded tokens per task. Jarvis, the strongest BenchAgent configuration in that run, reaches 46.67% with 332,285.96 tokens per task.

Retained traces show agent creation, messages between roles, saved evidence and verification before writing final answers. These observations suggest mechanisms worth testing, including preserving evidence and checking answers before release.

The runtime, batching and execution limits differ, and resource budgets are not matched. The study therefore cannot attribute the accuracy difference to delegation, verification or context management. Controlled component tests and repeated GAIA runs remain necessary.