Twelve language models managed the same subagent team, but none granted workspace permissions with better than 50% precision.

ClawArena-Team measures a single language model acting as a manager rather than a solo problem-solver. The main agent perceives only text and can reach only part of the workspace directly, so it must delegate to subagents to finish any task.

Every manager commands the same fixed pool of three locally served workers: an LLM and a VLM (vision model) both backed by gemma-4-31b-it, and an OMNI audio-capable model backed by gemma-4-e4b-it. Because the workers never change, score differences between runs reflect management skill rather than worker capability.

The authors frame this as a principal-agent problem: the manager (principal) must get work done through subagents (agents) whose inputs it cannot consume itself. They decompose the manager's job into six operations.

The six management operations every scenario forces the main agent to exercise:

• CreationThe manager writes a system prompt, picks a model key, chooses a tool subset and sets a path whitelist that must be a subset of its own access.
• Modality routingImages and video must go to a VLM subagent and audio to an OMNI subagent, because the manager cannot read them.
• Least-privilege empowermentTool and path grants should match what each subagent actually needs. Over-granting is treated as both wasteful and unsafe.
• SchedulingSubagents can run in the foreground or background, as new or continued sessions, and in parallel, with background tasks notifying the manager on completion.
• Dynamic workflowsA programmatic workflow tool lets the manager author parallel and pipeline stages over the same pool at runtime.
• IntegrationOnly the manager's fused deliverable is scored, so faithfully relaying correct subagent reports can still fail the round.

Scoring is execution-based: each round ships a shell command whose exit code, with optional output matching, decides pass or fail. No LLM judge is used, which the authors say avoids both rejecting legitimate alternative answers and judge drift.

The composite Subagent-Management Score (SMS) multiplies task correctness by the average of four management components. Because that management factor lies between 0 and 1, SMS can never exceed task correctness. Sloppy management discounts a correct answer, and perfect management of a failed task scores zero.

Workspaces are large and mixed: 170.5 MiB and 28.9 million tokens across text, code, office documents, images, audio and video, spread over at least eight top-level directories, some of which are decoys. Of the 258 rounds, 44 are preceded by staged updates that change the correct answer to later questions.

The five per-run components, all reported in the range 0 to 1:

• TCR (Task Completion Rate)The mean pass rate over user questions.
• TPP (Tool-Permission Precision)Per subagent, the fraction of granted tool types that were actually used.
• ROC (Read-Only Compliance)Zero if a read-only subagent was handed a mutating tool, otherwise one.
• WPP (Workspace-Permission Precision)Per subagent, files accessed divided by files granted.
• MCA (Modality-Choice Accuracy)Whether a VLM actually reads image or video and an OMNI reads audio; a plain LLM subagent scores one.

claude-fable-5 leads on SMS at 60.0% and on task completion at 74.4%, with gemini-3.5-flash second on SMS at 53.8%. The best open-weight manager is glm-5.2 at 50.4%, fourth overall and ahead of kimi-k2.6 at 49.0%.

The twelve-model SMS range is 44.7 points, but it concentrates in two gaps: a 6.2-point jump from runner-up to flagship, and a 28.6-point cliff from gemma-4-31b (43.9%) down to glm-4.7-flash (15.3%). The ten models in between sit within a 9.9-point band, from 43.9% to 53.8%.

Even the leader leaves 40% of the available score on the table. Every other manager loses roughly half its score to imperfect task correctness and privilege management.

The two easy management axes are nearly saturated. Read-only compliance and modality-choice accuracy are both at least 92% for all eleven capable models, because each encodes a clear-cut decision. The discriminating axes are the two privilege-precision metrics: tool-permission precision sits around 70 to 80%, and workspace-permission precision never reaches 50%.

Cost and management quality are decoupled. Cost spans over 100 times, from $0.8 to about $93 per run, while SMS spans under 4 times. claude-fable-5 buys the top score at roughly four times the cost of gemini-3.5-flash for six extra SMS points. Below the top, gpt-5.5 ($43.3), claude-sonnet-4-6 ($39.7) and kimi-k2.6 ($28.7) are each dominated by gemini-3.5-flash, while deepseek-v4-pro reaches 46.4% at $1.7 thanks to a steep cache-read discount.

Scores converge while behaviours diverge. Among capable models the mean forbidden-access rate per subagent ranges from 0.48 (gpt-5.5) to 5.78 (deepseek-v4-pro), with glm-4.7-flash at 11.44. Dynamic-workflow use ranges from 8 to 112 invocations, and background runs range from 8 (glm-4.7-flash) to 119 (claude-sonnet-4-6). Managers overwhelmingly default to the LLM key and create few vision or audio specialists, so routing accuracy is high mainly because routing is rare.

No scenario is solved by every model, and the full-pass rate (every round in a scenario correct) is near zero for all of them. The authors read this as evidence that sustained management across an evolving, multimodal workspace is the hard part, not any single round.

Three recurring failures emerge from the case studies. Managers take text shortcuts on modality-decoy rounds, trusting a plausible but flagged transcript instead of delegating to a vision or audio worker. Stale beliefs persist across staged updates, with a correction acknowledged in conversation but never written into the deliverable. And over-granting does not buy correctness: subagents simply receive paths they never read and tools they never call.

The authors note several limits. The fixed worker pool ties the findings to one level of worker capability, and a stronger or weaker pool could change what good management looks like. Execution checks can reject correct reasoning that does not match an expected literal form. claude-fable-5 is scored as shipped with a refusal fallback to claude-opus-4-8, which fired 124 times, concentrated in one security scenario, so its result is a composite rather than the base model alone. All token counts and costs are post-hoc estimates from a single tokenizer rather than provider bills.