Is a "Five-Agent" Team Really Worth It? Agent Teams Cost Up to 5.1× More Than a Single Agent—And Scores Barely Moved

Reorganize a coding agent into "a team where a lead designs and subordinates do the hands-on work," and you'll get faster, better results than running a single agent—or so the hope goes. Vals AI, a firm specializing in evaluation, just threw cold water on that expectation with hard numbers. They pitted GPT-6 Sol and Claude Opus 5.5 against the same tasks in both single-agent and team configurations and lined up the cost, time, and scores. The result: teams paid 1.8 to 5.1 times the cost of a single agent, yet their scores improved meaningfully in only one of four comparisons.

Running "build an entire web app from a spec" 50 times each

The study used the firm's own "Vibe Code Bench." You hand it a product spec, have it build a complete web app—database, API, UI, and deployment included—then actually spin it up with Docker Compose and score it by what percentage of browser-based UI tests pass. The key is that it measures "did a working app come out?" rather than "did it write plausible-looking code?"

The comparison was between a single agent that sees the job through from start to finish, and a team made up of one lead plus up to five subordinates. They ran this for both GPT-6 Sol and Claude Opus 5.5, varying the reasoning intensity (medium / max), across 50 apps each.

Across four comparisons, the team clearly won only once

The results were blunt. The only case where the team beat the single agent by a statistically significant margin was Sol on medium as a team, with a UI test pass rate of +7.3 points (p=0.005). The other three differences ranged from −0.3 to +3.4 points—all within the margin of error (p-values of 0.16 to 0.81). As for Opus, neither teaming it up nor turning up the reasoning produced any meaningful difference in scores.

The bill, on the other hand, clearly went up. Here is the cost and time per app:

Source: reconstructed from figures in Vals AI, "Do Agent Teams Pay Off?" (October 9, 2026)

As a multiplier, even the lightest case, Sol / max, was 1.8×, while the heaviest, Opus / max, hit 5.1×. For Opus, teaming up even ballooned the time by up to 2.3×. The intuition that "adding more heads gets it done faster and cheaper" simply does not hold here.

Inside the teams, the two models worked in completely different ways

What's intriguing is that even within the same "team" setup, how the lead ran things diverged by model.

Sol: split first, run in parallel, then verify it yourself

Sol's lead carved the work along feature boundaries within the first few minutes and ran four to five subordinates in parallel. It then verified the output in the browser itself (in 46 of 50 runs), catching bugs at the seams where separately built parts joined together. With max reasoning, this parallelization paid off, shrinking the time from 38 minutes to 28.

Opus: write the "contract" first, then hand off in sequence

Opus's lead preferred to first write out a shared spec (a contract) for how the parts would mesh (in 40 to 42 of 50 runs), then pass the work along in order—builder, then feature developer, then tester. It also spent heavily on post-deployment verification (67.8 agent-minutes on testing with max reasoning, about 7× the 9.6 minutes of the single agent), and as a result the team took longer than a single agent.

Before "adding people," consider "strengthening the mind"

One practical takeaway emerges from this study: when you want to raise scores, there are cases where turning up the reasoning power works more directly than teaming up. With Sol, a single agent on max outscored a team on medium. But that isn't free either: a single Sol / max agent ($4.82) cost about 1.6× the money and about 3× the time of a Sol / medium team ($3.07).

Relatedly, reports say that in Anthropic's own parallel testing, scaling Opus 5.5 from 10 agents to 100 only increased token consumption while scores plateaued. That's an observation pointing in the same direction: "just crank up the parallelism" is no cure-all.

On the flip side, this doesn't mean teams are categorically useless. On tasks and models where parallel splitting clicks—as with Sol—the payoff showed up as reduced time. Vals AI itself concludes that "whether a team pays off depends on the model, the task, and the budget. There's a path to gains through teams, but it should be weighed against the path of strengthening reasoning."

The more hands-off the operation, the more a "purse string" matters

On the business side, what you can't overlook is how quietly costs creep up. The setting to scale agents in parallel takes just a few lines, yet the bill jumped from $23.77 to $122 per app in the Opus / max example. In a hands-off operation where no human approves each step, making "teams the default" risks piling up costs several times over while quality stays essentially unchanged. At rollout, the safe order is to first set a baseline with a single agent plus reasoning-level tuning, then limit teaming to tasks where parallelization pays off—measuring the cost-effectiveness before expanding.

Note that these results measured a single type of benchmark—web app construction—across two models and 50 runs each. A different problem domain or model could paint a different picture, so you can't generalize to "teams always lose." The shortcut to knowing whether it works in your own workflow is to measure it on a small scale and see.

References: Vals AI: Do Agent Teams Pay Off? A Case Study on Vibe Code Bench / The Decoder: AI agent teams waste massive tokens for barely measurable quality gains / AlphaSignal: Vals AI Finds Multi-Agent Coding Teams Cost 5x More but Rarely Score Better / Vibe Code Bench Leaderboard and Methodology