Letting a Swarm of Agents Train Themselves: Sakana AI's MASS Delivers Up to 1.6x on Research Tasks—But Coding Didn't Improve

Claude Code and Codex splitting a big job into multiple sub-agents that run in parallel is now a familiar sight. So the question becomes: could you use how that "swarm of agents" performs as training material to retrain the underlying model itself? MASS (Multi-Agent Self-Supervision), released on October 12, 2026 by Sakana AI in collaboration with UC Berkeley, tackles that question head-on. It showed promise on research tasks, but the candid results also revealed that it didn't work on the crucial domain of coding.

One model plays executor, orchestrator, and grader all at once

At the heart of MASS is a single shared model. This one model takes on three roles, switching between them.

• Executor: solves the actual tasks as a sub-agent on the team.
• Optimizer: searches for which team compositions and procedures work well.
• Evaluator: compares the outputs of multiple teams and picks the best one.

Here is how it flows. The optimizer leans on feedback from the evaluator to find good ways for a team to proceed, and the team's execution logs then become training material to retrain the executor (the same model). The updated model again takes on all three roles and runs the same loop. Continuously distilling the collective experience of the teams into a single model—this is the essence of "recursive self-improvement." The base was a 27-billion-parameter-class open-weight model, and two cycles of improvement were run.

On research benchmarks, 1.2x to 1.6x—and with fewer tokens

The gains came on four research-oriented benchmarks. According to Sakana AI, all of them improved in "score per token" over the base model.

The efficiency numbers are suggestive, too. Trajectories gathered by multiple agents recorded a 68.3% win rate against the case of using a single agent's trajectories as training material. What's more, training used 28.8 million tokens—about 31% fewer than the 42 million used by the single-agent approach. For now, the numbers back up the premise that "just as you would add more people, splitting the same model across multiple roles lets you produce better training material on less fuel."

But coding didn't get better

This is the crucial point for developers. While research tasks improved, the gains did not carry over to software development benchmarks. Even after two cycles of self-improvement, the report says scores on SWE-bench Verified and Terminal-Bench 2.0 fell below the starting base model.

In other words, MASS as it stands has not demonstrated that "bundling agents together and having them self-train will likewise raise coding performance." Sakana AI itself mentions a possible fix of adding coding tasks to the training data, but that verification is still ahead. Talk touting multi-agent self-improvement is on the rise, but it's reasonable to conclude that reading research-oriented numbers directly as productivity in a development setting is premature.

The pitfall of "grading with the same mind"

There is one more point Sakana AI itself flags as a caution. Because the executor and the evaluator are the same model, shared blind spots tend to arise. The kinds of errors the model overlooks, it also can't catch in its role as evaluator. Those errors can then be mistaken for "progress" and reinforced through training.

A design that runs self-improvement among homogeneous models carries this kind of risk. Sakana AI says the safety of this "homogeneous self-improvement" should be examined carefully by the community. For anyone considering operations that run agents autonomously, it's worth keeping in mind as the question of how to ensure diversity in grading (for example, cross-checking with a different family of models or with tests).

What's come into view

What MASS demonstrated is a concrete, numbers-backed path to "turning the collaboration of agents itself into training material for the next generation of models." At the same time, the same report honestly disclosed that gains on research tasks don't transfer cleanly to coding, and the danger specific to homogeneous self-improvement. It's still an early stage of just two cycles, and long-term reliability and real-world operating costs are unknowns. This is a method to watch for follow-up reports on, without excessive hope or excessive disappointment.

References: Sakana AI, "MASS: Recursive Self-Improvement through Multi-Agent Self-Supervision" / MASS technical page / AI Weekly / Superpower Daily / Runtime Wire