Over the past two years, much of the AI industry has bet on increasingly elaborate architectures for its agents: orchestrators that hand out tasks, subagents dedicated to retrieving information, search trees over candidate solutions and teams of agents that critique one another. A new study shared by Apple Machine Learning Research questions whether all that machinery adds value once the underlying model is already strong.
The paper is titled “How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?” It was posted to arXiv with a date of September 30, 2026, and drew attention in early October. Its authors are Kirill Brilliantov and Alejandro Hernández-Cano, as equal contributors, and Emmanuel Abbé, all affiliated with EPFL; according to the paper itself, part of the work was done during an internship at Apple.
In industry jargon, a harness is the software that wraps a language model so that it can act: tools, memory, planning loops and, in the most complex cases, several coordinated agents. The study's question is simple: if a modern model can already read files, write code, run it and debug errors on its own, which parts of that scaffolding are still useful?
To answer it, the researchers built Malena, a baseline they describe as a machine learning engineer agent. It is a single coding-agent session, built on the open OpenCode environment, with a minimal set of primitives: read, write and run shell commands. On top of that sit three lightweight tools: one to register submissions, another to check resources and remaining time, and a third to launch background jobs.
The comparison was run under matched conditions: same backbone model, same hardware and same time budget. Each agent had one 80 GB A100 GPU, 12 CPU cores and 144 GB of RAM. The main testbed was MLE-bench, which measures agents' ability to solve machine learning competitions derived from Kaggle, using a subset of 30 tasks and a 24-hour budget. The team also used NatureBench, with 40 scientific research tasks and an eight-hour budget.
Malena was pitted against four open-source systems designed specifically for this work: MLEvolve, AiScientist, Arbor and ScienceFlow. With GLM 5.2 as the backbone, Malena earned a medal on 62.5% of the competitions, compared with 47.1% for the best external system tested, AiScientist. That is a gap of 15.4 percentage points in favor of the option with fewer moving parts. On NatureBench, Malena and AiScientist were practically tied with the Kimi K3 model.
According to the paper, across 17 system-and-model pairings on MLE-bench and six on NatureBench, Malena matched or beat every rival with frontier backbones. The only partial exception appeared with the smallest model, Gemma 4 31B, where MLEvolve came out ahead on the mean estimate, although its confidence intervals overlapped with Malena's. The authors read this as a sign that weaker models still benefit from hand-designed structure.
The study does not stop at the final comparison. The researchers re-implemented the typical interventions of these systems in a single codebase and added or removed them one at a time. The most striking finding is that giving the model a real coding environment, rather than a plain chat interface, is by far the largest effect they measured. Beyond that, no search strategy, subagent delegation or parallelism produced statistically significant gains.
In some cases, complexity even made results worse. In one of the ablations, adding subagent delegation lowered Malena's medal rate from 55.7% to 45.2%, and combining three parallel agents with a tool to message one another dropped it to 33.3%. The authors note that properly coordinating strong agents toward a common goal remains an open question.
The trace analysis offers an explanation. A capable agent carries out on its own the search that other systems hard-code: it builds a general pipeline, reuses checkpoints and utility code, and keeps testing patches much as if it were tuning hyperparameters. According to the paper, Malena even independently rediscovered techniques popular in the Kaggle community, such as pseudo-labeling, with two different backbone models.
There are important caveats. Malena's inference cost was higher in some cases: with GLM 5.2, its modeled cost was about $12.12 per run, roughly 6.2 times AiScientist's, because its single session resends its entire context every turn. Even so, the authors point out that total cost is dominated by hardware: 24 hours on an A100 can cost between $24 and $85 depending on the provider.
The researchers acknowledge limitations themselves. Compute budgets restricted the number of repeated runs with different seeds, leaving some confidence intervals wide. MLE-bench, because it is built from public Kaggle competitions, is a plausible target for data contamination, which they tried to mitigate by fixing broken tasks and adding NatureBench. They also concede a possible asymmetry in tuning effort between Malena and the external systems.
The work adds to other evidence pointing in the same direction. Cryptobriefing noted that a July 2026 study found self-organizing multi-agent teams performed nearly 41.1% below their best individual expert agent on machine learning benchmarks. The paper itself cites software engineering research that found no significant improvement across 35 harness releases on a fixed model on SWE-bench.
For people building AI products, the practical takeaway is clear: before investing in complex multi-agent architectures, it pays to compare against a simple baseline using the same model. For research, the study raises a methodological flag: if improvements are reported without matching the backbone model, what looks like architectural ingenuity may simply be a more powerful model. As the authors conclude, the leverage lies in the model and in the environment it is given, not in the scaffolding built around it.
