Nvidia says the same model scores 30% or 100% on ARC-AGI-3 depending on its harness
Claude Opus 5 cleared all 183 levels wrapped in Nvidia's AVO system, against 30% on its own — a vendor-run result on a public benchmark, not yet independently reproduced.
Original cover art, generated for this story. THE VISSION does not republish third-party press imagery.
- Nvidia reports that Claude Opus 5 inside its Agentic Variation Operators harness scored 100.00 RHAE across all 25 public ARC-AGI-3 environments, completing all 183 levels.
- The same model scored about 30% without the harness, according to Nvidia's account — the model was not retrained or changed.
- AVO pairs a main agent that inspects, plans, implements and evaluates using persistent memory with a supervisor that redirects it when progress stalls.
- It used 6,624 environment actions against VISTA's 7,542 on the same levels, roughly 12% fewer, though Nvidia says this was not a controlled ablation because the two systems differ architecturally.
Nvidia has published a result that puts an uncomfortable number on how much of an AI agent's performance comes from the model and how much from the software around it. Running Claude Opus 5 inside its Agentic Variation Operators system, the company reports a 100.00 RHAE score across all 25 public environments of ARC-AGI-3, completing all 183 levels. The same model, Nvidia says, scores roughly 30% on the benchmark without that scaffolding. Nothing about Opus 5 was changed between the two figures.
ARC-AGI-3 is an interactive reasoning benchmark built from 2D games supplied without instructions, so a system has to work out the rules by playing. AVO's structure is a loop with a second opinion attached: a main agent inspects the current state, plans a move, implements it and evaluates the result, carrying implementations and reasoning forward in persistent memory, while a separate supervisor watches the wider trajectory and intervenes when the search stalls. Adel El Hallak, a vice-president of product in Nvidia's AI unit, described the harness to TechCrunch as "the scaffolding around the model...the set of tools that it utilizes" together with "the runtime and the associated skills and libraries."
Nvidia's own summary of the finding is the blunter version: "The model matters, but the model is not the entire agent." Against VISTA, the previous leading system on the same 183 public levels, AVO used 6,624 environment actions to VISTA's 7,542 — about 12% fewer. Nvidia is explicit that this is not a controlled ablation, because the two systems differ in architecture and in how they represent observations, so the action-count gap should not be read as a like-for-like efficiency measurement.
The caveat that matters most is one the number itself cannot resolve: this is a hardware and platform vendor reporting its own system's score on a public benchmark, and no independent party has yet reproduced it. The benchmark is public and the accompanying paper is on arXiv, so reproduction is possible; until it happens, 100% is a claim about AVO made by the company that built AVO. Nvidia also ran limited experiments pairing AVO with GPT-5.6 Sol on harder subsets, which suggests the harness is not tuned to a single model, though it has not published comparable full-set figures for that pairing.
If a 70-point swing on a reasoning benchmark can come from scaffolding alone, then most published model comparisons are measuring the evaluation harness as much as the model, and a team choosing between vendors on benchmark tables is partly choosing between other people's plumbing. The practical read for anyone building agents is that persistent memory and a supervisor loop are worth more engineering attention than the next model upgrade — and the commercial read is that Nvidia, which sells the hardware underneath all of it, has an obvious interest in the industry believing exactly that.