OCOPD SoL-Pi reduces token use on EdgeBench
Nvidia researchers say SoL-Pi cut coding-agent token use by 44.7% to 49% on EdgeBench’s 51 public tasks, compared with the original Pi harness. The most token-efficient configuration achieved 93.7% of Pi’s score, according to the researchers. The work targets the harness, the software layer that controls how an agent interacts with a model and its coding environment, rather than changing the model itself.
The benchmark results show a trade-off between token efficiency and task performance. The configuration with the largest reported token reduction used 49% fewer tokens than Pi but scored below the baseline. A separate variant, using the strongest single mechanism, scored 5.3% above Pi while reducing token use by 6.1%.
Nvidia’s research process tested 152 approaches across more than 3,000 runs, according to the news report. Those figures describe the system-development effort; they are not the size of the final EdgeBench evaluation, which comprised 51 public tasks.
Harness changes target agent overhead
Coding agents can make many model calls as they inspect a repository, use tools and respond to test results. SoL-Pi seeks to reduce token consumption in that interaction loop by changing the harness that manages the model’s work. This is a different efficiency lever from reducing the price of each token or modifying the underlying model.
The reported comparison uses Pi as the baseline and measures token use and task scores on EdgeBench. The available reporting does not specify a complete breakdown of token categories or enough detail to independently reproduce the benchmark. It also does not establish a single, general API-cost reduction: the researchers’ dollar estimates depend on API prices and the comparison harness. They estimate savings of $8.75 to $13.50 per hour against native Codex and Claude Code harnesses, and $4.36 to $5.71 per hour against Pi, using the API prices cited in their analysis. Those estimates should not be treated as provider-independent cost figures.
Results vary by model and benchmark
The researchers developed SoL-Pi using the model labeled GPT-5.6 Sol in the report, then applied it to Opus 5 without further optimization. On Opus 5, they report that SoL-Pi retained 94.3% of Pi’s performance with similar token savings. The report says the mechanisms activated less often and less aggressively on that model.
The researchers attribute that difference to optimizing the harness using GPT-5.6 Sol trajectories. That is their explanation for the observed cross-model variation, not a separately established causal finding. The report also says results were less consistent on other benchmarks, limiting what can be concluded from EdgeBench alone.
Efficiency gains remain configuration-dependent
The findings suggest that changes to an agent’s control layer can reduce token use without retraining its model. That could be relevant to teams running coding agents for extended tasks, where repeated model interactions contribute to operating costs. But the benchmark does not show that the same savings will carry over to all coding workloads, providers or harnesses.
The strongest verified result is specific: SoL-Pi used up to 49% fewer tokens than Pi on EdgeBench under the reported test conditions, while the highest-efficiency configuration scored 93.7% of the baseline. The results may depend on the model, benchmark and harness configuration.
Source
Original source: the-decoder.com