On 30 benchmark tasks covering reading comprehension, math, programming, and commonsense, Strategist beat the best baseline on 24 tasks and reached nearly the same overall accuracy (74.7 % vs. 74.0 %) while using 39 % fewer weighted inference tokens. The gains came from the library’s growing repertoire of proven blueprints.

There are caveats. On HotpotQA and COPA the agent matched baseline accuracy but ran at about 6‑9 % higher cost. In medical‑question and math‑competition benchmarks it chose cheaper, less accurate strategies, trading 5‑7 % accuracy for 22‑56 % cost savings. Across all tasks the average cost reduction was 39 %.

Importantly, the strategies discovered by Strategist transferred to tasks it had never seen before and to other language models, without a fresh search.

At the end of the paper the authors emphasize that the agent’s improvement comes from its own learning of inference strategies—what the field has previously left to human engineers.