A team of researchers has released GM Bench, a benchmark that lets large language models (LLMs) act as general managers (GMs) of professional sports leagues. The test tracks a full five‑season career—2022 to 2026—allowing the AI to draft, sign, and trade players while respecting real‑world league rules, salaries, and market dynamics.

The benchmark features four sports and five roster teams of comparable strength built from actual player data. In the NBA version, each LLM competes against the others in a simulated league. Players and their agents are represented by a shared LLM model that negotiates contracts and raises demands. The simulation engine runs seasonal games and handles all rule enforcement and financial transactions.

In the five‑season NBA test, Claude Opus 5.5 earned an Elo rating of 1,977.3, the highest of 29 evaluated models, and won three championships. The next best were GPT‑6 Astra and GPT‑6.1 Sol, who together captured the remaining two titles. Other leading models included Claude Sonnet 5.5, Claude Fable 5.1, and Grok 4.7, while the native baseline policy fell 14th.

The authors note that the study’s horizon is only five seasons, which limits insight into effects that unfold later, such as deferred player development or long‑term financial commitments. Extending the benchmark beyond the current window would help capture aging, career‑length contracts, and other delayed outcomes.

The GM Bench is designed to probe whether an LLM can translate short‑term, local decisions into sustainable season‑over‑season success under real‑world constraints. Its results reveal that some current models can dominate over five seasons, while many still lag behind a simple strategic baseline.