First-hand information for everyone
This paper looks at why large language models (LLMs) do not reliably stand in for human subjects in strategic experiments. The author argues that human strategic choices include a classical rational baseline plus an extra correction term called δ. That δ reflects bounded computation — the limits on what people can work out. LLMs, by contrast, often retrieve solutions from their training text and so reproduce the classical baseline without the human-style δ.
The paper surveys recent work comparing LLMs and humans in games. Across bargaining, cooperation, coordination and some auctions, LLMs tend to be fairer, more cooperative, or closer to textbook equilibrium play than humans. For example, one study cited finds LLM cooperation around 65% versus 37% for humans in a one-shot prisoner’s dilemma. In bargaining tasks LLMs make more equal offers and reject fewer unfair offers. Attempts to close the gap by fine-tuning models on human responses or by giving them a “persona” have not fully reconciled the differences.
The theoretical idea builds on decades of behavioural economics. The classic view decomposes human decisions into a rational baseline plus an additive correction term δ. The new reading interprets δ as the signature of people’s bounded computation: thought processes and mistakes that arise because humans cannot compute unboundedly. Next-token prediction architectures in LLMs lack the same additive form, so they tend to output the baseline solution that appears in their training text. Models that are trained to reason step-by-step (sometimes called reasoning-distilled models) can mimic multi-step strategic reasoning, but their limits are set by compute budget and prompt length rather than the cognitive limits that shape human δ. As a result, any residual δ from models will look structurally different from the human one.
To test these ideas, the paper proposes four operational checks: conditional dependence (whether choices change with particular pieces of information), distributional asymmetry (differences in the spread or shape of choices), path-dependence under repetition (whether past play changes later choices), and paraphrase-robustness (whether rewording the same scenario changes responses). The author also makes a quantitative moderator prediction: the size of |δ| should grow when decision environments make opponent identity more individualised. Concretely, the paper predicts a medium or larger effect (Cohen’s d ≥ 0.5) when opponents are named individuals versus when they are presented as an anonymous aggregate. The paper notes a clear empirical gap: no published LLM–human comparisons yet exist for Tullock contests or all-pay auctions, which it flags as useful tests.
Important caveats appear in the source material. The survey finds a consistent direction of LLM–human divergence, but many published abstracts do not report effect sizes or consistent outcome measures, so magnitude estimates are often not extractable. The excerpt may be truncated and does not contain every empirical detail. The framework is a theory-driven reorientation: it says when LLMs will be reliable stand-ins and when they will not, and it calls for focused experiments to map those decision environments rather than assuming fine-tuning will make models fully human-like.