First-hand information for everyone
AutoAnalysis is an automated, self‑updating meta‑analysis of human‑AI collaboration. The pipeline uses AI agents to perform every step of a systematic review—searching, screening, reviewing, extracting data, synthesizing results and producing a report—without human intervention. An audit agent cross‑checks each decision, agreeing 90 % of the time at the screening stage. The most recent run of AutoAnalysis combined data from 420 papers and 1,955 effect sizes. It found that human‑AI teams outperformed humans alone (g = 0.53, 95 % CI [0.45, 0.61]). However, they performed worse than the best of the two solo groups (g = –0.35, 95 % CI [–0.52, –0.18]). The analysis is limited to English peer‑reviewed experiments, may be biased by publication bias and high heterogeneity, and the synergy result relies on a small subset of studies that reported AI performance alone. These caveats mean that while the finding highlights important differences between improvement and synergy, it does not yet establish long‑term educational or organizational benefits.
