First-hand information for everyone
This paper documents a software error in the gsynth R package that could make analysts look far more certain than they should be. The problem affected versions of gsynth released before the 1.3.1 update on CRAN in December 2025. When users selected two options together — the parametric bootstrap for inference and the IFE-EM estimator (an Interactive Fixed Effects estimator that uses an Expectation-Maximization fitting step) — the package produced standard errors that were often far too small. The authors show that this behavior could lead to many false positive findings in realistic data settings.
At a technical level, the bug came from how the parametric bootstrap was implemented. A bootstrap is a resampling method used to estimate how much an estimate would vary from sample to sample. The published algorithm that gsynth intended to follow (from Xu 2017) adds simulated out-of-sample prediction errors to fitted values. The buggy implementation instead reused in-sample residuals — the differences between observed outcomes and their fitted values on the same data. Using in-sample residuals mechanically lowers the variation across bootstrap samples and so shrinks reported standard errors.
To test how harmful this was, the authors ran an empirical Monte Carlo study. They applied randomly assigned placebo treatments to a set of state-level panel datasets and recorded how often gsynth reported statistically significant effects when there should be none. The package’s historical behavior produced high false positive rates. The authors also reanalyzed three papers published in the American Political Science Review that had used the affected code path. They compare (1) the original published results using the historical gsynth implementation, (2) results using a corrected implementation of the parametric bootstrap that restores the omitted leave-one-out step, and (3) results using the Generalized Synthetic Control method (GSC) as described in Xu (2017). Correcting the implementation made most previously significant findings become insignificant. Using GSC made every finding in these checks insignificant.
This matters because Interactive Fixed Effects models and the gsynth package have become popular tools for analyzing panel data in social science. The authors note gsynth had over 300 downloads per week as of April 2026. If standard errors are understated by orders of magnitude, researchers can draw overly confident conclusions and publish results that do not hold up under correct uncertainty estimates.
The authors are careful to note important caveats. First, the package behavior they document was removed in version 1.3.1 and the current gsynth documentation warns against using the parametric bootstrap together with IFE-EM. Second, Xu (the author of the GSC method) engaged with the authors, acknowledged the implementation error, and emphasized a theoretical incompatibility between parametric-bootstrap inference and the IFE-EM fitting approach. For this reason the paper’s “corrected” bootstrap is an implementation-specific fix that helps quantify the consequences of the historical bug, but is not presented as a general endorsement that parametric-bootstrap inference is valid for IFE-EM. Xu’s reanalyses broadly support the conclusion that the affected analyses were overconfident, though he reports a few exceptions under alternative technical choices.
The manuscript gives dates and updates: the first draft was dated April 13, 2026, Xu’s public response is noted on April 28, 2026, and the current manuscript date is June 8, 2026. The authors recommend that applied users distinguish the Generalized Synthetic Control method from IFE-EM, report sensitivity to the number of latent factors, run placebo diagnostics, and use inferential procedures that have theoretical support for the estimator actually fit.