Researchers compared three LLMs—Claude Haiku 4.5, Gemini 2.5 Flash, and Qwen 2.5 7B—to seven traditional name‑based tools (WRU, predictrace, ethnicolr, rethnicity, pyethnicity, and two local Qwen variants). They used three independent datasets: Chinese‑Pinyin names, 19,998 North Carolina voters who self‑reported race, and 300 political‑science authors coded manually. Coverage, conditional accuracy, and individual accuracy were measured for each method.

When hard‑label (argmax) estimators were applied to the voter data, they consistently under‑estimated the true share of Black voters. For example, Haiku’s hard labels implied a Black share of only 9.7 % versus the true 25 %. Across all tools, hard‑label methods missed the Black share by 15–25 percentage points and understated the White–Black gap in Democratic registration by 36 % to 65 %. In contrast, Bayesian posterior–weighted (BIRDiE) estimators that combine tool outputs with census priors produced estimates much closer to the ground truth.

A further issue is informative missingness. The LLMs’ refusal or “Unknown” outputs tend to occur more for older voters, inflating the estimated White–Black age gap by about six years. This bias cannot be corrected with a simple confusion‑matrix because the missingness pattern depends on age.

The study thus shows that selective labeling by large language models can distort demographic inferences and that Bayesian weighting can mitigate, but not eliminate, these distortions.