arXiv News

First-hand information for everyone

Site

AboutPrivacy PolicyTerms of Use

© 2026 arXiv News

Large Language Models Lag in Capturing Diverse Narratives Across 23 Languages | arXiv News

arXiv News

EnglishJapanese

Switch language

EnglishJapanese
Loading account…

First-hand information for everyone

Back to front page
Natural Language ProcessingEnglishPublished

Large Language Models Lag in Capturing Diverse Narratives Across 23 Languages

November 12, 2026arXiv: 2611.12750v1

Researchers have defined six key capabilities needed for multilingual information‑seeking, including three foundational tasks—selecting what stories to show, what to omit, and spotting missing accounts—and three higher‑level tasks that involve presenting diverse narratives, keeping coverage when users confirm the same story and making sure information balance is preserved across languages. This framework underpins the new benchmark, NarrativeBench, which contains 1,333 news articles in 23 languages covering contested topics and 460 Reddit‑derived queries expanded into 24 languages for evaluation. The article texts are on average 5,977 characters long, with a range from 610 to 46,252 characters, and their similarity analysis confirms that the collection includes distinct narratives.

On NarrativeBench, 15 model configurations were tested. Results show average score of 19.9 % for Narrative Inclusion (the ability to surface all relevant accounts), 49.3 % for Narrative Exclusion (the ability to filter unwanted content), and 56.8 % recall for Missing Narrative Identification (finding accounts the system overlooked). Each configuration performed better at exclusion than inclusion, and even the best model—Gemini 3.8 Flash—reached only 52.5 % for inclusion and 82.8 % for missing account recall. Gemini 2.5 Pro led with an exclusion score of 74.2 %.

These findings highlight that current large language models still fall short of reliably exploring diverse perspectives. Users cannot fully control which viewpoints appear, and substantial gaps remain in exposing competing accounts when diversity is explicitly requested.

Source paper

The Narrative Gap: Can LLMs Help Us Navigate Diverse Narratives Across Languages?

Natural Language ProcessingArtificial IntelligenceHuman-Computer InteractionInformation Retrieval
View on arXiv