First-hand information for everyone
This paper shows that text placed by outsiders on public websites can sometimes end up in the data used to train large language models (LMs). The authors introduce a new way to estimate that risk, called HalfLife, and use it to test whether common web features — in particular public comment sections — can serve as a practical route for “poisoning” pretraining data. Poisoning means adding text that later makes a model produce unwanted or harmful outputs.
The team looks at a realistic attacker who does not have access to model training systems. Instead, the attacker posts or automates posts on other people’s web pages. The paper points to public discussion interfaces as a convenient place to inject text. The authors note attackers could use browser automation tools such as Selenium (a tool that automates web browsers) to make many posts automatically. They use Common Crawl, a large web archive, as a proxy for which pages crawlers tend to see.
HalfLife is their main technical idea. It breaks the end-to-end path from an injected post to model training into three steps. First, can the attacker successfully place text on a page at all? Second, will a web crawler or scraper capture that text when it copies the page? Third, will data-cleaning and filtering used to prepare LM training data remove the text before it is used for training? HalfLife estimates the probability that an injected piece of text survives all three steps and is included in a final training corpus.
Applying HalfLife, the authors find that public comments can be a viable path into training data. They report an estimated 0.15% probability of inclusion over Common Crawl for public comments in their analysis, and note that this can amount to more documents than a Wikipedia slice that other work has attacked (the Wikipedia slice is reported as about 0.067% of documents in a modern pretraining corpus). They also use HalfLife to show that an alternative injection route — programmatic advertisements — is unlikely to succeed. In later sections they report controlled experiments where included poisoned content changed outputs of both pretrained and instruction-tuned models, even when the injected text used formats that are harder to detect.
The authors stress important limits and uncertainties. The attacker’s influence is indirect and probabilistic: they cannot choose exactly which pages crawlers will include. Whether a post survives depends on site owners, crawler behavior, and the model developer’s filtering steps. HalfLife provides an estimate, not a certainty, because it multiplies several probabilities that vary by site and by the tools used to crawl and filter data. The paper assumes a minimal attacker who does not control training infrastructure and who must rely on ordinary web features to distribute content.
Why this matters: many LM training sets come from broad web crawls that are hard to inspect by hand. The work highlights a practical route by which third-party content on web pages can reach those corpora. The authors provide code for their analysis and call attention to the need for measuring whether injected content is present in pretraining datasets. Their results suggest model developers should consider how open web features, crawler design, and data curation affect the risk that maliciously placed text becomes part of training data.