MIT AI Risk Repository · Risk Sub-Category · 02.09.02
Noisy Training Data
Category: Hallucinations
Description
"Another important source of hallucinations is the noise in training data, which introduces errors in the knowledge stored in model parameters [111]–[113]. Generally, the training data inherently harbors misinformation. When training on large-scale corpora, this issue becomes more serious because it is difficult to eliminate all the noise from the massive pre-training data."
From Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Domain
- 3. Misinformation
- Subdomain
- 3.1 False or misleading information
- Causal entity
- AI
- Intent
- Unintentional
- Timing
- Pre-deployment
Subdomain definition: AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Real-world incidents in this subdomain
- Nonfiction Book 'The Future of Truth' Reportedly Included AI-Generated and Misattributed Quotations
- Claude Console Reportedly Generated Phantom Legal Quotations in Trump Layoffs Court Filing
- Purportedly AI-Enhanced Images of Iranian Women Protesters Were Reportedly Spread With Unverified Execution Claims
- South Africa Draft National AI Policy Reportedly Included Fictitious References Believed to Be AI Hallucinations
- Purportedly AI-Generated Image Reportedly Misled Daejeon Authorities Searching for Escaped Wolf Neukgu
- Gemini and Grok Reportedly Misidentified Authentic Minab School-Strike Graveyard Photo as Unrelated Disaster Imagery