MIT AI Risk Repository · Risk Category · 48.02.00
Confabulation
Description
"The production of confidently stated but erroneous or false content (known colloquially as “hallucinations” or “fabrications”) by which users may be misled or deceived."
From Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Domain
- 3. Misinformation
- Subdomain
- 3.1 False or misleading information
- Causal entity
- AI
- Intent
- Unintentional
- Timing
- Post-deployment
Subdomain definition: AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
Real-world incidents in this subdomain
- Nonfiction Book 'The Future of Truth' Reportedly Included AI-Generated and Misattributed Quotations
- Claude Console Reportedly Generated Phantom Legal Quotations in Trump Layoffs Court Filing
- Purportedly AI-Enhanced Images of Iranian Women Protesters Were Reportedly Spread With Unverified Execution Claims
- South Africa Draft National AI Policy Reportedly Included Fictitious References Believed to Be AI Hallucinations
- Purportedly AI-Generated Image Reportedly Misled Daejeon Authorities Searching for Escaped Wolf Neukgu
- Gemini and Grok Reportedly Misidentified Authentic Minab School-Strike Graveyard Photo as Unrelated Disaster Imagery
How other frameworks describe this risk
Other entries from NIST2024
- CBRN Information or Capabilities
- CBRN Information or Capabilities
- CBRN Information or Capabilities
- Confabulation
- Confabulation
- Dangerous, Violent or Hateful Content
- Dangerous, Violent or Hateful Content
- Dangerous, Violent or Hateful Content
- Data Privacy
- Data Privacy
- Data Privacy
- Environmental Impacts