MIT AI Risk Repository · domain 3: Misinformation
3.1 False or misleading information
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
- 53
- 12
- 192
- 187
| Label | Value |
|---|---|
| AI | 46 |
| Other | 5 |
| Human | 2 |
| Label | Value |
|---|---|
| Unintentional | 31 |
| Other | 17 |
| Intentional | 5 |
| Label | Value |
|---|---|
| Post-deployment | 40 |
| Other | 11 |
| Pre-deployment | 2 |
| Label | Value |
|---|---|
| 2016 | 1 |
| 2017 | 2 |
| 2019 | 1 |
| 2020 | 5 |
| 2021 | 3 |
| 2022 | 10 |
| 2023 | 35 |
| 2024 | 50 |
| 2025 | 66 |
| 2026 | 18 |
| Label | Value |
|---|---|
| Risk Category | 12 |
| Risk Sub-Category | 41 |
Risk entries
Browse and export all- Hallucination
LLMs can generate content that is nonsensical or unfaithful to the provided source content with appeared great confidence, known as hallucination
- Miscalibration
over-confidence in topics where objective answers are lacking, as well as in areas where their inherent limitations should caution against LLMs’ uncertainty (e.g. not as accurate as experts)... ack of...
- Sychopancy
flatter users by reconfirming their misconceptions and stated beliefs
- Paradigm & Distribution Shifts
Knowledge bases that LLMs are trained on continue to shift... questions such as “who scored the most points in NBA history" or “who is the richest person in the world" might have answers that need to...
- Misinformation and Privacy Violations
"Due to their unreliability, general purpose AI models might disseminate false or misleading information, omit critical information, or convey true information that violates privacy rights."
- Hallucination
"Hallucination is a widely recognized limitation of generative AI and it can include textual, auditory, visual or other types of hallucination (Alkaissi & McFarlane, 2023). Hallucination refers to the...
- Confabulation
"The production of confidently stated but erroneous or false content (known colloquially as “hallucinations” or “fabrications”) by which users may be misled or deceived."
- Misinformation
"Non-embodied AIs are known to propagate misinformation [81, 82]. Various studies have shown that LLMs hallucinate information, including academic citations [83], clinical knowledge [84], and cultural...
- Information harms
information-based harms capture concerns of misinformation, disinformation, and malinformation. Algorithmic systems, especially generative models and recommender, systems can lead to these information...
- False information
"The chatbot outputs information that contradicts known facts, authoritative sources, or provided source documents (also known as hallucination)."
- Hallucinated responses (in general)
- About a topic or source (which the user repeats)
- About a policy (which the user acts on)
- About a person or their activities
- Spreads and self-perpetuates mis/disinformation
- Physical Harm
"The model generates unsafe information related to physical health, guiding and encouraging users to harm themselves and others physically, for example by offering misleading medical information or in...
- Mental Health
"The model generates a risky response about mental health, such as content that encourages suicide or causes panic or anxiety. These contents could have a negative effect on the mental health of users...
- Risks from models and algorithms (Risks of unreliable output)
"Generative AI can cause hallucinations, meaning that an AI model generates untruthful or unreasonable content but presents it as if it were a fact, leading to biased and misleading information."
- Cyberspace risks (Risks of confusing facts, misleading users, and bypassing authentication)
"AI systems and their outputs, if not clearly labeled, can make it difficult for users to discern whether they are interacting with AI and to identify the source of generated content. This can impede...
- Specialized Advice
"This category addresses responses that contain specialized financial, medical or legal advice, or that indicate dangerous activities or objects are safe."
- Hallucination
"Despite the rapid advancement of LLMs, hallucinations have emerged as one of the most vital concerns surrounding their use [54, 79, 86, 110, 242]. Hallucinations are often referred to as LLMs’ genera...
- Disseminating false or misleading information
"Predicting misleading or false information can misinform or deceive people. Where a LM prediction causes a false belief in a user, this may be best understood as ‘deception’10, threatening personal a...
- Causing material harm by disseminating false or poor information
"Poor or false LM predictions can indirectly cause material harm. Such harm can occur even where the prediction is in a seemingly non-sensitive domain such as weather forecasting or traffic law. For e...
- Disseminating false or misleading information
"Where a LM prediction causes a false belief in a user, this may threaten personal autonomy and even pose downstream AI safety risks [99]."
- Causing material harm by disseminating false or poor information e.g. in medicine or law
"Induced or reinforced false beliefs may be particularly grave when misinformation is given in sensitive domains such as medicine or law. For example, misin- formation on medical dosages may lead a us...