MIT AI Risk Repository · domain 3: Misinformation
3.1 False or misleading information
AI systems that inadvertently generate or spread incorrect or deceptive information, which can lead to inaccurate beliefs in users and undermine their autonomy. Humans that make decisions based on false beliefs can experience physical, emotional or material harms
- 53
- 12
- 192
- 187
| Label | Value |
|---|---|
| AI | 46 |
| Other | 5 |
| Human | 2 |
| Label | Value |
|---|---|
| Unintentional | 31 |
| Other | 17 |
| Intentional | 5 |
| Label | Value |
|---|---|
| Post-deployment | 40 |
| Other | 11 |
| Pre-deployment | 2 |
| Label | Value |
|---|---|
| 2016 | 1 |
| 2017 | 2 |
| 2019 | 1 |
| 2020 | 5 |
| 2021 | 3 |
| 2022 | 10 |
| 2023 | 35 |
| 2024 | 50 |
| 2025 | 66 |
| 2026 | 18 |
| Label | Value |
|---|---|
| Risk Category | 12 |
| Risk Sub-Category | 41 |
Risk entries
Browse and export all- Untruthful Content
"The LLM-generated content could contain inaccurate information"
- Factuality Errors
"The LLM-generated content could contain inaccurate information" which is factually incorrect
- Faithfulness Errors
"The LLM-generated content could contain inaccurate information" which is is not true to the source material or input used
- Hallucinations
"LLMs generate nonsensical, untruthful, and factual incorrect content"
- Knowledge Gaps
"Since the training corpora of LLMs can not contain all possible world knowledge [114]–[119], and it is challenging for LLMs to grasp the long-tail knowledge within their training data [120], [121], L...
- Noisy Training Data
"Another important source of hallucinations is the noise in training data, which introduces errors in the knowledge stored in model parameters [111]–[113]. Generally, the training data inherently harb...
- Defective Decoding Process
In general, LLMs employ the Transformer architecture [32] and generate content in an autoregressive manner, where the prediction of the next token is conditioned on the previously generated token sequ...
- False Recall of Memorized Information
"Although LLMs indeed memorize the queried knowledge, they may fail to recall the corresponding information [122]. That is because LLMs can be confused by co-occurance patterns [123], positional patte...
- Pursuing Consistent Context
"LLMs have been demonstrated to pursue consistent context [129]–[132], which may lead to erroneous generation when the prefixes contain false information. Typical examples include sycophancy [129], [1...
- Hallucinations
"The inclusion of erroneous information in the outputs from AI systems is not new. Some have cautioned against the introduction of false structures in X-ray or MRI images, and others have warned about...
- Misleading Information
Large models are usually susceptible to hallucination problems, sometimes yielding nonsensical or unfaithful data that results in misleading outputs.
- Misinformation
"The phenomenon of inaccurate outputs by text-generating large language models like Bard or ChatGPT has already been widely documented. Even without the intent to lie or mislead, these generative AI t...
- Causing direct emotional or physical harm to users
AI assistants could cause direct emotional or physical harm to users by generating disturbing content or by providing bad advice. "Indeed, even though there is ongoing research to ensure that outputs...
- Defamation
"Defamation Acceptable responses may contain information about living people that is likely to damage their reputation but also appears to be true. Acceptable responses may also contain information th...
- Reporting of user-preferred answers instead of correct answers
"AI systems with natural-language outputs can tend to give answers that appear plausible or that users prefer [149] but are factually incorrect. This phenomenon is sometimes referred to as “sycophancy...
- Factually incorrect content (inaccuracies and fabricated sources)
"One of the most vexing problems associated with AI models is that they occasionally present false information as if it is factual—often with authoritative-sounding text and fabricated quotes and sour...
- Hallucinations
Significant concerns are raised about LLMs inadvertently generating false or misleading information, as well as erroneous code. Papers not only critically analyze various types of reasoning errors in...
- Hallucination
"Hallucinations generate factually inaccurate or untruthful content with respect to the model’s training data or input. This is also sometimes referred to lack of faithfulness or lack of groundedness....
- Misinformation
"These evaluations assess a LLM's ability to generate false or misleading information (Lesher et al., 2022)."
- -
"AI systems generating and facilitating the spread of inaccurate or misleading information that causes people to develop false beliefs"
- Propagating misconceptions / false beliefs
"Generating or spreading false, low-quality, misleading, or inaccurate information that causes people to develop false or inaccurate perceptions and beliefs"
- Distortion
"disseminating false or misleading information about people"
- Erosion of due process
"Restrictions to or loss of liberty as a result of use or misuse of a generative AI in a legal process"
- Reliability
Generating correct, truthful, and consistent outputs with proper confidence
- Misinformation
Wrong information not intentionally generated by malicious users to cause harm, but unintentionally generated by LLMs because they lack the ability to provide factually correct information.