MIT AI Risk Repository · domain 3: Misinformation
3.2 Pollution of information ecosystem and loss of consensus reality
Highly personalized AI-generated misinformation creating “filter bubbles” where individuals only see what matches their existing beliefs, undermining shared reality, weakening social cohesion and political processes.
- 22
- 12
- 4
- 3
| Label | Value |
|---|---|
| Other | 9 |
| AI | 8 |
| Human | 5 |
| Label | Value |
|---|---|
| Other | 13 |
| Unintentional | 8 |
| Intentional | 1 |
| Label | Value |
|---|---|
| Post-deployment | 17 |
| Other | 5 |
| Label | Value |
|---|---|
| 2019 | 1 |
| 2021 | 1 |
| 2022 | 1 |
| 2024 | 1 |
| Label | Value |
|---|---|
| Risk Category | 3 |
| Risk Sub-Category | 19 |
Risk entries
Browse and export all- Radicalisation
"Radicalisation - Adoption of extreme political, social, or religious ideals and aspirations due to the nature or misuse of an algorithmic system, potentially resulting in abuse, violence, or terroris...
- Information degradation
"Information degradation - Creation or spread of false, hallucinatory, low-quality, misleading, or inaccurate information that degrades the information ecosystem and causes people to develop false or...
- Institutional trust loss
"Institutional trust loss - Erosion of trust in public institutions and weakened checks and balances due to mis/disinformation, influence operations, over-dependence on technology, etc."
- Worsened epistemic processes for society
"Epistemic processes and problem solving: we currently see more reasons to be concerned about AI worsening society's epistemic processes than reasons to be optimistic about AI helping us better solve...
- AI contributes to increased online polarisation
"One of the most significant commercial uses of current AI systems is in the content recommendation algorithms of social media companies, and there are already concerns that this is contributing to wo...
- Widespread use of persuasive tools contributes to splintered epistemic communities
"Even without deliberate misuse, widespread use of powerful persuasion tools could have negative impacts. If such tools were used by many different groups to advance many different ideas, we could see...
- Reduced decision-making capacity as a result of decreased trust in information
"In addition, the increased awareness of these trends in information production and distribution could make it harder for anyone to evaluate the trustworthiness of any information source, reducing ove...
- Degradation of the information environment
"Frontier AI can cheaply generate realistic content which can falsely portray people and events. There is potential risk of compromised decision-making by individuals and institutions who rely on inac...
- Clickbait and feeding the surveillance advertising ecosystem
"Beyond misinformation and disinformation, generative AI can be used to create clickbait headlines and articles, which manipulate how users navigate the internet and applications. For example, generat...
- Entrenched viewpoints and reduced political efficacy
"Design choices such as greater personalisation of AI assistants and efforts to align them with human preferences could also reinforce people’s pre-existing biases and entrench specific ideologies. In...
- Degraded and homogenised information environments
"Beyond this, the widespread adoption of advanced AI assistants for content generation could have a number of negative consequences for our shared information ecosystem. One concern is that it could r...
- Entrenching specific ideologies
"AI assistants may provide ideologically biased or otherwise partial information in attempting to align to user expectations. In doing so, AI assistants may reinforce people’s pre-existing biases and...
- Eroding trust and undermining shared knowledge
"AI assistants may contribute to the spread of large quantities of factually inaccurate and misleading content, with negative consequences for societal trust in information sources and institutions, a...
- Eroded epistemics
Strong AI may... enable personally customized disinformation campaigns at scale... AI itself could generate highly persuasive arguments that invoke primal human responses and inflame crowds... d under...
- Erosion of Society
"With online news feeds, both on websites and social media platforms, the news is now highly personalized for us. We risk losing a shared sense of reality, a basic solidarity."
- Pollution of information ecosystems
"Contaminating publicly available information with false or inaccurate information (i.e., the generative tool's output is disseminated beyond the end user)"
- Erosion of trust in public information
"Eroding trust in public information and knowledge"
- Overburdening ecosystems
"Pollution of a space/ecosystem that is expected to be free of AI involvement/influence (e.g., creative material submission portals, job applications)"
- Impacts on “epistemic security” and the information environment
-
- Cognitive risks (Risks of amplifying the effects of "information cocoons")
"AI can be extensively utilized for customized information services, collecting user information, and analyzing types of users, their needs, intentions, preferences, habits, and even mainstream public...
- Detection challenges in content
"The difficulty in distinguishing synthetic content from authentic material adds to information risks."
- Pollution of information ecosystem
"Contaminating publicly available information with false or inaccurate information"