MIT AI Risk Repository · Risk Sub-Category · 45.02.09
Cognitive risks (Risks of amplifying the effects of "information cocoons")
Category: Safety risks in AI Applications
Description
"AI can be extensively utilized for customized information services, collecting user information, and analyzing types of users, their needs, intentions, preferences, habits, and even mainstream public awareness over a certain period. It can then be used to offer formulaic and tailored information and services, aggravating the effects of "information cocoons.""
From AI Safety Governance Framework (TC2602024), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Domain
- 3. Misinformation
- Causal entity
- Human
- Intent
- Unintentional
- Timing
- Post-deployment
Subdomain definition: Highly personalized AI-generated misinformation creating “filter bubbles” where individuals only see what matches their existing beliefs, undermining shared reality, weakening social cohesion and political processes.
Real-world incidents in this subdomain
- Google Books Appears to Be Indexing Works Written by AI
- Uptick in Low-Quality AI-Produced Content Degraded Publishers' Submission Management
- Korean Politician Employed Deepfake as Campaign Representative
- Facebook Political Ad Delivery Algorithms Inferred Users' Political Alignment, Inhibiting Political Campaigns' Reach
How other frameworks describe this risk
- Information degradation
- Radicalisation
- Institutional trust loss
- Worsened epistemic processes for society
- Reduced decision-making capacity as a result of decreased trust in information
- Widespread use of persuasive tools contributes to splintered epistemic communities
- AI contributes to increased online polarisation
- Degradation of the information environment
Other entries from TC2602024
- AI's inherent safety risks
- Risks from models and algorithms (Risks of explainability)
- Risks from models and algorithms (Risks of bias and discrimination)
- Risks from models and algorithms (Risks of robustness)
- Risks from models and algorithms (Risks of stealing and tampering)
- Risks from models and algorithms (Risks of unreliable output)
- Risks from models and algorithms (Risks of adversarial attack)
- Risks from data (Risks of illegal collection and use of data)
- Risks from data (Risks of improper content and poisoning in training data)
- Risks from data (Risks of unregulated training data annotation)
- Risks from data (Risks of data leakage)
- Risks from AI systems (Risks of exploitation through defects and backdoors)