Impacts on “epistemic security” and the information environment

Classification

Domain

3. Misinformation

Subdomain

3.2 Pollution of information ecosystem and loss of consensus reality

Causal entity

AI

Intent

Unintentional

Timing

Post-deployment

Subdomain definition: Highly personalized AI-generated misinformation creating “filter bubbles” where individuals only see what matches their existing beliefs, undermining shared reality, weakening social cohesion and political processes.

Real-world incidents in this subdomain

• 4 Apr 2024Google Books Appears to Be Indexing Works Written by AI
• 30 Nov 2022Uptick in Low-Quality AI-Produced Content Degraded Publishers' Submission Management
• 6 Dec 2021Korean Politician Employed Deepfake as Campaign Representative
• 10 Jul 2019Facebook Political Ad Delivery Algorithms Inferred Users' Political Alignment, Inhibiting Political Campaigns' Reach

How other frameworks describe this risk

• Information degradation

  A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)

• Radicalisation

  A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)

• Institutional trust loss

  A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)

• Worsened epistemic processes for society

  A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values (Clarke2023)

• Reduced decision-making capacity as a result of decreased trust in information

  A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values (Clarke2023)

• Widespread use of persuasive tools contributes to splintered epistemic communities

  A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values (Clarke2023)

• AI contributes to increased online polarisation

  A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values (Clarke2023)

• Degradation of the information environment

  Capabilities and Risks from Frontier AI (DSIT2023)

Other entries from Maas2023

• Alignment failures in existing ML systems Risk Category · 7.1
• Faulty reward functions in the wild Risk Sub-Category · 7.1
• Specification gaming Risk Sub-Category · 7.1
• Reward model overoptimization Risk Sub-Category · 7.1
• Instrumental convergence Risk Sub-Category · 7.1
• Goal misgeneralization Risk Sub-Category · 7.1
• Inner misalignment Risk Sub-Category · 7.1
• Language model misalignment Risk Sub-Category · 7.1
• Harms from increasingly agentic algorithmic systems Risk Sub-Category · 7.2
• Dangerous capabilities in AI systems Risk Category · 7.2
• Situational awareness Risk Sub-Category · 7.2
• Acquisition of a goal to harm society Risk Sub-Category · 4.2