MIT AI Risk Repository · Risk Sub-Category · 67.04.02
Future AI systems might actively reduce human control
Category: Loss of control
Description
"Loss of control could be accelerated if AI systems take actions to increase their own influence and reduce human control. This threat model is controversial - experts in AI significantly disagree on how likely it is and those who deem it is likely disagree on the timeframe."
From Capabilities and Risks from Frontier AI (DSIT2023), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Causal entity
- AI
- Intent
- Other
- Timing
- Post-deployment
Subdomain definition: AI systems acting in conflict with human goals or values, especially the goals of designers or users, or ethical standards. These misaligned behaviors may be introduced by humans during design and development, such as through reward hacking and goal misgeneralisation, or may result from AI using dangerous capabilities such as manipulation, deception, situational awareness to seek power, self-proliferate, or achieve other goals.
Real-world incidents in this subdomain
- Reinforcement Learning Reward Functions in Video Games
- Predictive Policing Program by Florida Sheriff’s Office Allegedly Violated Residents’ Rights and Targeted Children of Vulnerable Groups
- Image Classification of Battle Tanks
How other frameworks describe this risk
Other entries from DSIT2023
- Societal harms
- Degradation of the information environment
- Degradation of the information environment
- Degradation of the information environment
- Degradation of the information environment
- Degradation of the information environment
- Labour market disruption
- Labour market disruption
- Bias, Fairness and Representational Harms
- Bias, Fairness and Representational Harms
- Bias, Fairness and Representational Harms
- Bias, Fairness and Representational Harms