MIT AI Risk Repository · Risk Sub-Category · 49.02.03
Loss of control
Category: Risks from Malfunctions
Description
"'Loss of control’ scenarios are potential future scenarios in which society can no longer meaningfully constrain some advanced general- purpose AI agents, even if it becomes clear they are causing harm. These scenarios are hypothesised to arise through a combination of social and technical factors, such as pressures to delegate decisions to general- purpose AI systems, and limitations of existing techniques used to influence the behaviours of general- purpose AI systems."
From International Scientific Report on the Safety of Advanced AI (Bengio2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Causal entity
- Other
- Intent
- Other
- Timing
- Post-deployment
Subdomain definition: AI systems acting in conflict with human goals or values, especially the goals of designers or users, or ethical standards. These misaligned behaviors may be introduced by humans during design and development, such as through reward hacking and goal misgeneralisation, or may result from AI using dangerous capabilities such as manipulation, deception, situational awareness to seek power, self-proliferate, or achieve other goals.
Real-world incidents in this subdomain
- Reinforcement Learning Reward Functions in Video Games
- Predictive Policing Program by Florida Sheriff’s Office Allegedly Violated Residents’ Rights and Targeted Children of Vulnerable Groups
- Image Classification of Battle Tanks
How other frameworks describe this risk
- Natural Language Underspecifies Goals
- Loss of control
- Sudden loss of control
- AI leads to humans losing control of the future
- Risks from delegating decision-making power to misaligned AIs
- Risks from AIs developing goals and values that are different from humans
- Future AI systems might actively reduce human control
- Value specification
Other entries from Bengio2024
- Malicious Use Risks
- Harm to individuals through fake content
- Harm to individuals through fake content
- Disinformation and manipulation of public opinion
- Cyber offence
- Cyber offence
- Cyber offence
- Dual use science risks
- Risks from Malfunctions
- Risks from product functionality issues
- Risks from product functionality issues
- Risks from product functionality issues