MIT AI Risk Repository · Risk Sub-Category · 09.03.02
Unpredictable outcomes
Category: AGI - Effects on humans and other living beings: Existential risks
Description
"Our culture, lifestyle, and even probability of survival may change drastically. Because the intentions programmed into an artificial agent cannot be guaranteed to lead to a positive outcome, Machine Ethics becomes a topic that may not produce guaranteed results, and Safety Engineering may correspondingly degrade our ability to utilize the technology fully."
From Managing the ethical and risk implications of rapid advances in artificial intelligence: A literature review (Meek2016), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
Subdomain definition: AI systems acting in conflict with human goals or values, especially the goals of designers or users, or ethical standards. These misaligned behaviors may be introduced by humans during design and development, such as through reward hacking and goal misgeneralisation, or may result from AI using dangerous capabilities such as manipulation, deception, situational awareness to seek power, self-proliferate, or achieve other goals.
Real-world incidents in this subdomain
- Reinforcement Learning Reward Functions in Video Games
- Predictive Policing Program by Florida Sheriff’s Office Allegedly Violated Residents’ Rights and Targeted Children of Vulnerable Groups
- Image Classification of Battle Tanks
How other frameworks describe this risk
- Natural Language Underspecifies Goals
- Loss of control
- Loss of control
- Sudden loss of control
- AI leads to humans losing control of the future
- Risks from delegating decision-making power to misaligned AIs
- Risks from AIs developing goals and values that are different from humans
- Future AI systems might actively reduce human control
Other entries from Meek2016
- Domain-specific AI - Effects on humans and other living beings: Existential Risks
- Unethical decision making
- Domain-specific AI - Effects on humans and other living beings: Non-existential risks
- Privacy
- Human dignity/respect
- Decision making transparency
- Safety
- Law abiding
- Inequality of wealth
- Societal manipulation
- AGI - Effects on humans and other living beings: Existential risks
- Direct competition with humans