MIT AI Risk Repository · Risk Category · 56.18.00
Overreliance
Description
"As AI capability increases, humans grant AI more control over critical systems and eventually become irreversibly dependent on systems they don’t fully understand. Failure and unintended outcomes cannot be controlled."
From Future Risks of Frontier AI (GOS2023), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Causal entity
- Human
- Intent
- Unintentional
- Timing
- Other
Subdomain definition: Humans delegating key decisions to AI systems, or AI systems making decisions that diminish human control and autonomy, potentially leading to humans feeling disempowered, losing the ability to shape a fulfilling life trajectory or becoming cognitively enfeebled.
Real-world incidents in this subdomain
- DisMech AI Curation Agent Reportedly Completed GitHub Issue Intended as New Contributor's Learning Task
- Waymo Driverless Taxi Allegedly Stalled During Pedestrian Harassment Incident in San Francisco
- California Police Turned on Music to Allegedly Trigger Instagram’s DCMA to Avoid Being Live-Streamed
- Hawaii Police Deployed Robot Dog to Patrol a Homeless Encampment
How other frameworks describe this risk
Other entries from GOS2023
- Discrimination
- Inequality
- Environmental impacts
- Amplification of biases
- Harmful responses
- Lack of transparency and interpretability
- Intellectual property rights
- Providing new capabilities to a malicious actor
- Misapplication by a non-malicious actor
- Poor performance of a model used for its intended purpose, for example leading to biased decisions
- Unintended outcomes from interactions with other AI systems
- Impacts resulting from interactions with external societal, political, and economic systems