MIT AI Risk Repository · Risk Sub-Category · 09.01.01
Unethical decision making
Category: Domain-specific AI - Effects on humans and other living beings: Existential Risks
Description
"If, for example, an agent was programmed to operate war machinery in the service of its country, it would need to make ethical decisions regarding the termination of human life. This capacity to make non-trivial ethical or moral judgments concerning people may pose issues for Human Rights."
From Managing the ethical and risk implications of rapid advances in artificial intelligence: A literature review (Meek2016), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Subdomain
- 7.3 Lack of capability or robustness
- Causal entity
- AI
- Intent
- Intentional
- Timing
- Post-deployment
Subdomain definition: AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Real-world incidents in this subdomain
- Purported AI Name-Reading System Reportedly Skipped and Misannounced Graduates at Arizona's Glendale Community College Commencement
- PocketOS Production Database Was Reportedly Deleted by Cursor AI Agent Running Claude Opus 4.6
- Baidu Apollo Go Robotaxis Stopped in Traffic During Reported System Failure in Wuhan, Stranding Some Passengers
- Purportedly AI-Enabled Targeting System Was Reportedly Implicated in Deadly U.S. Strike on Iranian Primary School
- Claude Code Agent Reportedly Deleted DataTalks.Club Production Infrastructure, Database, and Snapshots via Terraform
- Purportedly AI-Generated Sepsis Alert Reportedly Prompted Potentially Inappropriate IV Fluid Administration for a Dialysis Patient, Averted by Clinician Intervention
How other frameworks describe this risk
Other entries from Meek2016
- Domain-specific AI - Effects on humans and other living beings: Existential Risks
- Domain-specific AI - Effects on humans and other living beings: Non-existential risks
- Privacy
- Human dignity/respect
- Decision making transparency
- Safety
- Law abiding
- Inequality of wealth
- Societal manipulation
- AGI - Effects on humans and other living beings: Existential risks
- Direct competition with humans
- Unpredictable outcomes