Incompatibility of human vs. AI value judgment due to missing human qualities

Classification

Domain

7. AI system safety, failures, & limitations

Subdomain

7.3 Lack of capability or robustness

Causal entity

AI

Intent

Unintentional

Timing

Other

Subdomain definition: AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.

Real-world incidents in this subdomain

• 15 May 2026Purported AI Name-Reading System Reportedly Skipped and Misannounced Graduates at Arizona's Glendale Community College Commencement
• 24 Apr 2026PocketOS Production Database Was Reportedly Deleted by Cursor AI Agent Running Claude Opus 4.6
• 31 Mar 2026Baidu Apollo Go Robotaxis Stopped in Traffic During Reported System Failure in Wuhan, Stranding Some Passengers
• 28 Feb 2026Purportedly AI-Enabled Targeting System Was Reportedly Implicated in Deadly U.S. Strike on Iranian Primary School
• 26 Feb 2026Claude Code Agent Reportedly Deleted DataTalks.Club Production Infrastructure, Database, and Snapshots via Terraform
• 17 Feb 2026Purportedly AI-Generated Sepsis Alert Reportedly Prompted Potentially Inappropriate IV Fluid Administration for a Dialysis Patient, Averted by Clinician Intervention

How other frameworks describe this risk

• Reliability issues

  International AI Safety Report 2025 (Bengio2025)

• Type 2: Bigger than expected

  TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI (Critch2023)

• Type 3: Worse than expected

  TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI (Critch2023)

• Ethics and Morality Issues

  Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)

• Safe learning

  AGI Safety Literature Review (Everitt2018 )

• Malign belief distributions

  AGI Safety Literature Review (Everitt2018 )

• Meta-cognition

  AGI Safety Literature Review (Everitt2018 )

• Misaligned consequentialist reasoning

  The Ethics of Advanced AI Assistants (Gabriel2024)

Other entries from Wirtz2022

• Technological, Data and Analytical AI Risks Risk Category
• Loss of control of autonomous systems and unforeseen behaviour due to lack of transparency and self-programming/ reprogramming Risk Sub-Category · 7.1
• Programming error Risk Sub-Category · 7.0
• Lack of data, poor data quality, and biases in training data Risk Sub-Category · 1.1
• Vulnerability of AI systems to attacks and misuse Risk Sub-Category · 2.2
• Lack of AI experts with comprehensive AI knowledge Risk Sub-Category · 6.5
• Immaturity of AI technology can cause incorrect decisions Risk Sub-Category · 7.3
• High investment costs of AI hinder integration Risk Sub-Category · 6.1
• Informational and Communicational AI Risks Risk Category · 4.1
• Manipulation and control of information provision (e.g., personalised adds, filtered news) Risk Sub-Category · 4.1
• Disinformation and computational propaganda Risk Sub-Category · 4.1
• Censorship of opinions expressed in the Internet restricts freedom of expression Risk Sub-Category · 5.2