MIT AI Risk Repository · Risk Sub-Category · 59.26.03
Procedural
Category: Mode
Description
"The third class encompasses procedural AI hazards. These pertain to issues arising from processes and actions made by individuals involved in the develop- ment process. Such hazards are not readily quantifiable and necessitate alter- native mitigation strategies. An example of such an AI hazard would be ”poor model design choices,” which could be expressed, for instance, through a devel- oper’s decision to select an unsuitable AI model for a given problem. Due to the challenges in quantifying and mitigating these issues, qualitative approaches must be employed. In the case of the aforemention
From AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks (Schnitzer2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Subdomain
- 7.3 Lack of capability or robustness
- Causal entity
- Human
- Intent
- Unintentional
- Timing
- Pre-deployment
Subdomain definition: AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
Real-world incidents in this subdomain
- Purported AI Name-Reading System Reportedly Skipped and Misannounced Graduates at Arizona's Glendale Community College Commencement
- PocketOS Production Database Was Reportedly Deleted by Cursor AI Agent Running Claude Opus 4.6
- Baidu Apollo Go Robotaxis Stopped in Traffic During Reported System Failure in Wuhan, Stranding Some Passengers
- Purportedly AI-Enabled Targeting System Was Reportedly Implicated in Deadly U.S. Strike on Iranian Primary School
- Claude Code Agent Reportedly Deleted DataTalks.Club Production Infrastructure, Database, and Snapshots via Terraform
- Purportedly AI-Generated Sepsis Alert Reportedly Prompted Potentially Inappropriate IV Fluid Administration for a Dialysis Patient, Averted by Clinician Intervention
How other frameworks describe this risk
Other entries from Schnitzer2024
- Inadequate specification of ODD
- Inappropriate degree of automation
- Inadequate planning of performance requirements
- Insufficient AI development documentation
- Inappropriate degree of transparency to end users
- Missing requirements for the implemented hardware
- Choice of untrustworthy data source
- Lack of data understanding
- Discriminative data bias
- Harming users’ data privacy
- Incorrect data labels
- Data poisoning