MIT AI Risk Repository · Risk Category · 59.02.00
Inappropriate degree of automation
Description
"The AI application’s degree of automation ranges from no automation to fully autonomous. AI applications with a high degree of automation may exhibit unexpected behaviour and pose risks in terms of their reliability and safety."
From AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks (Schnitzer2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Causal entity
- AI
- Intent
- Unintentional
- Timing
- Post-deployment
Subdomain definition: AI systems that develop, access, or are provided with capabilities that increase their potential to cause mass harm through deception, weapons development and acquisition, persuasion and manipulation, political strategy, cyber-offense, AI development, situational awareness, and self-proliferation. These capabilities may cause mass harm due to malicious human actors, misaligned AI systems, or failure in the AI system.
How other frameworks describe this risk
- Safety Risks from Affordances Provided to LLM-agents
- Agentic LLMs Pose Novel Risks
- Goal-Directedness Incentivizes Undesirable Behaviors
- Capabilities that could be used to reduce human control - Cyber offence
- Capabilities that could be used to reduce human control - Autonomous replication and adaptation
- Capabilities that could be used to reduce human control - Manipulation
- Subagents
- AI Influence
Other entries from Schnitzer2024
- Inadequate specification of ODD
- Inadequate planning of performance requirements
- Insufficient AI development documentation
- Inappropriate degree of transparency to end users
- Missing requirements for the implemented hardware
- Choice of untrustworthy data source
- Lack of data understanding
- Discriminative data bias
- Harming users’ data privacy
- Incorrect data labels
- Data poisoning
- Insufficient data representation