MIT AI Risk Repository · Risk Sub-Category · 34.02.02
Broadly-Scoped Goals
Category: Double edge components
Description
"Advanced AI systems are expected to develop objectives that span long timeframes,deal with complex tasks, and operate in open-ended settings (Ngo et al., 2024). ...However, it can also bring about the risk of encouraging manipulatingbehaviors (e.g., AI systems may take some bad actions to achieve human happiness, such as persuadingthem to do high-pressure jobs (Jacob Steinhardt, 2023))."
From AI Alignment: A Comprehensive Survey (Ji2023), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Causal entity
- Human
- Intent
- Intentional
- Timing
- Post-deployment
Subdomain definition: AI systems that develop, access, or are provided with capabilities that increase their potential to cause mass harm through deception, weapons development and acquisition, persuasion and manipulation, political strategy, cyber-offense, AI development, situational awareness, and self-proliferation. These capabilities may cause mass harm due to malicious human actors, misaligned AI systems, or failure in the AI system.
How other frameworks describe this risk
- Safety Risks from Affordances Provided to LLM-agents
- Agentic LLMs Pose Novel Risks
- Goal-Directedness Incentivizes Undesirable Behaviors
- Capabilities that could be used to reduce human control - Cyber offence
- Capabilities that could be used to reduce human control - Autonomous replication and adaptation
- Capabilities that could be used to reduce human control - Manipulation
- Subagents
- AI Influence