MIT AI Risk Repository · Risk Sub-Category · 53.02.05
Autonomous replication
Category: Dangerous capabilities in AI systems
On this page 4 sections
Description
"the ability of simple software to autonomously spread around the internet in spite of countermeasures (various software worms and computer viruses)"
From Advancing AI Governance: A Literature Review of Problems, Options, and Proposals (Maas2023), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Causal entity
- AI
- Intent
- Intentional
- Timing
- Post-deployment
Subdomain definition: AI systems that develop, access, or are provided with capabilities that increase their potential to cause mass harm through deception, weapons development and acquisition, persuasion and manipulation, political strategy, cyber-offense, AI development, situational awareness, and self-proliferation. These capabilities may cause mass harm due to malicious human actors, misaligned AI systems, or failure in the AI system.
How other frameworks describe this risk
- Goal-Directedness Incentivizes Undesirable Behaviors
- Safety Risks from Affordances Provided to LLM-agents
- Agentic LLMs Pose Novel Risks
- Capabilities that could be used to reduce human control - Manipulation
- Capabilities that could be used to reduce human control - Autonomous replication and adaptation
- Capabilities that could be used to reduce human control - Cyber offence
- Subagents
- Nascent capabilities (emergent capabilities)
Other entries from Maas2023
- Alignment failures in existing ML systems
- Faulty reward functions in the wild
- Specification gaming
- Reward model overoptimization
- Instrumental convergence
- Goal misgeneralization
- Inner misalignment
- Language model misalignment
- Harms from increasingly agentic algorithmic systems
- Dangerous capabilities in AI systems
- Situational awareness
- Acquisition of a goal to harm society