MIT AI Risk Repository · Risk Sub-Category · 72.06.05

Multi-agent collusion propensity:

Category: Model Propensities

Open the MIT Risk Navigator All entries from Tse2025

Description

"Multiple agents tend to coordinate actions through covert means to maximize common interests (possibly harming third-party interests or evading regulation), even if individual agents are designed with safety constraints, their collusive behavior may still trigger systemic risks such as market manipulation or cascading failures that are difficult to detect and mitigate, and may develop specialized communication protocols to avoid monitoring."

From Frontier AI Risk Management Framework (v1.0) (Tse2025), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 7. AI system safety, failures, & limitations
Subdomain: 7.6 Multi-agent risks
Causal entity: AI
Intent: Unintentional
Timing: Post-deployment

Subdomain definition: Risks from multi-agent interactions, due to incentives (which can lead to conflict or collusion) and/or the structure of multi-agent systems, which can create cascading failures, selection pressures, new security vulnerabilities, and a lack of shared information and trust.

How other frameworks describe this risk

Collusion between LLM-Agents
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Multi-Agent Safety Is Not Assured by Single-Agent Safety
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Foundationality May Cause Correlated Failures
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Groups of LLM-Agents May Show Emergent Functionality
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Financial instability due to model homogeneity
Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024)
Conflict
Multi-Agent Risks from Advanced AI (Hammond2025)
Limited Interactions
Multi-Agent Risks from Advanced AI (Hammond2025)
Incompatible strategies
Multi-Agent Risks from Advanced AI (Hammond2025)

Other entries from Tse2025

Misuse Risks Risk Category · 4.0
Misuse Risks Additional evidence
Cyber Offense Risks Risk Sub-Category · 4.2
Biological and Chemical Risks Risk Sub-Category · 4.2
Biological and Chemical Risks Additional evidence
Physical Harm and Injury Risks Risk Sub-Category · 4.2
Physical Harm and Injury Risks Additional evidence
Large-Scale Persuasion and Harmful Manipulation Risks Risk Sub-Category · 4.1
Large-Scale Persuasion and Harmful Manipulation Risks Additional evidence
Loss of Control Risks Risk Category · 5.2
Loss of Control Risks Additional evidence
Passive loss of control Risk Sub-Category · 5.2