MIT AI Risk Repository · Risk Sub-Category · 73.02.01
Foundationality May Cause Correlated Failures
Category: Multi-Agent Safety Is Not Assured by Single-Agent Safety
Description
"Another important characteristic of LLM development is foundationality — due to the expense of large- scale pretraining, many deployed instances share similar or identical learned components. Foundation- ality may both be a blessing and a curse. On the one hand, it may be possible to exploit the similarity in the design of LLM-agents to facilitate cooperation (Critch et al., 2022; Conitzer and Oesterheld, 2023; Oesterheld et al., 2023). On the other hand, foundationality may leave LLM-agents vulnerable to correlated failures both in terms of safety and capabilities due to increased output hom
From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Subdomain
- 7.6 Multi-agent risks
- Causal entity
- Other
- Intent
- Other
- Timing
- Post-deployment
Subdomain definition: Risks from multi-agent interactions, due to incentives (which can lead to conflict or collusion) and/or the structure of multi-agent systems, which can create cascading failures, selection pressures, new security vulnerabilities, and a lack of shared information and trust.
How other frameworks describe this risk
Other entries from Anwar2024
- Agentic LLMs Pose Novel Risks
- Natural Language Underspecifies Goals
- Goal-Directedness Incentivizes Undesirable Behaviors
- Safety Risks from Affordances Provided to LLM-agents
- Multi-Agent Safety Is Not Assured by Single-Agent Safety
- Groups of LLM-Agents May Show Emergent Functionality
- Collusion between LLM-Agents
- Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs
- Misinformation and Manipulation
- Cybersecurity
- Cybersecurity
- Cybersecurity