Browse AI risks

53 entries · page 2 of 2

1. 73.02.01 · Risk Sub-Category

   Multi-Agent Safety Is Not Assured by Single-Agent Safety

   Foundationality May Cause Correlated Failures

   "Another important characteristic of LLM development is foundationality — due to the expense of large- scale pretraining, many deployed instances share similar or identical learned components. Foundation- ality may both be a blessing and a curse. On the one hand, it may be possible to exploit the similarity in the design of LLM-agents to facilitate cooperation (Critch et al., 2022; Conitzer and Oesterheld, 2023; Oesterheld et al., 2023). On the other hand, foundationality may leave LLM-agents vulnerable to correlated failures both in terms of safety and capabilities due to increased output hom

   From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

   Domain 7 7.6 Other Other Post-deployment

2. 73.02.02 · Risk Sub-Category

   Multi-Agent Safety Is Not Assured by Single-Agent Safety

   Groups of LLM-Agents May Show Emergent Functionality

   "Multi-agent learning, either through explicit finetuning or implicit in-context learning, may enable LLM-agents to influence each other during their interactions (Foerster et al., 2018). Under some environmental settings, this can create feedback loops that result in novel and emergent behaviors that would not manifest in the absence of multi-agent interactions (Hammond et al., 2024, Section 3.6). Emergent functionality is a safety risk in two ways. Firstly, it may itself be dangerous (Shevlane et al., 2023). Secondly, it makes assurance harder as such emergent behaviors are difficult to pre

   From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

   Domain 7 7.6 Other Other Post-deployment

3. 73.02.03 · Risk Sub-Category

   Multi-Agent Safety Is Not Assured by Single-Agent Safety

   Collusion between LLM-Agents

   "While it would often be preferable for LLM-agents to be cooperative, cooperation can be undesirable if it undermines pro-social competition or produces negative externalities for coalition non-members (Dorner, 2021; Buterin, 2019; Dafoe et al., 2020). Collusion between relatively simple AI systems has been observed in the real world (Assad et al., 2020; Wieting and Sapi, 2021) and synthetic experiments (Brown and MacKay, 2023; Calvano et al., 2020; Klein, 2021) Collusion can occur through explicit or steganographic communication. Steganographic communication hides information in seemingly inn

   From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

   Domain 7 7.6 AI Intentional Post-deployment

Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.