Browse AI risks

3 entries

1. 73.01.00 · Risk Category

   Agentic LLMs Pose Novel Risks

   "Currently, LLMs are chiefly being used in search and chat applications. This reactive nature limits the risks posed by LLMs. However, an LLM can be enhanced in various ways to create an LLM-agent to autonomously plan and act in the real-world and proactively perform its assigned tasks (Ruan et al., 2023). Such enhancements can come from further specialized training (ARC, 2022; Chen et al., 2023a), specialized prompting (Huang et al., 2022a), access to external tools (Ahn et al., 2022; Mialon et al., 2023), or other forms of “scaffolding” (Wang et al., 2023a; Park et al., 2023a). Due to increa

   From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

   Domain 7 7.2 AI Other Post-deployment

2. 73.01.03 · Risk Sub-Category

   Agentic LLMs Pose Novel Risks

   Goal-Directedness Incentivizes Undesirable Behaviors

   "Goal-directedness can cause agents to exhibit unethical and undesirable behaviors, such as deception (Ward et al., 2023), self-preservation (Hadfield-Menell et al., 2017), power-seeking, and immoral rea- soning (Pan et al., 2023a). Pan et al. (2023a) find that LLM-agents exhibit power-seeking behavior in text-based adventure games. LLM-agents have also been shown to use deception to achieve assigned goals when explicitly required by the task (Ward et al., 2023), or when the tasks can be more easily completed by employing deception and the prompt does not disallow deception (Scheurer et al., 2

   From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

   Domain 7 7.2 AI Intentional Other

3. 73.01.05 · Risk Sub-Category

   Agentic LLMs Pose Novel Risks

   Safety Risks from Affordances Provided to LLM-agents

   "The capabilities of LLM-agents can be enhanced in significant ways by providing the LLM-agent with novel affordances, e.g. the ability to browse the web (Nakano et al., 2021), to manipulate objects in the physical world (Ahn et al., 2022; Huang et al., 2022a), to create and instruct copies of itself (Richards, 2023), to create and use new tools (Wang et al., 2023a), etc. Affordances can create additional risks, as they often increase the impact area of the language-agent, and they amplify the consequences of an agent’s failures and enable novel forms of failure modes (Ruan et al., 2023; Pan e

   From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

   Domain 7 7.2 Human Unintentional Pre-deployment

Disclaimer: informational only, not legal advice. Verify every claim against the linked official sources and consult a qualified lawyer before acting.