MIT AI Risk Repository · Risk Sub-Category · 73.07.03

Adversarial Optimization:

Category: Jailbreaks and Prompt Injections Threaten Security of LLMs

Open the MIT Risk Navigator All entries from Anwar2024

Description

"Jailbreak attacks can be discovered by performing manual or auto- mated adversarial optimization against a proxy objective that is noisily correlated with the success of a jailbreak. These are mostly gradient-based attacks (Zou et al., 2023b; Shin et al., 2020) as described in the previous two challenges, but gradient-free methods also exist (Prasad et al., 2022; Deng et al., 2022; Lapid et al., 2023)."

From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: —
Subdomain: —
Causal entity: Not coded
Intent: Not coded
Timing: Not coded

Other entries from Anwar2024

Agentic LLMs Pose Novel Risks Risk Category · 7.2
Natural Language Underspecifies Goals Risk Sub-Category · 7.1
Goal-Directedness Incentivizes Undesirable Behaviors Risk Sub-Category · 7.2
Safety Risks from Affordances Provided to LLM-agents Risk Sub-Category · 7.2
Multi-Agent Safety Is Not Assured by Single-Agent Safety Risk Category · 7.6
Foundationality May Cause Correlated Failures Risk Sub-Category · 7.6
Groups of LLM-Agents May Show Emergent Functionality Risk Sub-Category · 7.6
Collusion between LLM-Agents Risk Sub-Category · 7.6
Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs Risk Category · 4.0
Misinformation and Manipulation Risk Sub-Category · 4.3
Cybersecurity Risk Sub-Category · 4.3
Cybersecurity Additional evidence