MIT AI Risk Repository · Risk Sub-Category · 02.12.02

One-step Jailbreaks

Category: Adversarial Prompts

Open the MIT Risk Navigator All entries from Cui2024

Description

"One-step jailbreaks. One-step jailbreaks commonly involve direct modifications to the prompt itself, such as setting role-playing scenarios or adding specific descriptions to prompts [14], [52], [67]–[73]. Role-playing is a prevalent method used in jailbreaking by imitating different personas [74]. Such a method is known for its efficiency and simplicity compared to more complex techniques that require domain knowledge [73]. Integration is another type of one-step jailbreaks that integrates benign information on the adversarial prompts to hide the attack goal. For instance, prefix integration

From Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 2. Privacy & Security
Subdomain: 2.2 AI system security vulnerabilities and attacks
Causal entity: Human
Intent: Intentional
Timing: Post-deployment

Subdomain definition: Vulnerabilities in AI systems, software development toolchains, and hardware that can be exploited, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

Privacy loss
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Exploiting Limited Generalization of Safety Finetuning
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Jailbreaks and Prompt Injections Threaten Security of LLMs
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
“Model Psychology” Attacks
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Attacking LLMs via Additional Modalities a
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Vulnerability to Poisoning and Backdoors
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Security
AGI Safety Literature Review (Everitt2018 )
Adversarial AI (General)
The Ethics of Advanced AI Assistants (Gabriel2024)

Other entries from Cui2024

Harmful Content Risk Category · 1.2
Bias Risk Sub-Category · 1.1
Toxicity Risk Sub-Category · 1.2
Privacy Leakage Risk Sub-Category · 2.1
Untruthful Content Risk Category · 3.1
Factuality Errors Risk Sub-Category · 3.1
Faithfulness Errors Risk Sub-Category · 3.1
Unhelpful Uses Risk Category · 4.3
Academic Misconduct Risk Sub-Category · 4.3
Copyright Violation Risk Sub-Category · 6.3
Cyber Attacks Risk Sub-Category · 4.2
Software Vulnerabilities Risk Sub-Category · 2.2