MIT AI Risk Repository · Risk Sub-Category · 74.02.04

Jailbreak in LLM Malicious Use - White & Black Box Attacks

Category: Malicious Use

Open the MIT Risk Navigator All entries from Wang2025

Description

"In the fine-tuning and alignment phase, elaborately- designed instruction datasets can be utilized to fine-tune LLMs to drive them to perform undesirable behaviors, such as generating harmful information or content that violates ethical norms, and thus achieve a jailbreak. Based on the accessibility to the model parameters, we can categorize them into white-box and black-box attacks. For white-box attacks, we can jailbreak the model by modifying its parameter weights. In [107], Lermen et al. used LoRA to fine-tune the Llama2’s 7B, 13B, and 70B as well as Mixtral on AdvBench and RefusalBench d

From A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy (Wang2025), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 2. Privacy & Security
Subdomain: 2.2 AI system security vulnerabilities and attacks
Causal entity: Human
Intent: Intentional
Timing: Pre-deployment

Subdomain definition: Vulnerabilities in AI systems, software development toolchains, and hardware that can be exploited, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

Privacy loss
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Exploiting Limited Generalization of Safety Finetuning
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Jailbreaks and Prompt Injections Threaten Security of LLMs
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
“Model Psychology” Attacks
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Attacking LLMs via Additional Modalities a
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Vulnerability to Poisoning and Backdoors
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Model Attacks
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Hardware Vulnerabilities
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

Other entries from Wang2025

Inherent Risk Risk Category
Privacy - Membership Inference Attack (MIA) Risk Sub-Category · 2.2
Privacy - Data Extraction Attack (DEA) Risk Sub-Category · 2.2
Privacy - Prompt Inversion Attack (PIA) Risk Sub-Category · 2.2
Privacy - Attribute Inference Attack (AIA) Risk Sub-Category · 2.2
Privacy - Model Extraction Attack (MEA) Risk Sub-Category · 2.2
Hallucination Risk Sub-Category · 3.1
Hallucination Additional evidence
Value-related risks in LLMs Risk Sub-Category · 7.1
Value-related risks in LLMs Additional evidence
Malicious Use Risk Category
Toxicity in LLM Malicious Use Risk Sub-Category · 1.2