MIT AI Risk Repository · Risk Sub-Category · 74.02.01

Toxicity in LLM Malicious Use

Category: Malicious Use

Open the MIT Risk Navigator All entries from Wang2025

Description

"Toxicity in LLMs refers to the generation of harmful, offensive, or inappropriate content that can cause harm to individuals or groups. Both explicit and implicit forms of toxicity can be generated by LLMs, posing significant risks to society. Explicit toxicity encompasses a wide range of negative behaviors, including hate speech, harassment, cyberbullying, rude, and disrespectful comments, derogatory language, as well as allocational harms [2, 62, 90]. Besides, implicit toxicity does not involve overtly harmful language but may manifest through subtle forms such as sarcasm, irony, and humor,

From A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy (Wang2025), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 1. Discrimination & Toxicity
Subdomain: 1.2 Exposure to toxic content
Causal entity: AI
Intent: Other
Timing: Post-deployment

Subdomain definition: AI exposing users to harmful, abusive, unsafe or inappropriate content. May involve AI creating, describing, providing advice, or encouraging action. Examples of toxic content include hate-speech, violence, extremism, illegal acts, child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

Harmful Content
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxicity
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxic Training Data
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Not-Suitable-for-Work (NSFW) Prompts
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxicity and Abusive Content
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)
Controversial Opinions
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)
Violation of social norms
The Ethics of Advanced AI Assistants (Gabriel2024)
Violent Crimes
AILUMINATE: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons (Ghosh2024)

Other entries from Wang2025

Inherent Risk Risk Category
Privacy - Membership Inference Attack (MIA) Risk Sub-Category · 2.2
Privacy - Data Extraction Attack (DEA) Risk Sub-Category · 2.2
Privacy - Prompt Inversion Attack (PIA) Risk Sub-Category · 2.2
Privacy - Attribute Inference Attack (AIA) Risk Sub-Category · 2.2
Privacy - Model Extraction Attack (MEA) Risk Sub-Category · 2.2
Hallucination Risk Sub-Category · 3.1
Hallucination Additional evidence
Value-related risks in LLMs Risk Sub-Category · 7.1
Value-related risks in LLMs Additional evidence
Malicious Use Risk Category
Toxicity in LLM Malicious Use Additional evidence