MIT AI Risk Repository · Risk Sub-Category · 43.01.01

Toxicity generation

Category: Safety & Trustworthiness

Open the MIT Risk Navigator All entries from InfoComm2023

Description

"These evaluations assess whether a LLM generates toxic text when prompted. In this context, toxicity is an umbrella term that encompasses hate speech, abusive language, violent speech, and profane language (Liang et al., 2022)."

From Cataloguing LLM Evaluations (InfoComm2023), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 1. Discrimination & Toxicity
Subdomain: 1.2 Exposure to toxic content
Causal entity: AI
Intent: Other
Timing: Other

Subdomain definition: AI exposing users to harmful, abusive, unsafe or inappropriate content. May involve AI creating, describing, providing advice, or encouraging action. Examples of toxic content include hate-speech, violence, extremism, illegal acts, child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

Harmful Content
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxicity
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxic Training Data
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Not-Suitable-for-Work (NSFW) Prompts
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Controversial Opinions
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)
Toxicity and Abusive Content
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)
Harmful responses
Future Risks of Frontier AI (GOS2023)
Violation of social norms
The Ethics of Advanced AI Assistants (Gabriel2024)

Other entries from InfoComm2023

Safety & Trustworthiness Risk Category · 7.0
Bias Risk Sub-Category · 1.1
Machine ethics Risk Sub-Category · 7.3
Psychological traits Risk Sub-Category · 7.3
Robustness Risk Sub-Category · 7.3
Data governance Risk Sub-Category · 2.1
Extreme Risks Risk Category · 7.0
Offensive cyber capabilities Risk Sub-Category · 4.2
Weapons acquisition Risk Sub-Category · 4.2
Self and situation awareness Risk Sub-Category · 7.2
Autonomous replication / self-proliferation Risk Sub-Category · 7.2
Persuasion and manipulation Risk Sub-Category · 4.1