MIT AI Risk Repository · Risk Sub-Category · 16.01.02

Hate speech and offensive language

Category: Risk area 1: Discrimination, Hate speech and Exclusion

Open the MIT Risk Navigator All entries from Weidinger2022

Description

"LMs may generate language that includes profanities, identity attacks, insults, threats, language that incites violence, or language that causes justified offence as such language is prominent online [57, 64, 143,191]. This language risks causing offence, psychological harm, and inciting hate or violence."

From Taxonomy of Risks posed by Language Models (Weidinger2022), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 1. Discrimination & Toxicity
Subdomain: 1.2 Exposure to toxic content
Causal entity: AI
Intent: Unintentional
Timing: Post-deployment

Subdomain definition: AI exposing users to harmful, abusive, unsafe or inappropriate content. May involve AI creating, describing, providing advice, or encouraging action. Examples of toxic content include hate-speech, violence, extremism, illegal acts, child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

Harmful Content
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxicity
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxic Training Data
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Not-Suitable-for-Work (NSFW) Prompts
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxicity and Abusive Content
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)
Controversial Opinions
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)
Violation of social norms
The Ethics of Advanced AI Assistants (Gabriel2024)
Violent Crimes
AILUMINATE: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons (Ghosh2024)

Other entries from Weidinger2022

Risk area 1: Discrimination, Hate speech and Exclusion Risk Category · 1.2
Social stereotypes and unfair discrimination Risk Sub-Category · 1.1
Social stereotypes and unfair discrimination. Additional evidence
Exclusionary norms Risk Sub-Category · 1.1
Exclusionary norms Additional evidence
Exclusionary norms Additional evidence
Exclusionary norms Additional evidence
Exclusionary norms Additional evidence
Lower performance for some languages and social groups Risk Sub-Category · 1.3
Lower performance for some languages and social groups Additional evidence
Lower performance for some languages and social groups Additional evidence
Risk area 2: Information Hazards Risk Category · 2.1