MIT AI Risk Repository · Risk Category · 48.03.00

Dangerous, Violent or Hateful Content

Open the MIT Risk Navigator All entries from NIST2024

Description

"Eased production of and access to violent, inciting, radicalizing, or threatening content as well as recommendations to carry out self-harm or conduct illegal activities. Includes difficulty controlling public exposure to hateful and disparaging or stereotyping content."

From Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 1. Discrimination & Toxicity
Subdomain: 1.2 Exposure to toxic content
Causal entity: AI
Intent: Other
Timing: Post-deployment

Subdomain definition: AI exposing users to harmful, abusive, unsafe or inappropriate content. May involve AI creating, describing, providing advice, or encouraging action. Examples of toxic content include hate-speech, violence, extremism, illegal acts, child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

Harmful Content
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxicity
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxic Training Data
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Not-Suitable-for-Work (NSFW) Prompts
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Controversial Opinions
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)
Toxicity and Abusive Content
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)
Harmful responses
Future Risks of Frontier AI (GOS2023)
Violation of social norms
The Ethics of Advanced AI Assistants (Gabriel2024)

Other entries from NIST2024

CBRN Information or Capabilities Risk Category · 4.2
CBRN Information or Capabilities Additional evidence
CBRN Information or Capabilities Additional evidence
Confabulation Risk Category · 3.1
Confabulation Additional evidence
Confabulation Additional evidence
Dangerous, Violent or Hateful Content Additional evidence
Dangerous, Violent or Hateful Content Additional evidence
Data Privacy Risk Category · 2.1
Data Privacy Additional evidence
Data Privacy Additional evidence
Environmental Impacts Risk Category · 6.6