MIT AI Risk Repository · Risk Sub-Category · 45.02.01

Cyberspace risks (Risks of information and content safety)

Category: Safety risks in AI Applications

Open the MIT Risk Navigator All entries from TC2602024

Description

"AI-generated or synthesized content can lead to the spread of false information, discrimination and bias, privacy leakage, and infringement issues, threatening the safety of citizens' lives and property, national security, ideological security, and causing ethical risks. If users’ inputs contain harmful content, the model may output illegal or damaging information without robust security mechanisms."

From AI Safety Governance Framework (TC2602024), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 1. Discrimination & Toxicity
Subdomain: 1.2 Exposure to toxic content
Causal entity: Other
Intent: Other
Timing: Post-deployment

Subdomain definition: AI exposing users to harmful, abusive, unsafe or inappropriate content. May involve AI creating, describing, providing advice, or encouraging action. Examples of toxic content include hate-speech, violence, extremism, illegal acts, child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

Harmful Content
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxicity
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxic Training Data
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Not-Suitable-for-Work (NSFW) Prompts
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxicity and Abusive Content
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)
Controversial Opinions
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)
Violation of social norms
The Ethics of Advanced AI Assistants (Gabriel2024)
Violent Crimes
AILUMINATE: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons (Ghosh2024)

Other entries from TC2602024

AI's inherent safety risks Risk Category
Risks from models and algorithms (Risks of explainability) Risk Sub-Category · 7.4
Risks from models and algorithms (Risks of bias and discrimination) Risk Sub-Category · 1.1
Risks from models and algorithms (Risks of robustness) Risk Sub-Category · 7.3
Risks from models and algorithms (Risks of stealing and tampering) Risk Sub-Category · 2.2
Risks from models and algorithms (Risks of unreliable output) Risk Sub-Category · 3.1
Risks from models and algorithms (Risks of adversarial attack) Risk Sub-Category · 2.2
Risks from data (Risks of illegal collection and use of data) Risk Sub-Category · 2.1
Risks from data (Risks of improper content and poisoning in training data) Risk Sub-Category · 1.2
Risks from data (Risks of unregulated training data annotation) Risk Sub-Category · 7.3
Risks from data (Risks of data leakage) Risk Sub-Category · 2.1
Risks from AI systems (Risks of exploitation through defects and backdoors) Risk Sub-Category · 2.2