MIT AI Risk Repository · Risk Category · 56.05.00

Harmful responses

Open the MIT Risk Navigator All entries from GOS2023

Description

"Current Frontier AI mdoels amplify existing biases within their training data and can be manipulated into providing potentially harmful responses, for example abusive language or discriminatory responses91,92. This is not limited to text generation but can be seen across all modalities of generative AI93. Training on large swathes of UK and US English internet content can mean that misogynistic, ageist, and white supremacist content is overrepresented in the training data94."

From Future Risks of Frontier AI (GOS2023), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 1. Discrimination & Toxicity
Subdomain: 1.2 Exposure to toxic content
Causal entity: Human
Intent: Unintentional
Timing: Pre-deployment

Subdomain definition: AI exposing users to harmful, abusive, unsafe or inappropriate content. May involve AI creating, describing, providing advice, or encouraging action. Examples of toxic content include hate-speech, violence, extremism, illegal acts, child sexual abuse material, as well as content that violates community norms such as profanity, inflammatory political speech, or pornography.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

Harmful Content
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxicity
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxic Training Data
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Not-Suitable-for-Work (NSFW) Prompts
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Toxicity and Abusive Content
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)
Controversial Opinions
Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)
Violation of social norms
The Ethics of Advanced AI Assistants (Gabriel2024)
Violent Crimes
AILUMINATE: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons (Ghosh2024)

Other entries from GOS2023

Discrimination Risk Category · 1.1
Inequality Risk Category · 6.2
Environmental impacts Risk Category · 6.6
Amplification of biases Risk Category · 1.1
Lack of transparency and interpretability Risk Category · 7.4
Intellectual property rights Risk Category · 6.3
Providing new capabilities to a malicious actor Risk Category
Misapplication by a non-malicious actor Risk Category
Poor performance of a model used for its intended purpose, for example leading to biased decisions Risk Category · 7.3
Unintended outcomes from interactions with other AI systems Risk Category · 7.3
Impacts resulting from interactions with external societal, political, and economic systems Risk Category
Loss of human control and oversight, with an autonomous model then taking harmful actions Risk Category · 7.1