Risks from models and algorithms (Risks of bias and discrimination)

Classification

Domain

1. Discrimination & Toxicity

Subdomain

1.1 Unfair discrimination and misrepresentation

Causal entity

Human

Intent

Other

Timing

Pre-deployment

Subdomain definition: Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and representation of those groups.

Real-world incidents in this subdomain

• 2 Apr 2025DOGE Reportedly Relied on Unvetted ChatGPT Outputs in Canceling National Endowment for the Humanities Grants
• 23 Mar 2025Sora Video Generator Has Reportedly Been Creating Biased Human Representations Across Race, Gender, and Disability
• 3 Jan 2025Meta AI Characters Allegedly Exhibited Racism, Fabricated Identities, and Exploited User Trust
• 15 Oct 2024Alleged AI-Generated Photo Alteration Leads to Inappropriate Modifications in Speaker's Conference Picture
• 15 Oct 2024Algorithmic Bias in French Welfare System Allegedly Discriminates Against Marginalized Groups
• 10 Jul 2024Department for Work and Pensions (DWP) AI Systems Allegedly Discriminate Against Single Mothers

How other frameworks describe this risk

• Discrimination

  A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)

• Harms of Representation and Other Biases

  Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

• Risks from bias and underrepresentation

  International Scientific Report on the Safety of Advanced AI (Bengio2024)

• Bias

  International AI Safety Report 2025 (Bengio2025)

• Bias

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

• Toxicity and Bias Tendencies

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

• Biased Training Data

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

• Broken systems

  Navigating the Landscape of AI Ethics and Responsibility (Cunha2023)

Other entries from TC2602024

• AI's inherent safety risks Risk Category
• Risks from models and algorithms (Risks of explainability) Risk Sub-Category · 7.4
• Risks from models and algorithms (Risks of robustness) Risk Sub-Category · 7.3
• Risks from models and algorithms (Risks of stealing and tampering) Risk Sub-Category · 2.2
• Risks from models and algorithms (Risks of unreliable output) Risk Sub-Category · 3.1
• Risks from models and algorithms (Risks of adversarial attack) Risk Sub-Category · 2.2
• Risks from data (Risks of illegal collection and use of data) Risk Sub-Category · 2.1
• Risks from data (Risks of improper content and poisoning in training data) Risk Sub-Category · 1.2
• Risks from data (Risks of unregulated training data annotation) Risk Sub-Category · 7.3
• Risks from data (Risks of data leakage) Risk Sub-Category · 2.1
• Risks from AI systems (Risks of exploitation through defects and backdoors) Risk Sub-Category · 2.2
• Risks from AI systems (Risks of computing infrastructure security) Risk Sub-Category · 2.2