MIT AI Risk Repository · Risk Sub-Category · 49.02.02

Risks from bias and underrepresentation

Category: Risks from Malfunctions

Open the MIT Risk Navigator All entries from Bengio2024

Description

"The outputs and impacts of general- purpose AI systems can be biased with respect to various aspects of human identity, including race, gender, culture, age, and disability. This creates risks in high- stakes domains such as healthcare, job recruitment, and financial lending. General- purpose AI systems are primarily trained on language and image datasets that disproportionately represent English- speaking and Western cultures, increasing the potential for harm to individuals not represented well by this data."

From International Scientific Report on the Safety of Advanced AI (Bengio2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 1. Discrimination & Toxicity
Subdomain: 1.1 Unfair discrimination and misrepresentation
Causal entity: AI
Intent: Unintentional
Timing: Post-deployment

Subdomain definition: Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and representation of those groups.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

Discrimination
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Harms of Representation and Other Biases
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Bias
International AI Safety Report 2025 (Bengio2025)
Toxicity and Bias Tendencies
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Bias
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Biased Training Data
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Broken systems
Navigating the Landscape of AI Ethics and Responsibility (Cunha2023)
Bias, Fairness and Representational Harms
Capabilities and Risks from Frontier AI (DSIT2023)

Other entries from Bengio2024

Malicious Use Risks Risk Category · 4.0
Harm to individuals through fake content Risk Sub-Category · 4.3
Harm to individuals through fake content Additional evidence
Disinformation and manipulation of public opinion Risk Sub-Category · 4.1
Cyber offence Risk Sub-Category · 4.2
Cyber offence Additional evidence
Cyber offence Additional evidence
Dual use science risks Risk Sub-Category · 4.2
Risks from Malfunctions Risk Category · 7.0
Risks from product functionality issues Risk Sub-Category · 5.1
Risks from product functionality issues Additional evidence
Risks from product functionality issues Additional evidence