MIT AI Risk Repository · Risk Category · 10.01.00
Bias and discrimination
Description
"The decision process used by AI systems has the potential to present biased choices, either because it acts from criteria that will generate forms of bias or because it is based on the history of choices."
From Social Impacts of Artificial Intelligence and Mitigation Recommendations: An Exploratory Study (Paes2023), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Causal entity
- AI
- Intent
- Unintentional
- Timing
- Post-deployment
Subdomain definition: Unequal treatment of individuals or groups by AI, often based on race, gender, or other sensitive characteristics, resulting in unfair outcomes and representation of those groups.
Real-world incidents in this subdomain
- DOGE Reportedly Relied on Unvetted ChatGPT Outputs in Canceling National Endowment for the Humanities Grants
- Sora Video Generator Has Reportedly Been Creating Biased Human Representations Across Race, Gender, and Disability
- Meta AI Characters Allegedly Exhibited Racism, Fabricated Identities, and Exploited User Trust
- Alleged AI-Generated Photo Alteration Leads to Inappropriate Modifications in Speaker's Conference Picture
- Algorithmic Bias in French Welfare System Allegedly Discriminates Against Marginalized Groups
- Department for Work and Pensions (DWP) AI Systems Allegedly Discriminate Against Single Mothers