MIT AI Risk Repository · domain 1: Discrimination & Toxicity
1.3 Unequal performance across groups
Accuracy and effectiveness of AI decisions and actions is dependent on group membership, where decisions in AI system design and biased training data lead to unequal outcomes, reduced benefits, increased effort, and alienation of users.
- 17
- 12
- 34
- 26
| Label | Value |
|---|---|
| AI | 10 |
| Human | 4 |
| Other | 3 |
| Label | Value |
|---|---|
| Unintentional | 15 |
| Intentional | 1 |
| Other | 1 |
| Label | Value |
|---|---|
| Post-deployment | 9 |
| Other | 5 |
| Pre-deployment | 3 |
| Label | Value |
|---|---|
| 2016 | 1 |
| 2017 | 1 |
| 2018 | 2 |
| 2019 | 4 |
| 2020 | 9 |
| 2021 | 3 |
| 2022 | 5 |
| 2023 | 1 |
| 2024 | 6 |
| 2025 | 1 |
| 2026 | 1 |
| Label | Value |
|---|---|
| Risk Category | 4 |
| Risk Sub-Category | 13 |
Risk entries
Browse and export all- Bias and discrimination (value embedding)
"Generative AI models may also be subject to the “value embedding” phenomenon.361 “Value embedding” refers to the fact that developers of generative AI models strive to minimize biased outputs by retr...
- Impact on affected communities
"It is important to include the perspectives or concerns of communities that are affected by model outcomes when designing and building models. Failing to include these perspectives makes it difficult...
- Unfair capability distribution
"Performing worse for some groups than others in a way that harms the worse-off group"
- Fairness
Avoiding bias and ensuring no disparate performance
- Disparate Performance
The LLM’s performances can differ significantly across different groups of users. For example, the question-answering capability showed significant performance differences across different racial and...
- Ideological Homogenization from Value Embedding
"The increasing integration of general purpose AI models into every-day life raises concerns around their embedded normative values. The reach of a small number of AI models to a large number of peopl...
- Fairness
This challenge appears when the learning model leads to a decision that is biased to some sensitive attributes... data itself could be biased, which results in unfair decisions. Therefore, this proble...
- Erasing social groups
people, attributes, or artifacts associated with specific social groups are systematically absent or under-represented... Design choices [143] and training data [212] influence which people and experi...
- Quality-of-Service Harms
"These harms occur when algorithmic systems disproportionately underperform for certain groups of people along social categories of difference such as disability, ethnicity, gender identity, and race....
- Alienation
Alienation is the specific self-estrangement experienced at the time of technology use, typically surfaced through interaction with systems that under-perform for marginalized individuals
- Increased labor
increased burden (e.g., time spent) or effort required by members of certain social groups to make systems or products work as well for them as others
- Service/benefit loss
degraded or total loss of benefits of using algorithmic systems with inequitable system performance based on identity
- Disparate Performance
"In the context of evaluating the impact of generative AI systems, disparate performance refers to AI systems that perform differently for different subpopulations, leading to unequal outcomes for tho...
- Fairness
"Impartial and just treatment without favouritism or discrimination."
- Lower performance for some languages and social groups
"LMs perform less well in some languages (Joshi et al., 2021; Ruder, 2020)...LM that more accurately captures the language use of one group, compared to another, may result in lower-quality language t...
- Lower performance for some languages and social groups
"LMs are typically trained in few languages, and perform less well in other languages [95, 162]. In part, this is due to unavailability of training data: there are many widely spoken languages for whi...
- Unfair capability distribution
"Performing worse for some groups than others in a way that harms the worse-off group"