AI incident ·
Kenyan Data Annotators Allegedly Exposed to Graphic Content for OpenAI's AI
On this page 9 sections
In brief
An AI system built by Openai, Chatbot Developers and 1 other and deployed by Openai allegedly harmed Sama, Employees Of Sama and 1 other.
- Risk domain
- Socioeconomic & Environmental Harms
- Occurred
- Coverage
- 8 reports
What happened
Sama AI's Kenyan contractors were reportedly asked with excessively low pay to annotate a large volume of disturbing content to improve OpenAI's generative AI systems such as ChatGPT, and whose contract was terminated prior to completion by Sama AI.
Laws on record for this risk area
Policy angle: Classified under Socioeconomic & Environmental Harms (Increased inequality and decline in employment quality) in the MIT AI Risk Repository taxonomy; 4 instruments on record cover this risk area.
- Framework Convention on AI (CETS 225)
- India DPDP Act
- Law on Artificial Intelligence (2025)
- Law No. 132/2025 on artificial intelligence
Matched from the record's risk category and country to the instruments in effect recorded here: a law listed covers the area, which is not a finding that it applies to this incident. A reviewer can correct the match in the repository (data/external/incident_overrides.yaml).
News reports (8)
Titles link to the original publisher; report text is not reproduced here.
Who was involved
- Alleged deployer
- Openai
- Alleged developer
- Openai, Chatbot Developers, Large Language Model Developers
- Alleged harmed party
- Sama, Employees Of Sama, Data Annotators
Classification (MIT AI Risk Repository taxonomy)
- Risk domain
- Socioeconomic & Environmental Harms
- Causal entity
- Human
- Intent
- Intentional
- Timing
- Pre-deployment
- Harm level
- —
- Sectors
- —
- Countries
- —
Risk entries describing this failure mode
Entries from the MIT AI Risk Repository coded to subdomain 6.2.
- Societal inequality
"Societal inequality - Increased difference in social status or wealth between individuals or groups caused or amplified by a technology system, leading to the loss of social and community wellbeing/cohesion and destabil...
- Societal destabilisation
"Societal destabilisation - Societal instability in the form of strikes, demonstrations and other types of civil unrest caused by loss of jobs to technology, unfair algorithmic outcomes, disinformation, etc."
- Labour exploitation
"Labour exploitation - Use of under-paid and/or offshore labour to develop, manage or optimise a technology system."
- Job loss/losses
"Job loss/losses - Replacement/displacement of human jobs by a technology system, leading to increased unemployment, inequality, reduced consumer spending, and social friction."
- Political instability
"Political instability - Political polarisation or unrest caused by increased inequality, job losses, over- dependence on technology making societies vulnerable to systemic failures, etc, arising from or amplified by the...
- Increased income disparity
"While AI is predicted to bring increased GDP per capita by performing existing jobs more efficiently and compensating for a decline in the workforce, especially due to population aging, the potential substitution of man...
- Economic
"AI is predicted to bring increased GDP per capita by performing existing jobs more efficiently and compensating for a decline in the workforce, especially due to population aging, the potential substitution of many low-...
- Effects on Inequality
"LLMs could potentially worsen socioeconomic inequalities (Capraro et al., 2023). Effects on inequal- ity are closely linked to the effects of LLMs on workers but ultimately depend on how the fruits of technological prog...
Incidents in the same risk subdomain
- Polish Radio Station Replaces Human Hosts with AI-Generated Presenters to Simulate Interviewing Deceased Poet Wisława Szymborska
- Fast Food Chains' AI Chatbots Failed to Assist Job Applicants with Scheduling Interviews
- RealPage Algorithm Allegedly Inflates Rents and Reduces Competition in Housing Market
- Amazon Flex Drivers Allegedly Fired via Automated Employee Evaluations
- Kronos Scheduling Algorithm Allegedly Caused Financial Issues for Starbucks Employees
Other incidents involving Openai
- Nippon Life Alleged ChatGPT Practiced Law Without a License in Illinois Disability Case
- ChatGPT Reportedly Found to Reproduce Protected German Lyrics in Copyright Case
- Alleged Harmful Health Outcomes Following Reported Use of Purported ChatGPT-Generated Medical Advice in Hyderabad
- Lawsuit Alleged ChatGPT (GPT-4o) Encouraged Colorado Man's Suicide After Prolonged 'AI Companion' Chats
- Large-Scale Mental Health Crises Allegedly Associated with ChatGPT Interactions
- OpenAI ChatGPT Models Reportedly Jailbroken to Provide Chemical, Biological, and Nuclear Weapons Instructions
Source record: incident #450 on the AI Incident Database · all 8 reports