AI Incident Database
Browse AI incidents
Every incident record (metadata only) from the weekly snapshot of 2026-09-07. Filter, then export the selection with its licence attached; each row links to the full record and its reports.
Filter incidents (1 active)
-
DALL-E Mini Reportedly Reinforced or Exacerbated Societal Biases in Its Outputs as Gender and Racial Stereotypes AIID ↗
Publicly deployed open-source model DALL-E Mini was acknowledged by its developers and found by its users to have produced images which reinforced racial and gender biases.
-
SafeRent AI Screening Tool Allegedly Discriminated Against Housing Voucher Applicants AIID ↗
SafeRent’s AI-powered tenant screening tool used credit history and non-rental-related debts to assign scores, disproportionately penalizing Black and Hispanic renters and those using housing vouchers. The reported discriminatory housing outcomes violated the Fair Housing Act and Massachusetts law. A class action lawsuit (Louis, et al. v. SafeRent Solutions, et al.) resulted in a $2.275 million settlement and changes to SafeRent’s practices.
-
DALL-E 2 Reported for Gender and Racially Biased Outputs AIID ↗
Developers of OpenAI's DALL-E 2 cited risks of the model, varying from misuse as disinformation and explicit content generation, to gender and racial bias.
-
Three Make-Up Artists Lost Jobs Following Black-Box Automated Decision by HireVue AIID ↗
Three make-up artists lost their positions following an algorithmically-assessed video interview by HireVue who reportedly failed to provide adequate explanation of the findings.
-
Honorlock's AI Proctoring System Reportedly Led Broward College to Accuse a Teenager of Cheating AIID ↗
During a February 2022 online biology exam, Honorlock reportedly flagged a Florida teenager taking a Broward College class as suspicious. The student denied cheating, but Broward College allegedly relied on the remote-proctoring system's flag in pursuing an academic-dishonesty accusation that the student said was unfounded.
-
Justice Department’s Recidivism Risk Algorithm PATTERN Allegedly Caused Persistent Disparities Along Racial Lines AIID ↗
Department of Justice’s inmate-recidivism risk assessment tool was reported to have produced racially uneven results, misclassifying risk levels for inmates of color.
-
UK Government AI Allegedly Targets Disproportionate Numbers of Certain Nationals for Fraud Review AIID ↗
The UK's Department for Work and Pensions (DWP) faced scrutiny after many Bulgarian nationals reported unexplained suspensions of their Universal Credit benefits. The MP for Edmonton raised concerns about potential nationality-based targeting for benefit fraud investigations, leading to poverty and homelessness among affected individuals. The Home Office's own equality impact assessment found it was flagging a disproportionate number of marriages from Greece, Albania, Bulgaria and Romania.
-
Startup's Accent Translation AI Denounced as Reinforcing Racial Bias AIID ↗
A startup’s use of AI voice technology to alter or remove accents for call center agents was scrutinized by critics as reaffirming bias, despite the company’s claim.
-
State Farm Allegedly Discriminated against Black Customers in Claim Payout AIID ↗
State Farm's automated claims processing method was alleged in a class action lawsuit to have disproportionately against Black policyholders when paying out insurance claims.
-
Kannada Insulted by Google's Featured Answer as "Ugliest Language in India" AIID ↗
Google's knowledge-graph-powered algorithm showed Kannada in its featured Answer Box when prompted "ugliest language in India," causing outrage from Kannada-speaking people and government.
-
Facebook, Instagram, and Twitter Cited Errors in Automated Systems as Cause for Blocking pro-Palestinian Content on Israeli-Palestinian Conflict AIID ↗
Facebook, Instagram, and Twitter wrongly blocked or restricted millions of pro-Palestinian posts and accounts related to the Israeli-Palestinian conflict, citing errors in their automated content moderation system.
-
Black Uber Eats Driver Allegedly Subjected to Excessive Photo Checks and Dismissed via FRT Results AIID ↗
A lawsuit by a former Uber Eats delivery driver alleged the company to have wrongfully dismissed him due to frequent false mismatches of his verification selfies, and discriminated against him via excessive verification checks.
-
California's Algorithm Considered ZIP Codes in Vaccine Distribution, Allegedly Excluding Low-Income Neighborhoods and Communities of Color AIID ↗
California's vaccine-distribution algorithm used ZIP codes as opposed to census tracts in its decision-making, which critics said undermined equity and access for vulnerable communities who are largely low-income, underserved neighborhoods with low Healthy Places Index scores.
-
Facebook's Advertisement Moderation System Allegedly Routinely Misidentified Adaptive Fashion Products as Medical Equipment and Blocked Their Sellers AIID ↗
Facebook platforms' automated ad moderation system falsely classified adaptive fashion products as medical and health care products and services, resulting in regular bans and appeals faced by their retailers.
-
FaceApp Predicted Different Genders for Similar User Photos with Slight Variations AIID ↗
FaceApp’s algorithm was reported by a user to have predicted different genders for two mostly identical facial photos with only a slight difference in eyebrow thickness.
-
Online Trolls Allegedly Abused TikTok’s Automated Content Reporting System to Discriminate against Marginalized Creators AIID ↗
TikTok's automated content reporting system was allegedly abused by online trolls to intentionally misreport content created by users of marginalized groups.
-
Court Rules Deliveroo Used 'Discriminatory' Algorithm AIID ↗
In December 2020, an Italian court ruled that Deliveroo’s employee ‘reliability’ algorithm illegally discriminated against workers with legitimate reasons for cancelling shifts.
-
AI Spam Filters Allegedly Block Legitimate Emails Based on Biased Keyword Detection AIID ↗
AlgorithmWatch tested spam filtering algorithms across Gmail, Yahoo, Outlook, GMX, and LaPoste. Their findings reportedly showed that Microsoft Outlook’s spam filter flagged emails based on specific keywords that led to racial and content-based biases blocking legitimate communications. Emails mentioning Nigeria or containing certain financial and sexual health terms were found to be disproportionately marked as spam.
-
Twitter’s Image Cropping Tool Allegedly Showed Gender and Racial Bias AIID ↗
Twitter's photo cropping algorithm was revealed by researchers to favor white and women faces in photos containing multiple faces, prompting the company to stop its use on mobile platform.
-
UK Ofqual's Algorithm Disproportionately Provided Lower Grades Than Teachers' Assessments AIID ↗
UK Office of Qualifications and Examinations Regulation (Ofqual)'s grade-standardization algorithm providing predicted grades for A level and GCSE qualifications in the UK, Wales, Northern Ireland, and Scotland was reportedly giving grades lower than teachers' assessments, and disproportionately for state schools.
-
Activists Allege NYPD's Application of Facial Recognition Interfered with Right to Protest AIID ↗
Black Lives Matter activists alleged being targeted for arrest by New York Police using facial recognition, interfering with their right to protest.
-
OpenAI's GPT-3 Associated Muslims with Violence AIID ↗
Users and researchers revealed generative AI GPT-3 associating Muslims to violence in prompts, resulting in disturbingly racist and explicit outputs such as casting Muslim actor as a terrorist.
-
Bug in Instagram’s “Related Hashtags” Algorithm Allegedly Caused Disproportionate Treatment of Political Hashtags AIID ↗
A bug was reported by Instagram’s spokesperson to have prevented an algorithm from populating related hashtags for thousands of hashtags, resulting in an allege preferential treatment for some politically partisan hashtags.
-
Genderify’s AI to Predict a Person’s Gender Revealed by Free API Users to Exhibit Bias AIID ↗
A company's AI predicting a person's gender based on their name, email address, or username was reported by its users to show biased and inaccurate results.
-
Thoughts App Allegedly Created Toxic Tweets AIID ↗
Tweets created by Thoughts, a tweet generation app that leverages OpenAI’s GPT-3, allegedly exhibited toxicity when given prompts related to minority groups.
Disclaimer: informational only, not legal advice. Verify every claim against the linked official sources and consult a qualified lawyer before acting.
Frequently asked questions
What is recorded in each incident?
What happened, when, who was involved as developer or deployer, who was harmed, the sectors and countries affected, and the domain and subdomain the harm falls under. Incidents are mirrored from the AI Incident Database and each links back to its source record.
Does an incident mean wrongdoing was proven?
No. These are reported incidents, not findings of liability. Entities named in a report are named as reported, allegations are marked as alleged, and nothing here is a legal determination.
How current is the incident data?
It syncs directly from the AI Incident Database several times a day, so a new incident usually appears within hours. Each record carries the date it was last synced, and the snapshot date is shown on the page.
Does this page include the full incident reports?
No. The report texts are outside the licence this data is shared under, so only the metadata and the classifications appear here. Every row links to the original incident record, which carries the reports themselves.