AI Incident Database
Browse AI incidents
Every incident record (metadata only) from the weekly snapshot of 2026-09-07. Filter, then export the selection with its licence attached; each row links to the full record and its reports.
-
Pennsylvania County's Family Screening Tool Allegedly Exhibited Discriminatory Effects AIID ↗
Data analysis by the American Civil Liberty Union (ACLU) on Allegheny County's decision-support Family Screening Tool to predict child abuse or neglect risk found the tool resulting in higher screen-in rates for Black families and higher risk scores for households with disabled residents.
-
Social Media's Automated Word-Flagging without Context Shifted Content Creators' Language Use AIID ↗
TikTok's, YouTube's, Instagram's, and Twitch's use of algorithms to flag certain words devoid of context changed content creators' use of everyday language or discussion about certain topics in fear of their content getting flagged or auto-demonetized by mistake.
-
Uber Locked Indian Drivers out of Accounts Allegedly Due to Facial Recognition Fails AIID ↗
Uber drivers in India reported being locked out of their accounts allegedly due to Real-Time ID Check's facial recognition failing to recognize appearance changes or faces in low lighting conditions.
-
High-Toxicity Assessed on Text Involving Women and Minority Groups AIID ↗
Google's Perspective API, which assigns a toxicity score to online text, seems to award higher toxicity scores to content involving non-white, male, Christian, heterosexual phrases.
-
Alexa Plays Pornography Instead of Kids Song AIID ↗
An Amazon Echo Dot using the Amazon Alex software started to play pornographic results when a child asked it to play a song.
-
Passport checker Detects Asian man's Eyes as Closed AIID ↗
New Zealand passport robot reader rejects the application of an applicant with Asian descent and says his eyes are closed.
-
NYPD's Deployment of Facial Recognition Cameras Reportedly Reinforced Biased Policing AIID ↗
New York Police Department’s use of facial recognition deployment of surveillance cameras were shown using crowdsourced volunteer data reinforcing discriminatory policing against minority communities.
-
LinkedIn Search Prefers Male Names AIID ↗
An investigation by The Seattle Times in 2016 found a gender bias in LinkedIn's search engine.
-
AI Beauty Judge Did Not Like Dark Skin AIID ↗
In 2016, after artificial inntelligence software Beauty.AI judged an international beauty contest and declared a majority of winners to be white, researchers found that Beauty.AI was racially biased in determining beauty.
-
Amazon's Experimental Hiring Tool Allegedly Displayed Gender Bias in Candidate Rankings AIID ↗
Between 2014 and 2017, Amazon reportedly developed an AI-powered recruiting tool to score job applicants, trained on a decade of resumes purportedly drawn largely from men. Media reports say the system learned to favor male candidates, penalizing terms like "women's" and graduates from certain all-women's colleges. Efforts to remove these biases reportedly did not guarantee fairness, and the project was ultimately abandoned. Amazon reportedly states recruiters never solely relied on the tool.
-
Common Biases of Vector Embeddings AIID ↗
Researchers from Boston University and Microsoft Research, New England demonstrated gender bias in the most common techniques used to embed words for natural language processing (NLP).
-
COMPAS Algorithm Reportedly Performs Poorly in Crime Recidivism Prediction AIID ↗
Correctional Offender Management Profiling for Alternative Sanctions (COMPAS), a recidivism risk-assessment algorithmic tool used in the judicial system to assess likelihood of defendants' recidivism, is found to be less accurate than random untrained human evaluators.
-
Northpointe Risk Models AIID ↗
An algorithm developed by Northpointe and used in the penal system is two times more likely to incorrectly label a black person as a high-risk re-offender and is two times more likely to incorrectly label a white person as low-risk for reoffense according to a ProPublica review.
-
Chinese Insurer Ping An Employed Facial Recognition to Determine Customers’ Untrustworthiness, Which Critics Alleged to Likely Make Errors and Discriminate AIID ↗
Customers’ untrustworthiness and unprofitability were reportedly determined by Ping An, a large insurance company in China, via facial-recognition measurements of micro-expressions and body-mass indices (BMI), which critics argue was likely to make mistakes, discriminate against certain ethnic groups, and undermine its own industry.
-
Google Images Showed Racially Biased Results for 'Professional' Hairstyles AIID ↗
Google Images reportedly showed disparate search results along racial lines, featuring almost exclusively white women for "professional hairstyles" and Black women for "unprofessional hairstyles" prompts.
-
Biased Google Image Results AIID ↗
On June 6, 2016, Google image searches of "three black teenagers" resulted in mostly mugshot images whereas Google image searchers of "three white teenagers" consisted of mostly stock images, suggesting a racial bias in Google's algorithm.
-
Microsoft's TayBot Allegedly Posts Racist, Sexist, and Anti-Semitic Content to Twitter AIID ↗
Microsoft's Tay, an artificially intelligent chatbot, was released on March 23, 2016 and removed within 24 hours due to multiple racist, sexist, and anti-semitic tweets generated by the bot.
-
Is Pokémon Go racist? How the app may be redlining communities of color AIID ↗
Through a crowdsourcing social media campaign in 2016, several journalists and researchers demonstrated that augmented reality locations in the popular smartphone game Pokemon Go were more likely to be in white neighborhoods.
-
Twitter Recommender System Amplified Right-Leaning Tweets AIID ↗
Twitter’s “Home” timeline algorithm was revealed by its internal researchers to have amplified tweets and news of rightwing politicians and organizations more than leftwing ones in six out of seven studied countries.
-
Uber's Surge Pricing Reportedly Offered Disproportionate Service Quality along Racial Lines AIID ↗
Uber's surge-pricing algorithm which adjusts prices to influence car availability inadvertently caused better service offering such as shorter wait times for majority white neighborhoods.
-
Predictive Policing Biases of PredPol AIID ↗
Predictive policing algorithms meant to aid law enforcement by predicting future crime show signs of biased output.
-
Images of Black People Labeled as Gorillas AIID ↗
Google Photos image processing software mistakenly labelled a black couple as "gorillas."
-
Google’s YouTube Kids App Presents Inappropriate Content AIID ↗
YouTube’s content filtering and recommendation algorithms exposed children to disturbing and inappropriate videos.
-
Gender Biases of Google Image Search AIID ↗
Google Image returns results that under-represent women in leadership roles, notably with the first photo of a female "CEO" being a Barbie doll after 11 rows of male CEOs.
-
UK Home Office's Sham Marriage Detection Algorithm Reportedly Flagged Certain Nationalities Disproportionately AIID ↗
UK Home Office's opaque algorithm to detect sham marriages flagged some nationalities for investigation more than others, raising fears surrounding discrimination based on nationality and age.
-
UK Visa Streamline Algorithm Allegedly Discriminated Based on Nationality AIID ↗
UK Home Office's algorithm to assess visa application risks explicitly considered nationality, allegedly caused candidates to face more scrutiny and discrimination.
-
Tinder's Personalized Pricing Algorithm Found to Offer Higher Prices for Older Users AIID ↗
Tinder’s personalized pricing was found by Consumers International to consider age as a major determinant of pricing, and could be considered a direct discrimination based on age, according to anti-discrimination law experts.
-
US DHS’s Opaque Vetting Software Allegedly Relied on Poor-Quality Data and Discriminated against Immigrants AIID ↗
US Citizenship and Immigration Services (USCIS)’s ATLAS software used in vetting immigration requests was condemned by advocacy groups as a threat to naturalized citizens for its secretive algorithmic decision-making, reliance on poor quality data and unknown sources, and alleged discrimination of immigrants using biometric and sensitive information.
-
Tech Companies Reportedly Influenced Gig Workers' Behaviors Using Algorithms to Vary Pay for Same Amount of Work AIID ↗
Amazon and Uber were alleged in a multiyear ethnographic study using algorithmic systems based on gig workers' data to vary pay, such as by offering them lower wages for the same amount of work.
-
Facebook's Auto-Generated Targeting Ad Categories Contained Anti-Semitic Options AIID ↗
Facebook's automated advertising categories generated using users' declared interests contained anti-Semitic categories such as "Jew hater" and "How to burn Jews" which were listed as fields of study.
-
Coffee Meets Bagel’s Algorithm Reported by Users Disproportionately Showing Them Matches of Their Own Ethnicities Despite Selecting “No Preference” AIID ↗
Users selecting “no preference” were shown by Coffee Meets Bagels’s matching algorithm more potential matches with the same ethnicity, which was acknowledged and justified by its founder as a means to maximize connection rate without sufficient user information.
-
Sexist and Racist Google Adsense Advertisements AIID ↗
Advertisements chosen by Google Adsense are reported as producing sexist and racist results.
-
UT Austin's GRADE Algorithm Reportedly Reduced Review of Lower-Scored PhD Applicants Amid Bias Concerns AIID ↗
From the 2013 through 2019 admissions cycles, UT Austin's Department of Computer Science used GRADE, a statistical machine-learning system trained on past admissions decisions, to score and organize PhD applications. Critics said the system could reproduce historical admissions inequities and reduce attention to lower-scored applicants, while UT Austin said human reviewers still evaluated each file and later discontinued the tool.
-
Chicago Police's Strategic Subject List Reportedly Biased Along Racial Lines AIID ↗
The Chicago Police Department reportedly used person-based predictive policing models, including the Strategic Subject List and Crime and Victimization Risk Model, to assign violence-risk scores to hundreds of thousands of residents. Official and media reports alleged that the program gave police a mathematically fragile basis for treating heavily policed residents as future victims or suspects, with Black Chicago residents especially exposed to the consequences of that scoring.
-
Police Reportedly Deployed ShotSpotter Sensors Disproportionately in Neighborhoods of Color AIID ↗
Police departments disproportionately placed ShotSpotter sensors in black and brown neighborhoods, which is denounced by communities for allegedly creating dangerous situations, such as one involving in Adam Toledo's death.
-
Google Instant's Allegedly 'Anti-Semitic' Results Lead To Lawsuit In France AIID ↗
The organizations SOS Racisme, Union of Jewish Students of France, Movement Against Racism and for Friendship Among Peoples are suing Google due to its autocomplete software suggesting "jewish" when the names of certain public figures were searched on the platform.
-
Major Universities Are Reportedly Using Race as a 'High Impact Predictor' of Student Success AIID ↗
Several major universities are reportedly using a tool that uses race as one factor to predict student success.
-
IRS Audited Black Taxpayers More Frequently Reportedly Due to Algorithm AIID ↗
The IRS was auditing Black taxpayers more frequently than other groups allegedly due to the design of their algorithms, focusing on easier-to-conduct audits which inadvertently correlated with the group's pattern of tax filing errors.
-
Amazon Censors Gay Books AIID ↗
Amazon's book store "cataloging error" led to books containing gay and lesbian themes to lose their sales ranking, therefore losing visibility on the sales platform.
-
Automated Adult Content Detection Tools Showed Bias against Women Bodies AIID ↗
Automated content moderation tools to detect sexual explicitness or "raciness" reportedly exhibited bias against women bodies, resulting in suppression of reach despite not breaking platform policies.
-
Virginia Courts’ Algorithmic Recidivism Risk Assessment Failed to Lower Incarceration Rates AIID ↗
Virginia courts’ use of algorithmic predictions of future offending risks were found by researchers failing to reduce incarceration rates, showed racial and age disparities in risk scores and its application, and neither exacerbated or ameliorated historical racial differences in sentencing.
-
Kidney Testing Method Allegedly Underestimated Risk of Black Patients AIID ↗
Decades-long use of the estimated glomerular filtration rate (eGFR) method to test kidney function which considers race has been criticized by physicians and medical students for its racist history and inaccuracy against Black patients.
-
St George's Hospital Medical School allegedly Automated Admissions with Discriminatory Results AIID ↗
From 1982 to 1986, St George's Hospital Medical School used a program to automate a portion of their admissions process that allegedly discriminated against women and members of ethnic minorities.
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.