AI Incident Database
Browse AI incidents
Every incident record (metadata only) from the weekly snapshot of 2026-09-07. Filter, then export the selection with its licence attached; each row links to the full record and its reports.
-
TikTok’s Content Moderation Allegedly Failed to Adequately Take down Videos Promoting Eating Disorders AIID ↗
Videos promoting eating disorders evaded TikTok's automated violation detection system without difficulty via common misspellings of search terms, bypassing its ban of violating hashtags such as "proana" and "anorexia".
-
Robot in Chinese Shopping Mall Fell off the Escalator, Knocking down Passengers AIID ↗
A shopping guide robot deployed by the Fuzhou Zhongfang Marlboro Mall was shown on video allegedly walking to the escalator by itself, falling down, and knocking over passengers, which prompted its suspension.
-
FaceApp Predicted Different Genders for Similar User Photos with Slight Variations AIID ↗
FaceApp’s algorithm was reported by a user to have predicted different genders for two mostly identical facial photos with only a slight difference in eyebrow thickness.
-
Korean Chatbot Luda Reportedly Made Offensive Remarks Towards Minority Groups in South Korea AIID ↗
A Korean interactive chatbot was allegedly shown in screenshots to have used derogatory and bigoted language when asked about lesbians, Black people, and people with disabilities.
-
Frontline workers protest at Stanford after hospital distributed vaccine to administrators AIID ↗
In 2020, Stanford Medical Center's distribution algorithm only designated 7 of 5,000 vaccines to Medical Residents, who are frontline workers regularly exposed to COVID-19.
-
Japanese Porn Depixelated by Man using Deepfake AIID ↗
A man allegedly unblurred, using deepfake technology, pixelated pornographic images and videos of pornographic actors, which violated Japan’s obscenity law requiring images of genitalia to be obscured.
-
Online Trolls Allegedly Abused TikTok’s Automated Content Reporting System to Discriminate against Marginalized Creators AIID ↗
TikTok's automated content reporting system was allegedly abused by online trolls to intentionally misreport content created by users of marginalized groups.
-
Brand Safety Tech Firms Falsely Claimed Use of AI, Blocking Ads Using Simple Keyword Lists AIID ↗
Brand safety tech firms falsely claimed use of AI, blocking ads using simple keyword lists.
-
Proctoring Algorithm in Online California Bar Exam Flagged an Unusually High Number of Alleged Cheaters AIID ↗
The proctoring algorithm used in a California bar exam cited a third of thousands of applicants as cheaters, resulting in allegations where exam takers were instructed to prove otherwise without seeing their incriminating video evidence.
-
Court Rules Deliveroo Used 'Discriminatory' Algorithm AIID ↗
In December 2020, an Italian court ruled that Deliveroo’s employee ‘reliability’ algorithm illegally discriminated against workers with legitimate reasons for cancelling shifts.
-
Poachers Evaded AI Cameras and Killed Four Rhinos AIID ↗
AI cameras installed by Ezemvelo KZN Wildlife failed to detect poachers when four dehorned rhino carcasses were found.
-
Facebook Mistakenly Blocked Small Business Ads AIID ↗
Facebook’s AI mistakenly blocked advertisements by small and struggling businesses, after the company allegedly leaned more on algorithms to monitor ads on the platform with little review from human moderators.
-
YouTube Recommendation Reportedly Pushed Election Fraud Content to Skeptics Disproportionately AIID ↗
YouTube's recommendation algorithm allegedly pushed 2020's US Presidential Election fraud content to users most skeptical of the election's legitimacy disproportionately compared to least skeptical users.
-
Autonomous Roborace Car Drove Directly into a Wall AIID ↗
An autonomous Roborace car drove itself into a wall in round one of the Season Beta 1.1 race.
-
OpenAI’s GPT-3 Reported as Unviable in Medical Tasks by Healthcare Firm AIID ↗
The French digital care company, Nabla, in researching GPT-3’s capabilities for medical documentation, diagnosis support, and treatment recommendation, found its inconsistency and lack of scientific and medical expertise unviable and risky in healthcare applications. This incident has been downgraded to an issue as it does not meet current ingestion criteria.
-
Meta Allegedly Used Books3, a Dataset of 191,000 Pirated Books, to Train LLaMA AI AIID ↗
Meta and Bloomberg allegedly used Books3, a dataset containing 191,000 pirated books, to train their AI models, including LLaMA and BloombergGPT, without author consent. Lawsuits from authors such as Sarah Silverman and Michael Chabon claim this constitutes copyright infringement. Books3 includes works from major publishers like Penguin Random House and HarperCollins. Meta argues its AI outputs are not "substantially similar" to the original books, but legal challenges continue.
-
AI mistakes referee’s bald head for football — hilarity ensued AIID ↗
In a Scottish soccer match the AI-enabled ball-tracking camera used to livestream the game repeatedly tracked an official’s bald head as though it were the soccer ball.
-
Tesla Autopilot Mistakes Red Letters on Flag for Red Traffic Lights AIID ↗
A Tesla Model 3 misidentified flags with "COOP" written vertically on them as traffic lights.
-
AI Spam Filters Allegedly Block Legitimate Emails Based on Biased Keyword Detection AIID ↗
AlgorithmWatch tested spam filtering algorithms across Gmail, Yahoo, Outlook, GMX, and LaPoste. Their findings reportedly showed that Microsoft Outlook’s spam filter flagged emails based on specific keywords that led to racial and content-based biases blocking legitimate communications. Emails mentioning Nigeria or containing certain financial and sexual health terms were found to be disproportionately marked as spam.
-
#LekkiMassacre: Why Facebook labelled content from October 20 incident ‘false’ AIID ↗
Facebook incorrectly labels content relating to an incident between #EndSARS protestors and the Nigerian army as misinformation.
-
Researchers find evidence of racial, gender, and socioeconomic bias in chest X-ray classifiers AIID ↗
A study by the University of Toronto, the Vector Institute, and MIT showed the input databases that trained AI systems used to classify chest X-rays led the systems to show gender, socioeconomic, and racial biases.
-
Tesla Disabled “Rolling Stop” Functionality Associated with the “Aggressive” Driving Mode AIID ↗
The “rolling stop” functionality within the “Aggressive” Full Self Driving (FSD) profile that was released via a Tesla firmware update was recalled and disabled.
-
AI attempts to ease fear of robots, blurts out it can’t ‘avoid destroying humankind’ AIID ↗
On September 8, 2020, the Guardian published an op-ed generated by OpenAI’s GPT-3 text generating AI that included threats to destroy humankind. This incident has been downgraded to an issue as it does not meet current ingestion criteria.
-
Tiny Changes Let False Claims About COVID-19, Voting Evade Facebook Fact Checks AIID ↗
Avaaz, an international advocacy group, released a review of Facebook's misinformation identifying software showing that the labeling process failed to label 42% of false information posts, most surrounding COVID-19 and the 2020 USA Presidential Election.
-
Ireland's 2020 Leaving Certificate Calculated-Grades Model Reportedly Issued Incorrect Scores to Thousands of Students AIID ↗
After Covid-19 disrupted the 2020 Leaving Certificate exams, Ireland's Department of Education and Skills used a calculated-grades model to issue student results on September 7. The Department later confirmed coding errors in how the model handled Junior Cycle data, resulting in incorrect grades. About 6,100 students received upgraded results, and college-offer eligibility had to be reassessed.
-
UK passport photo checker shows bias against dark-skinned women AIID ↗
UK passport photo checker shows bias against dark-skinned women.
-
Korean Internet Portal Giant Naver Manipulated Shopping and Video Search Algorithms to Favor In-House Services AIID ↗
The Korean Fair Trade Commission (FTC) imposed a 26.7B KRW on Naver for manipulating shopping and video search algorithms, favoring its own online shopping business to boost its market share.
-
Facebook’s Algorithm Mistook an Advertisement of Onions as Sexual Suggestive Content AIID ↗
Facebook’s content moderation algorithm misidentified and removed a Canadian business’s advertisement containing a photo of onions as products of overtly sexual content, which was later reinstated after review.
-
Amazon’s Robotic Fulfillment Centers Have Higher Serious Injury Rates AIID ↗
Amazon’s robotic fulfillment centers have higher serious injury rates.
-
Suicide Clips Evaded TikTok's Automated Moderation in Coordinated Attack AIID ↗
Many clips showing a suicide evaded TikTok's automated content moderation system allegedly in a coordinated attack, which resulted in exposure of violating content to its users.
-
Twitter’s Image Cropping Tool Allegedly Showed Gender and Racial Bias AIID ↗
Twitter's photo cropping algorithm was revealed by researchers to favor white and women faces in photos containing multiple faces, prompting the company to stop its use on mobile platform.
-
Philosophy AI Tentatively Produced Offensive Results for Certain Prompts AIID ↗
Philosopher AI as built on top of GPT-3 was reported by its users for having strong tendencies to produce offensive results when given prompts on certain topics such as feminism and Ethiopia.
-
Philosophy AI Allegedly Used To Generate Mixture of Innocent and Harmful Reddit Posts AIID ↗
Philosopher AI, a GPT-3-powered controversial text generator, was allegedly used by an anonymous actor on AskReddit subreddit, whose posts featured a mixture of harmless stories, conspiracy theories, and sensitive topic discussions.
-
UK Ofqual's Algorithm Disproportionately Provided Lower Grades Than Teachers' Assessments AIID ↗
UK Office of Qualifications and Examinations Regulation (Ofqual)'s grade-standardization algorithm providing predicted grades for A level and GCSE qualifications in the UK, Wales, Northern Ireland, and Scotland was reportedly giving grades lower than teachers' assessments, and disproportionately for state schools.
-
Activists Allege NYPD's Application of Facial Recognition Interfered with Right to Protest AIID ↗
Black Lives Matter activists alleged being targeted for arrest by New York Police using facial recognition, interfering with their right to protest.
-
OpenAI's GPT-3 Associated Muslims with Violence AIID ↗
Users and researchers revealed generative AI GPT-3 associating Muslims to violence in prompts, resulting in disturbingly racist and explicit outputs such as casting Muslim actor as a terrorist.
-
Bug in Instagram’s “Related Hashtags” Algorithm Allegedly Caused Disproportionate Treatment of Political Hashtags AIID ↗
A bug was reported by Instagram’s spokesperson to have prevented an algorithm from populating related hashtags for thousands of hashtags, resulting in an allege preferential treatment for some politically partisan hashtags.
-
Colorado Police’s Automated License Plate Reader (ALPR) Matched a Family’s Minivan’s Plate to That of a Stolen Vehicle Allegedly, Resulting in Detainment at Gunpoint AIID ↗
An automated plate reader reportedly matched a license plate information, but of a family’s minivan and an alleged motorcycle in Montana that was reportedly stolen earlier in the year, resulting in them and their children being held at gunpoint and detained in handcuffs by multiple Aurora police officers.
-
Facebook Provided Offensive Translation for King of Thailand's Birthday Ceremony AIID ↗
Facebook's Thai-English translation gave an inappropriate mistranslation on Thai PBS's Facebook live broadcast of the King of Thailand’s candle-lighting birthday ceremony.
-
Genderify’s AI to Predict a Person’s Gender Revealed by Free API Users to Exhibit Bias AIID ↗
A company's AI predicting a person's gender based on their name, email address, or username was reported by its users to show biased and inaccurate results.
-
Thoughts App Allegedly Created Toxic Tweets AIID ↗
Tweets created by Thoughts, a tweet generation app that leverages OpenAI’s GPT-3, allegedly exhibited toxicity when given prompts related to minority groups.
-
Deepfake Images of Australian Teacher Hannah Grundy and 25 Others Created and Circulated Online by Former Friend AIID ↗
Australian teacher Hannah Grundy discovered her face had been superimposed onto pornographic images using AI deepfake technology, which were shared online alongside personal details. A former trusted friend, Andrew Thomas Hayler, was found responsible, targeting 26 women, including Grundy and close friends, over two years. Hayler pleaded guilty to multiple charges and was sentenced to nine years in prison.
-
International Baccalaureate’s 2020 Calculated-Grades Model Reportedly Lowered Students' Scores and College Prospects AIID ↗
After Covid-19 canceled May 2020 exams, the Geneva-based International Baccalaureate reportedly used a calculated-grades model based on coursework, predicted grades, and historical school results to estimate final marks. Students, parents, and teachers reportedly said the opaque model produced unexpectedly low scores, threatening university admissions, scholarships, and study plans across multiple countries.
-
WeChat Pay's Facial Recognition Security Evaded by Scammers Using Victims’ Social Media Content AIID ↗
In China, fraudsters bypassed facial-recognition security for online financial transactions on WeChat Pay by crafting identity-verification GIFs of victims from their selfies on WeChat Moments, a social media platform.
-
Facebook’s Political Ad Detection Reportedly Showed High and Geographically Uneven Error Rates AIID ↗
The performance of Facebook’s political ad detection was revealed by researchers to be imprecise, uneven across countries in errors, and inadequate for preventing systematic violations of political advertising policies.
-
NarxCare’s Risk Score Model Allegedly Lacked Validation and Trained on Data with High Risk of Bias AIID ↗
NarxCare's overdose risk algorithm, lacking peer-reviewed validation, uses sensitive data like doctor visits, prescriptions, and possibly genetic information, leading to significant biases against women and Black patients. Factors like sexual abuse and criminal records exacerbate stigmas and disparities, often resulting in unjust denial of necessary pain medication. The newly approved AvertD genetic test shares similar issues, further complicating and potentially harming medical treatment decisi
-
Reported AI-Assisted Influence Campaigns by GoLaxy Allegedly Targeting Hong Kong and Taiwan Political Discourse AIID ↗
Reportedly leaked documents reviewed by researchers and media allege that Chinese firm GoLaxy used its "GoPro" AI system in multiple influence campaigns. In 2020, it reportedly tracked 180,000 Twitter accounts to counter opposition to Hong Kong's National Security Law; in 2022–2023, it amplified praise for China's Zero Covid policy using purportedly fake Facebook accounts; and in late 2023, allegedly it advised on messaging to weaken Taiwan's Democratic Progressive Party.
-
YouTube's AI Mistakenly Banned Chess Channel over Chess Language Misinterpretation AIID ↗
YouTube's AI-powered hate speech detection system falsely flagged chess content and banned chess creators allegedly due to its misinterpretation of strategy language such as "black," "white," and "attack" as harmful and dangerous.
-
Facebook's AI Put "Primates" Label on Video Featuring Black Men AIID ↗
Facebook's AI mislabeled video featuring Black men as a video about "primates," resulting in an offensive prompt message for users who watched the video.
-
Miami Police Deployed Facial Recognition to Arrest George Floyd Protestor Allegedly without Cause AIID ↗
Miami Police's arrest report for a George Floyd protestor did not disclose use of facial recognition, which allegedly did not meet the legal threshold for probable cause for arrest.
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.