AI Incident Database
Browse AI incidents
Every incident record (metadata only) from the weekly snapshot of 2026-09-07. Filter, then export the selection with its licence attached; each row links to the full record and its reports.
-
University of Toronto's ProctorU Deployment Reportedly Disadvantaged BIPOC Students During Exam Check-Ins AIID ↗
University of Toronto students reported that ProctorU and other online exam-monitoring services created discriminatory and stressful check-in experiences for BIPOC students during remote exams. One U of T student said ProctorU's identity-verification process often failed to recognize her passport, forcing manual checks and consuming exam check-in time. ProctorU said human proctors made final identity and integrity decisions.
-
University of Illinois' Proctorio Remote-Proctoring Deployment Reportedly Raised Student Rights Concerns AIID ↗
Beginning in March 2020, the University of Illinois Urbana-Champaign deployed Proctorio for remote exam proctoring after the COVID-19 shift to online instruction. Students and faculty raised concerns that the software surveilled students' homes and devices, created accessibility barriers, increased testing anxiety, and could flag students when facial-detection tools failed to detect their faces, with particular concern for students of color. UIUC later declined to renew its emergency contract.
-
Personal voice assistants struggle with black voices, new study shows AIID ↗
A study found that voice recognition tools from Apple, Amazon, Google, IBM, and Microsoft disproportionately made errors when transcribing black speakers.
-
Canadian Universities' AI Proctoring Tools Reportedly Collected Student Biometric Data Without Adequate Consent AIID ↗
During the COVID-19 pandemic, Canadian universities reportedly used AI-enabled remote-proctoring tools for online exams. A University of Ottawa project supported by the Office of the Privacy Commissioner of Canada found that tools such as Respondus Monitor, ProctorU, ProctorTrack, Proctorio, ProctorExam, and Examity collected student personal or biometric data under conditions not conducive to meaningful consent.
-
TikTok's "Suggested Accounts" Algorithm Allegedly Reinforced Racial Bias through Feedback Loops AIID ↗
TikTok's "Suggested Accounts" recommendations allegedly reinforced racial bias despite not basing recommendations on race or creators' profile photo.
-
Algorithm Used by the Malaysian Judiciary Reportedly Recommended Unusually High Sentencing to a Drug Possession Case AIID ↗
The AI system used by the Malaysian judiciary which explicitly considered age, employment, and socio-economic data provided sentencing to a drug possession case that was alleged by lawyer to be disproportionately high for the crime committed.
-
Networking Platform Giggle Employs AI to Determine Users’ Gender, Allegedly Excluding Transgender Women AIID ↗
A social networking platform, Giggle, allegedly collected, shared to third-parties, and used sensitive information and biometric data to verify whether a person is a woman via facial recognition, which critics claimed to be discriminatory against women of color and harmful towards trans women.
-
Detroit Police Allegedly Wrongfully Arrested Black Man Due to Purportedly Faulty Facial Recognition Technology AIID ↗
A Black man was allegedly wrongfully detained by the Detroit Police Department as a result of a purportedly false facial recognition technology (FRT) result.
-
SN Technologies Reportedly Lied to a New York State School District about Its Facial and Weapon Detection Systems’ Performance AIID ↗
SN Technologies allegedly misled Lockport City Schools about the performance of its AEGIS face and weapons detection systems, downplaying error rates for Black faces and weapon misidentification.
-
Facebook Feed Algorithms Exposed Low Digitally Skilled Users to More Disturbing Content AIID ↗
Facebook feed algorithms were known by internal research to have harmed people having low digital literacy by exposing them to disturbing content they did not know how to avoid or monitor.
-
Apple Card's Credit Assessment Algorithm Allegedly Discriminated against Women AIID ↗
Apple Card's credit assessment algorithm was reported by Goldman-Sachs customers to have shown gender bias, in which men received significantly higher credit limits than women with equal credit qualifications.
Discrimination and Toxicity Unfair discrimination and misrepre... AI tangible harm event UNITED STATES -
Job Screening Service Halts Facial Analysis of Applicants AIID ↗
In January 2021, HireVue removed the controversial AI expression tracking tool from its virtual job interview software.
-
Optum Algorithmic Health Risk Scores Reportedly Underestimated Black Patients' Needs AIID ↗
Researchers analyzing an Optum algorithm used by a large academic hospital reportedly found evidence that it tended to under-predict the health care needs of Black patients, which in turn appeared to result in lower prioritization for extra care programs compared with white patients exhibiting similar health profiles.
-
Opaque Fraud Detection Algorithm by the UK’s Department of Work and Pensions Allegedly Discriminated against People with Disabilities AIID ↗
People with disabilities were allegedly disproportionately targeted by a benefit fraud detection algorithm which the UK’s Department of Work and Pensions was urged to disclose.
-
Grab Tweaked Matchmaking Algorithm, Providing Preferential Treatment to Drivers Registered with Affiliated Car Rental Service AIID ↗
Grab Indonesia was fined by the Indonesian Competition Commission (KPPU) for unfairly favoring drivers who rented cars via the Grab-affiliated company Teknologi Pengangkutan Indonesia (TPI), including offering more rides via their matchmaking algorithm.
-
Detroit Police Wrongfully Arrested Black Man Due To Faulty Facial Recognition AIID ↗
A Black man was wrongfully detained by the Detroit Police Department as a result of a false facial recognition (FRT) result.
-
Amazon’s "Time Off Task" System Made False Assumptions about Workers' Time Management AIID ↗
Amazon’s warehouse worker “time off task" (TOT) tracking system was used to discipline and dismiss workers, falsely assuming workers to have wasted time and failing to account for breaks or equipment issues.
-
TikTok’s “For You” Algorithm Exposed Young Users to Pro-Eating Disorder Content AIID ↗
TikTok’s young users were allegedly exposed to community-guideline-violating pro-eating disorder content on their algorithmically curated “For You” page that serves videos from any user on its platform.
-
Facebook Allegedly Failed to Police Hate Speech Content That Contributed to Ethnic Violence in Ethiopia AIID ↗
Facebook allegedly did not adequately remove hate speech, some of which was extremely violent and dehumanizing, on its platform including through automated means, contributing to the violence faced by ethnic communities in Ethiopia.
-
Workday's AI Tools Allegedly Enabled Employers to Discriminate against Applicants of Protected Groups AIID ↗
Workday's algorithmic screening systems were alleged in a lawsuit allowing employers to discriminate against African-Americans, people over 40, and people with disabilities.
-
Algorithm to Distribute Social Welfare Reported for Oversimplifying Economic Vulnerability AIID ↗
Takaful cash transfer program's algorithm which ranks families by their economic vulnerability level to determine financial assistance reportedly oversimplified people's economic situation, fueling social tension and perceptions of unfairness.
-
Facebook's Ad Delivery Reportedly Excluded Audience along Racial and Gender Lines AIID ↗
Facebook's housing and employment ad delivery process allegedly resulted in skews in exposure for some users along demographic lines such as gender and racial identity.
-
The Christchurch shooter and YouTube’s radicalization trap AIID ↗
A New Zealand government report released following a right-wing terrorist killing 51 worshippers at two New Zealand mosques which indicated that Youtube's recommendation algorithm played an important role in the terrorist's radicalization.
-
YouTube's Algorithms Failed to Remove Violating Content Related to Suicide and Self-Harm AIID ↗
Terms-of-service-violating videos related to suicide and self-harm reportedly bypassed YouTube’s content moderation algorithms, allegedly resulting in exposure of graphic content to young users via recommended videos.
-
New Jersey Police Wrongful Arrested Innocent Black Man via FRT AIID ↗
Woodbridge Police Department falsely arrested an innocent Black man following a misidentification by their facial recognition software, who was jailed for more than a week and paid thousands of dollar for his defense.
-
Facial Recognition Pilot in Bahia Reportedly Targeted Black and Poor People AIID ↗
Facial recognition deployed in a pilot project by the local government of Bahia despite having minimal hit rate reportedly targeted Black and poor people disproportionately.
-
Schufa Credit Scoring in Germany Reported for Unreliable and Imbalanced Scores AIID ↗
Creditworthiness Schufa scores in Germany reportedly privileged older and female consumers, and people who changed addresses less frequently, and were unreliable depending on scoring version.
-
Emotion Detection Models Showed Disparate Performance along Racial Lines AIID ↗
Emotion detection tools by Face++ and Microsoft's Face API allegedly scored smiling or defaulted ambiguous facial photos for Black faces as negative emotion more often than for white faces.
-
Oregon’s Screening Tool for Child Abuse Cases Discontinued Following Concerns of Racial Bias AIID ↗
Oregon’s Department of Human Services (DHS) stopped using its Safety at Screening Tool, that is aimed to predict the risk that children wind up in foster care or be investigated in the future, and opted for a new process allegedly to reduce disparities and improve racially equitable decision-making.
-
Facebook "News Feed" Allegedly Boosted Misinformation and Violating Content Following Use of MSI Metric AIID ↗
After the “News Feed” algorithm had been overhauled to boost engagement between friends and family in early 2018, its heavy weighting of re-shared content was alleged found by company researchers to have pushed content creators to reorient their posts towards outrage and sensationalism, causing a proliferation of misinformation, toxicity, and violent content.
-
Dutch Families Wrongfully Accused of Tax Fraud Due to Discriminatory Algorithm AIID ↗
A childcare benefits system in the Netherlands falsely accused thousands of families of fraud, in part due to an algorithm that treated having a second nationality as a risk factor.
-
Facebook Allegedly Failed to Police Anti-Rohingya Hate Speech Content That Contributed to Violence in Myanmar AIID ↗
Facebook allegedly did not adequately remove anti-Rohingya hate speech, some of which was extremely violent and dehumanizing, on its platform, contributing to the violence faced by Rohingya communities in Myanmar.
-
HUD charges Facebook with enabling housing discrimination AIID ↗
In March 2019 the U.S. Department of Housing and Urban Development charged Facebook with violating the Fair Housing Act by allowing real estate sellers to target advertisements in a discriminatory manner.
-
Amazon's Rekognition Falsely Matched Members of Congress to Mugshots AIID ↗
Rekognition's face comparison feature was shown by the ACLU to have misidentified members of congress, and particularly members of colors, as other people who have been arrested using a mugshot database built on publicly available arrest photos.
-
Chinese Tech Firms Allegedly Developed Facial Recognition to Identify People by Race, Targeting Uyghur Muslims AIID ↗
Various Chinese firms were revealed by patent applications to have developed facial recognition capable of detecting people by race, which critics feared would enable persecution and discrimination of Uyghur Muslims.
-
Transgender Uber Drivers Mistakenly Kicked off App for Appearance Change during Gender Transitions AIID ↗
Transgender Uber drivers reported being automatically deactivated from the app due to Real-Time ID Check failing to account for difference in appearance of people undergoing gender transitions.
-
Content Using Bestiality Thumbnails Allegedly Evaded YouTube’s Thumbnail Monitoring System AIID ↗
YouTube’s thumbnail monitoring system was allegedly evaded by content farms such as ones in Cambodia who spike viewership and generate ad revenue using bestiality-themed thumbnails.
-
Argentinian City Government Deployed Teenage-Pregnancy Predictive Algorithm Using Invasive Demographic Data AIID ↗
In 2018, during the abortion-decriminalization debate in Argentina, the Salta city government deployed a teenage-pregnancy predictive algorithm built by Microsoft that allegedly lacked a defined purpose, explicitly considered sensitive information such as disability and whether their home had access to hot water.
-
Man Arrested For Sock Theft by False Facial Match Despite Alibi AIID ↗
A man was arrested for theft of socks from a TJ Maxx store under the guise of an eyewitness ID case, after the local police asked the store's security guard to confirm the facial recognition match produced using surveillance footage, despite him having an alibi at the time of the theft.
-
Instagram's Exposure of Harmful Content Allegedly Contributed to Teenage Girl's Suicide AIID ↗
Instagram was ruled by a judge to have contributed to the death of a teenage girl in the UK allegedly through its exposure and recommendation of suicide, self-harm, and depressive content.
-
Robot Deployed by Animal Shelter to Patrol Sidewalks outside Its Office, Warding off Homeless People in San Francisco AIID ↗
Society for the Prevention of Cruelty to Animals (SPCA) deployed a Knightscope robot to autonomously patrol the area outside its office and ward off homeless people, which was criticized by residents as a tool of intimidation and ordered by the city of San Francisco to stop its use on a public right-of-way.
-
Biased Sentiment Analysis AIID ↗
Google Cloud's Natural Language API provided racist, homophobic, amd antisemitic sentiment analyses.
-
Russian Chatbot Supports Stalin and Violence AIID ↗
Yandex, a Russian technology company, released an artificially intelligent chat bot named Alice which began to reply to questions with racist, pro-stalin, and pro-violence responses
-
WeChat’s Machine Translation Gave a Racist English Translation for the Chinese Term for “Black Foreigner” AIID ↗
The Chinese platform WeChat provided an inappropriate and racist English translation for the Chinese term for “black foreigner” in its messaging app.
-
Offensive Instagram User Content Displayed as Facebook Ad AIID ↗
An Instagram user’s image containing violent content was reportedly used as advertisement on Facebook allegedly via automated means.
-
AI-Assisted Body Scanners Reportedly Subjected Transgender Travelers to Invasive Body Searches AIID ↗
Transportation Security Administration (TSA)'s use of image-processing body scanners at airports led transgender and gender-nonconforming travelers to be subjected to allegedly discriminatory and invasive searches, such as being asked to remove undergarments in private rooms by officers not of their gender.
-
AI-Designed Phone Cases Are Unexpected AIID ↗
A third-party Amazon merchant named “my_handy_design” was suspected of using a bot to generate cell phone case designs based on the bizarre and unattractive designs being offered.
-
Airbnb's Trustworthiness Algorithm Allegedly Banned Users without Explanation, and Discriminated against Sex Workers AIID ↗
Airbnb allegedly considered publicly available data on users to gauge their trustworthiness via algorithmic assessment of personality and behavioral traits, resulting in unexplained bans and discriminatory bans against sex workers.
-
-
Gender Biases in Google Translate AIID ↗
A Cornell University study in 2016 highlighted Google Translate's pattern of assigning gender to occupations in a way showing an implicit gender bias against women.
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.