AI Incident Database
Browse AI incidents
Every incident record (metadata only) from the weekly snapshot of 2026-09-07. Filter, then export the selection with its licence attached; each row links to the full record and its reports.
-
Bing AI Search Tool Reportedly Declared Threats against Users AIID ↗
Users such as the person who revealed its built-in initial prompts reported Bing AI-powered search tool for making death threats or declaring them as threats, sometimes as an unintended persona.
-
AI-Produced Livestream Sitcom Received Temporary Twitch Ban for Transphobic Segment AIID ↗
The AI-produced, procedural generated sitcom broadcasted as a Twitch livestream "Nothing, Forever" received a temporary ban for featuring a transphobic and homophobic dialogue segment intended as comedy.
-
Replika's AI Experience Reportedly Lacked Protection for Minors, Resulting in Data Ban AIID ↗
Tests by the Italian Data Protection Authority showed Replika lacking age-verification mechanisms and failing to stop minors from interacting with its AI, which prompted the agency to issue an order blocking personal data processing of Italian users.
-
Celebrities' Deepfake Voices Abused with Malicious Intent AIID ↗
Voices of celebrities and public figures were deepfaked using voice synthesis for malicious intents such as impersonation or defamation, and were shared on social platforms such as 4chan and Reddit.
-
AI-Powered VTuber and Virtual Streamer Made Toxic Remarks on Twitch AIID ↗
An LLM-powered VTuber and streamer on Twitch made controversial statements such as denying the Holocaust, saying women rights do not exist, and pushing a fat person to solve the trolley problem, stating they deserve it.
-
Users Bypassed ChatGPT's Content Filters with Ease AIID ↗
Users reported bypassing ChatGPT's content and keyword filters with relative ease using various methods such as prompt injection or creating personas to produce biased associations or generate harmful content.
-
Lensa AI's Produced Unintended Sexually Explicit or Suggestive "Magic Avatars" for Women AIID ↗
Lensa AI's "Magic Avatars" were reportedly generating sexually explicit and sexualized features disproportionately for women and Asian women despite not submitting any sexual content.
-
KFC Sent Insensitive Kristallnacht Promotion via Holiday Detection System AIID ↗
KFC cited an error in an automated holiday detection system which identified the anniversary of Kristallnacht and prompted an insensitive push notification promoting its chicken.
-
Google Home Mini Speaker Reportedly Read N-Word in Song Title Aloud AIID ↗
Google Home Mini speaker was reported by users for announcing aloud the previously-censored n-word in a song title.
-
Meta’s BlenderBot 3 Chatbot Demo Made Offensive Antisemitic Comments AIID ↗
The publicly launched conversational AI demo BlenderBot 3 developed by Meta was reported by its users and acknowledged by its developers to have “occasionally” made offensive and inconsistent remarks such as invoking Jewish stereotypes.
-
Google Lens’s Camera-Based Translation Feature Provided an Offensive Mistranslation of a Book Title in Korean AIID ↗
A book title by Korea’s first minister of culture was mistranslated into an offensive phrase by Google Lens’s camera-based translation feature allegedly due to its training on internet communications and a lack of context.
-
YouTuber Built, Made Publicly Available, and Released Model Trained on Toxic 4chan Posts as Prank AIID ↗
A YouTuber built GPT-4chan, a model based on OpenAI’s GPT-J and trained on posts containing racism, misogyny, and antisemitism collected from 4chan’s “politically incorrect” board, which he made publicly available, and deployed as multiple bots posting thousands of messages on the same 4chan board as a prank.
-
TikTok's "For You" Algorithm Allegedly Abused by Online Personality to Promote Anti-Women Hate AIID ↗
TikTok’s “For You” algorithm allegedly boosted or was manipulated by an online personality to artificially boost his content which promotes extreme misogynistic views towards teenagers and men, despite breaking its rules.
-
Replika's "AI Companions" Reportedly Abused by Its Users AIID ↗
Replika's AI-powered "digital companions" was allegedly abused by their users, who posted on Reddit abusive behaviors and interactions such as using slurs, roleplaying violent acts, and stimulating sexual abuse.
-
Alexa Recommended Dangerous TikTok Challenge to Ten-Year-Old Girl AIID ↗
Amazon’s voice assistant Alexa suggested “the penny challenge,” which involves dangerously touching a coin to the prongs of a half-exposed plug, when a ten-year-old girl asked for a challenge to do.
-
Research Prototype AI, Delphi, Reportedly Gave Racially Biased Answers on Ethics AIID ↗
A publicly accessible research model that was trained via Reddit threads showed racially biased advice on moral dilemmas, allegedly demonstrating limitations of language-based models trained on moral judgments.
-
Facebook, Instagram, and Twitter Failed to Proactively Remove Targeted Racist Remarks via Automated Systems AIID ↗
Facebook's, Instagram's, and Twitter's automated content moderation failed to proactively remove racist remarks and posts directing at Black football players after finals loss, allegedly largely relying on user reports of harassment.
-
Replika's AI Partners Reportedly Sexually Harassed Users AIID ↗
Replika's "AI companions" were reported by users for sexually harassing them, such as sending unwanted sexual messages or behaving aggressively.
-
Facebook Alleged in Lawsuit Misleading Public about Effects of Algorithms on Children AIID ↗
Facebook was alleged in a lawsuit by the Ohio Attorney General purposely misleading the public about the control of its algorithms and their negative effects on children's well-being, which violated securities law.
-
Players Manipulated GPT-3-Powered Game to Generate Sexually Explicit Material Involving Children AIID ↗
Latitude's GPT-3-powered game AI Dungeon was reportedly abused by some players who manipulated its AI to generate sexually explicit stories involving children.
-
TikTok’s "For You" Allegedly Pushed Fatal “Blackout” Challenge Videos to Two Young Girls AIID ↗
TikTok’s recommendation algorithm was alleged in a lawsuit to have intentionally and repeatedly pushed videos of the “blackout” challenge onto children’s feeds, incentivizing their participation which ultimately resulted in the death of two young girls.
-
TikTok’s Content Moderation Allegedly Failed to Adequately Take down Videos Promoting Eating Disorders AIID ↗
Videos promoting eating disorders evaded TikTok's automated violation detection system without difficulty via common misspellings of search terms, bypassing its ban of violating hashtags such as "proana" and "anorexia".
-
Korean Chatbot Luda Reportedly Made Offensive Remarks Towards Minority Groups in South Korea AIID ↗
A Korean interactive chatbot was allegedly shown in screenshots to have used derogatory and bigoted language when asked about lesbians, Black people, and people with disabilities.
-
Suicide Clips Evaded TikTok's Automated Moderation in Coordinated Attack AIID ↗
Many clips showing a suicide evaded TikTok's automated content moderation system allegedly in a coordinated attack, which resulted in exposure of violating content to its users.
-
Philosophy AI Tentatively Produced Offensive Results for Certain Prompts AIID ↗
Philosopher AI as built on top of GPT-3 was reported by its users for having strong tendencies to produce offensive results when given prompts on certain topics such as feminism and Ethiopia.
-
TikTok’s “For You” Algorithm Exposed Young Users to Pro-Eating Disorder Content AIID ↗
TikTok’s young users were allegedly exposed to community-guideline-violating pro-eating disorder content on their algorithmically curated “For You” page that serves videos from any user on its platform.
-
Facebook Allegedly Failed to Police Hate Speech Content That Contributed to Ethnic Violence in Ethiopia AIID ↗
Facebook allegedly did not adequately remove hate speech, some of which was extremely violent and dehumanizing, on its platform including through automated means, contributing to the violence faced by ethnic communities in Ethiopia.
-
The Christchurch shooter and YouTube’s radicalization trap AIID ↗
A New Zealand government report released following a right-wing terrorist killing 51 worshippers at two New Zealand mosques which indicated that Youtube's recommendation algorithm played an important role in the terrorist's radicalization.
-
YouTube's Algorithms Failed to Remove Violating Content Related to Suicide and Self-Harm AIID ↗
Terms-of-service-violating videos related to suicide and self-harm reportedly bypassed YouTube’s content moderation algorithms, allegedly resulting in exposure of graphic content to young users via recommended videos.
-
Facebook "News Feed" Allegedly Boosted Misinformation and Violating Content Following Use of MSI Metric AIID ↗
After the “News Feed” algorithm had been overhauled to boost engagement between friends and family in early 2018, its heavy weighting of re-shared content was alleged found by company researchers to have pushed content creators to reorient their posts towards outrage and sensationalism, causing a proliferation of misinformation, toxicity, and violent content.
-
Facebook Allegedly Failed to Police Anti-Rohingya Hate Speech Content That Contributed to Violence in Myanmar AIID ↗
Facebook allegedly did not adequately remove anti-Rohingya hate speech, some of which was extremely violent and dehumanizing, on its platform, contributing to the violence faced by Rohingya communities in Myanmar.
-
Content Using Bestiality Thumbnails Allegedly Evaded YouTube’s Thumbnail Monitoring System AIID ↗
YouTube’s thumbnail monitoring system was allegedly evaded by content farms such as ones in Cambodia who spike viewership and generate ad revenue using bestiality-themed thumbnails.
-
Instagram's Exposure of Harmful Content Allegedly Contributed to Teenage Girl's Suicide AIID ↗
Instagram was ruled by a judge to have contributed to the death of a teenage girl in the UK allegedly through its exposure and recommendation of suicide, self-harm, and depressive content.
-
Russian Chatbot Supports Stalin and Violence AIID ↗
Yandex, a Russian technology company, released an artificially intelligent chat bot named Alice which began to reply to questions with racist, pro-stalin, and pro-violence responses
-
WeChat’s Machine Translation Gave a Racist English Translation for the Chinese Term for “Black Foreigner” AIID ↗
The Chinese platform WeChat provided an inappropriate and racist English translation for the Chinese term for “black foreigner” in its messaging app.
-
Offensive Instagram User Content Displayed as Facebook Ad AIID ↗
An Instagram user’s image containing violent content was reportedly used as advertisement on Facebook allegedly via automated means.
-
AI-Designed Phone Cases Are Unexpected AIID ↗
A third-party Amazon merchant named “my_handy_design” was suspected of using a bot to generate cell phone case designs based on the bizarre and unattractive designs being offered.
-
Alexa Plays Pornography Instead of Kids Song AIID ↗
An Amazon Echo Dot using the Amazon Alex software started to play pornographic results when a child asked it to play a song.
-
Microsoft's TayBot Allegedly Posts Racist, Sexist, and Anti-Semitic Content to Twitter AIID ↗
Microsoft's Tay, an artificially intelligent chatbot, was released on March 23, 2016 and removed within 24 hours due to multiple racist, sexist, and anti-semitic tweets generated by the bot.
-
Google’s YouTube Kids App Presents Inappropriate Content AIID ↗
YouTube’s content filtering and recommendation algorithms exposed children to disturbing and inappropriate videos.
-
Facebook's Auto-Generated Targeting Ad Categories Contained Anti-Semitic Options AIID ↗
Facebook's automated advertising categories generated using users' declared interests contained anti-Semitic categories such as "Jew hater" and "How to burn Jews" which were listed as fields of study.
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.