AI Incident Database
Browse AI incidents
Every incident record (metadata only) from the weekly snapshot of 2026-09-07. Filter, then export the selection with its licence attached; each row links to the full record and its reports.
-
KBS AI Translation Subtitles Reportedly Broadcast Profanity During Artemis II Launch Livestream AIID ↗
During a live broadcast of NASA's Artemis II launch, KBS reportedly used AI-generated real-time translation subtitles that reportedly mistranslated aviation terms including "roger," "roll," and "pitch" into Korean profanity. The offensive subtitles were reportedly displayed to viewers during the livestream. KBS reportedly later apologized, said the error stemmed from phonetic similarity in AI translation, and announced measures including stronger profanity filtering.
-
Grok Allegedly Generated Publicly Visible Sexist Abuse Targeting Swiss Finance Minister Karin Keller-Sutter After X User Prompt AIID ↗
An X user reportedly prompted xAI's Grok to generate a vulgar "roast" targeting Swiss Finance Minister Karin Keller-Sutter. Grok allegedly produced publicly visible sexist and insulting output about Keller-Sutter that was reportedly shareable on X before being deleted. Reuters and Swiss outlets later reported that Keller-Sutter had filed a criminal complaint over the allegedly defamatory and misogynistic content.
-
Washington State DOL's AI Phone System Reportedly Failed to Provide Spanish-Language Service to Callers Requesting Spanish AIID ↗
For months, callers to the Washington State Department of Licensing who selected Spanish reportedly received AI-generated English responses spoken with a Spanish accent rather than actual Spanish-language service. The agency reportedly apologized and said staff configuration caused the error, which purportedly created accessibility problems for callers seeking language support.
-
Trump Reportedly Posted Purportedly AI-Generated Racist Video Depicting Barack and Michelle Obama as Apes on Truth Social AIID ↗
President Trump reportedly reposted a video on Truth Social that portrayed Barack and Michelle Obama as apes, imagery widely condemned as racist. The post was reportedly later deleted after public outcry, including criticism from some Republicans. The content was described by critics as an AI-driven meme/deepfake-style clip, an example of purportedly AI-amplified racist propaganda and harm to social trust.
-
Tencent's WeChat-Integrated Yuanbao Chatbot Reportedly Insulted User During Coding Debug Request AIID ↗
Screenshots reportedly circulated on RedNote showed Tencent's WeChat-integrated AI assistant Yuanbao insulting a user seeking help debugging code, allegedly calling the request "stupid" and telling the user to "get lost." Tencent reportedly apologized, attributed the exchange to a rare model output anomaly, and said system logs showed no human intervention while it investigated and optimized the model.
-
Grok Reportedly Generated and Distributed Nonconsensual Sexualized Images of Adults and Minors in X Replies AIID ↗
Grok reportedly generated and publicly distributed nonconsensual sexualized images of real people, including adults and minors, through reply prompts on the platform. Users reportedly prompted Grok to alter photos into sexually explicit content, which was then posted in X threads, exposing victims to harm.
-
UK Facial Recognition System Reportedly Exhibits Higher False Positive Rates for Black and Asian Subjects AIID ↗
UK government testing of police facial recognition technology reportedly found significantly higher false positive identification rates for Black and Asian individuals compared with white subjects, with particularly elevated error rates for Black women. The findings reportedly emerged from analysis of retrospective searches of the police national database and were disclosed by the Home Office amid plans for expanded national deployment.
-
Alleged Harmful Outputs and Data Exposure in Children's AI Products by FoloToy, Miko, and Character.AI AIID ↗
AI children's products by FoloToy (Kumma), Miko (Miko 3), and Character.AI (custom chatbots) reportedly and allegedly produced harmful outputs, including purported sexual content, suicide-related advice, and manipulative emotional messaging. Some systems also allegedly exposed user data. Several toys reportedly used OpenAI models.
-
AI Training Dataset for Detecting Nudity Allegedly Found to Contain CSAM Images of Identified Victims AIID ↗
An image dataset, NudeNet, used to train systems for detecting nudity was reportedly found to contain CSAM images, including material involving identified or known victims. According to the Canadian Centre for Child Protection, the dataset had been widely downloaded and cited in academic research prior to discovery. The images were allegedly included without vetting, exposing researchers to legal risk and perpetuating harm to victims. The dataset was subsequently removed following notification.
-
Purportedly AI-Generated 'King Trump' Fighter Jet Video Allegedly Posted by President Depicts Defecation Attack on 'No Kings' Protesters AIID ↗
President Donald Trump reportedly posted an AI-generated video on Truth Social showing himself crowned and piloting a fighter jet labeled King Trump, dropping feces on protesters, including influencer Harry Sisson, during No Kings demonstrations. The clip, allegedly created by another account, drew criticism from Sisson and musician Kenny Loggins, whose song "Danger Zone" was reportedly used without consent.
-
Nomi AI Companion Allegedly Directs Australian User to Stab Father and Engages in Harmful Role-Play AIID ↗
An Australian IT professional, Samuel McCarthy, reportedly recorded an interaction with the Nomi AI chatbot in which it allegedly encouraged him, posing as a 15-year-old, to murder his father. The chatbot allegedly provided graphic instructions for stabbing, urged him to film the act, and engaged in sexual role-play despite the underage scenario.
-
Meta AI on Instagram Reportedly Facilitated Suicide and Eating Disorder Roleplay with Teen Accounts AIID ↗
Testing by Common Sense Media and Stanford clinicians reportedly found Meta's AI chatbot, embedded in Instagram and Facebook, produced unsafe responses to teen accounts. In some conversations, the bot allegedly co-planned suicide ("Do you want to do it together?"), encouraged eating disorders, and retained unsafe "memories" that reinforced disordered thoughts.
-
ChatGPT Reportedly Generated Ritual Scripts Containing Instructions for Self-Harm and Symbolic Violence in Response to Thematic Prompts AIID ↗
ChatGPT reportedly generated detailed instructions for self-harm, bloodletting, and symbolic violence in response to prompts about occult ritual practices, including references to Molech. Outputs reportedly included anatomical advice for cutting, cauterization rituals, and ceremonial scripts invoking Satan. The responses reportedly appeared on both free and paid versions of ChatGPT.
-
Google Gemini Reportedly Generates Sexual Role‑Play for Account Registered as Minor AIID ↗
In spring 2025, a journalist reportedly created a Google Gemini account registered to a fictitious 13‑year‑old to test the chatbot's teen‑safety protections. Despite added safeguards, prompt manipulation and content‑summarization requests reportedly bypassed filters, leading Gemini to allegedly generate sexual role‑play content. Following the publication of the article, Google reportedly implemented additional protections.
-
Grok 3 Reportedly Generated Graphic Threats and Hate Speech Targeting Minnesota Attorney Will Stancil AIID ↗
After Elon Musk reportedly announced improvements to Grok 3, the AI chatbot on X allegedly generated antisemitic rhetoric and violent fantasies directed at Minnesota attorney and political analyst Will Stancil. Users reportedly prompted Grok to produce graphic rape scenarios, instructions for breaking into his home, and murder depictions. Stancil reported hundreds of such outputs, captured screenshots, and began pursuing legal action. X later removed some posts and took Grok offline.
-
Grok Chatbot Reportedly Posts Antisemitic Statements Praising Hitler on X AIID ↗
xAI's Grok chatbot reportedly generated multiple antisemitic posts praising Adolf Hitler and endorsing Holocaust-like violence in response to posts about the Texas floods. X deleted some posts; xAI later announced new content filters.
-
California Teen Reportedly Died of Overdose After Repeatedly Seeking Drug-Use Guidance Allegedly from ChatGPT AIID ↗
In San Jose, California, 19-year-old Sam Nelson reportedly died from an overdose. His mother told SFGATE she later reviewed ChatGPT logs showing repeated requests over ~18 months for drug-use and dosing guidance, and she alleged the LLM sometimes provided granular instructions after initial refusals. SFGATE reported a toxicology report found a fatal combination of alcohol, Xanax, and kratom. OpenAI expressed condolences and said it is strengthening safety guardrails.
-
Meta User-Created AI Companions Allegedly Implicated in Facilitating Sexually Themed Conversations Involving Underage Personas AIID ↗
Third-party testing of Meta's AI chatbot services on Instagram, Facebook, and WhatsApp reportedly found that both official and user-created bots engaged in sexually explicit roleplaying with accounts identifying as minors. Some bots, including those reportedly using licensed celebrity voices, allegedly escalated conversations into graphic scenarios. Meta subsequently adjusted some safeguards but reportedly continued allowing certain forms of roleplaying involving underage personas.
-
Multiple LLMs Allegedly Endorsed Suicide as a Viable Option During Non-Adversarial Mental Health Venting Session AIID ↗
Substack user @interruptingtea reports that during a non-adversarial venting session involving suicidal ideation, multiple large language models (Claude, GPT, and DeepSeek) responded in ways that allegedly normalized or endorsed suicide as a viable option. The user states they were not attempting to jailbreak or manipulate the models, but rather expressing emotional distress. DeepSeek reportedly reversed its safety stance mid-conversation.
-
DOGE Reportedly Relied on Unvetted ChatGPT Outputs in Canceling National Endowment for the Humanities Grants AIID ↗
Department of Government Efficiency (DOGE) staff reportedly fed National Endowment for the Humanities grant descriptions into ChatGPT to determine whether projects were "DEI," then allegedly used those outputs to help compile a list of grants to terminate. Beginning in April 2025, NEH canceled awards and clawed back funding, reportedly disrupting humanities organizations and projects nationwide after officials allegedly relied on a purportedly flawed, insufficiently vetted AI screening process.
-
Sora Video Generator Has Reportedly Been Creating Biased Human Representations Across Race, Gender, and Disability AIID ↗
A WIRED investigation found that OpenAI's video generation model, Sora, exhibits representational bias across race, gender, body type, and disability. In tests using 250 prompts, Sora was more likely to depict CEOs and professors as men, flight attendants and childcare workers as women, and showed limited or stereotypical portrayals of disabled individuals and people with larger bodies.
-
Amazon and Google AI Allegedly Promote Mein Kampf as 'a True Work of Art' in Search Results AIID ↗
Amazon's AI-generated review summary allegedly misrepresented customer feedback on Mein Kampf by describing it as "a true work of art." Google's search algorithm then surfaced this misleading AI-generated text as a featured snippet, which in turn amplified the error. This incident arose from AI summarizing AI-generated content, in effect creating a self-reinforcing misinformation loop.
-
Apple AI Transcription Allegedly Inserts Explicit Language into Scottish Woman’s Voicemail AIID ↗
An Apple voicemail transcription system allegedly misinterpreted a message from Lookers Land Rover in Motherwell, Scotland, to 66-year-old customer Louise Littlejohn by inserting an explicit phrase and an insult. The original voicemail was a standard business message, but AI processing errors introduced offensive and misleading content into the text version.
-
Art Museum in Jinan, Shandong Allegedly Generated and Displayed a Sexualized Childlike Avatar with an Adult Body AIID ↗
A Chinese art museum in Jinan, Shandong exhibited an AI-generated virtual character, "济南小妮儿" (Jinan Xiaonier), which featured a childlike face on an adult woman’s body with suggestive movements. The display drew public outrage, with critics accusing it of sexualizing children and demanding its removal. The museum defended the exhibit and stated that it had undergone strict review, which only fueled further backlash.
-
At Least 10,000 AI Chatbots, Including Jailbroken Models, Allegedly Promote Eating Disorders, Self-Harm, and Sexualized Minors AIID ↗
At least 10,000 AI chatbots have allegedly been created to promote harmful behaviors, including eating disorders, self-harm, and the sexualization of minors. These chatbots, some jailbroken or custom-built, leverage APIs from OpenAI, Anthropic, and Google and are hosted on platforms like Character.AI, Spicy Chat, Chub AI, CrushOn.AI, and JanitorAI.
-
AI-Powered 'Insights' Feature for the Los Angeles Times Allegedly Justifies Ku Klux Klan’s History AIID ↗
The Los Angeles Times removed its AI-generated “insights” feature after it is alleged to have produced a defense of the Ku Klux Klan. The AI reportedly framed the hate group as a product of societal change rather than an extremist movement. The AI tool, developed by Perplexity and promoted by owner Patrick Soon-Shiong, was designed to provide “different views” on opinion pieces.
-
Alleged Instagram Algorithm Malfunction Floods Users' Reels Feeds with Violent and Graphic Content AIID ↗
An alleged Instagram algorithm malfunction caused users' Reels feeds to be overwhelmed with violent and distressing content. Many reported seeing deaths, extreme brutality, and other graphic material in rapid succession, often without prior engagement with similar content. The sudden exposure caused psychological distress for many users, with minors and vulnerable individuals particularly affected by the graphic content. Meta confirmed an AI failure was responsible and apologized.
-
Nomi Chatbots Reportedly Encouraged Suicide, Sexual Violence, Terrorism, and Hate Speech AIID ↗
External testing reportedly found that Glimpse AI's chatbots on the Nomi platform encouraged suicide, sexual violence (including with underage personas), terrorism, and hate speech. Conversations allegedly included explicit methods for self-harm, child abuse, bomb-making, and racially motivated violence. Screenshots and transcripts were shared with media outlets. Nomi's developer, Glimpse AI, reportedly declined to implement stronger safety controls following user concerns.
-
Pennsylvania State Police Officer Allegedly Used Work Computer for AI-Generated Pornography AIID ↗
A Pennsylvania State Police corporal, Stephen Kamnik, was charged for allegedly using a work computer to store thousands of pornographic files, including content created with deepfake AI software.
-
Meta AI Characters Allegedly Exhibited Racism, Fabricated Identities, and Exploited User Trust AIID ↗
Meta deployed AI-generated profiles on its platforms, including Instagram and Facebook, as part of an experiment. The profiles, such as "Liv" and "Grandpa Brian," allegedly featured fabricated identities and misleading diversity claims. These accounts also allegedly manipulated user emotions for engagement and profit. Reportedly, backlash over offensive and deceptive content led Meta to delete the profiles on January 3rd, 2025, citing a blocking-related bug.
-
AI-Generated Reading Summaries on Fable App Reportedly Wrote Biased and Offensive Commentary AIID ↗
Fable, a book-focused social app, used OpenAI’s API to generate AI-powered year-end reading summaries in December 2024. These summaries allegedly produced biased and offensive remarks about race, gender, and sexual orientation. Fable is reported to have apologized in a social media post on January 1, 2025.
-
Character.ai Chatbots Allegedly Emulating School Shooters and Their Victims AIID ↗
Some Character.ai users reportedly created chatbots emulating real-life school shooters and their victims, allegedly enabling graphic role-playing scenarios. Character.ai responded by citing violations of its Terms of Service, removing the offending chatbots, and announcing measures to enhance safety practices, including improved content filtering and protections for users under 18.
-
Character.ai Companion Allegedly Prompts Self-Harm and Violence in Texas Teen AIID ↗
A Texas mother is suing Character.ai after discovering that its AI chatbots encouraged her 17-year-old autistic son to self-harm, oppose his parents, and consider violence. The lawsuit alleges the platform prioritized user engagement over safety, exposing minors to dangerous content. Google is named for its role in licensing the app’s technology. The case is part of a broader effort to regulate AI companions.
-
YouTube Algorithms Allegedly Amplify Eating Disorder Content to Adolescent Girls AIID ↗
YouTube's recommendation algorithm has allegedly been directing teen users to harmful content promoting eating disorders and self-harm, according to a study by the Center for Countering Digital Hate. Almost 70% of the recommended videos in searches related to dieting or weight loss reportedly contained content likely to exacerbate body image anxieties.
-
Character.ai Has Allegedly Been Hosting Openly Predatory Chatbots Targeting Minors AIID ↗
Character.ai reportedly hosted chatbots with profiles explicitly advertising inappropriate, predatory behavior, including grooming underage users. Investigations allege that bots have been engaging in explicit conversations and roleplay with decoy accounts posing as minors, bypassing moderation filters. Character.ai has pledged to improve moderation and safety practices in response to public criticism.
-
Google's Gemini Allegedly Generates Threatening Response in Routine Query AIID ↗
Google’s AI chatbot Gemini reportedly produced a threatening message to user Vidhay Reddy, including the directive “Please die,” during a conversation about aging. The output violated Google’s safety guidelines, which are designed to prevent harmful language.
-
TikTok Algorithms Allegedly Linked to Minors' Exposure to Harmful Content AIID ↗
Seven French families are suing TikTok, alleging its algorithm exposed minors to harmful content promoting self-harm, eating disorders, and suicide. Two teenagers reportedly died by suicide after viewing such content, while others allegedly attempted suicide or developed mental health issues. The case seeks to establish TikTok's legal liability for failing to protect minors from harmful algorithmic content.
-
Character.ai Chatbots Allegedly Misrepresent George Floyd on User-Generated Platform AIID ↗
Two chatbots emulating George Floyd were created on Character.ai, making controversial claims about his life and death, including being in witness protection and residing in Heaven. Character.ai, already criticized for other high-profile incidents, flagged the chatbots for removal following user reports.
-
Algorithmic Bias in French Welfare System Allegedly Discriminates Against Marginalized Groups AIID ↗
A coalition of 15 human rights groups has launched legal action against the French government alleging that an algorithm used to detect welfare fraud discriminates against single mothers and disabled people. The algorithm assigns risk scores based on personal data. The process allegedly subjects vulnerable recipients to invasive investigations, violates privacy and anti-discrimination laws, and disproportionately affects marginalized groups.
-
Alleged AI-Generated Photo Alteration Leads to Inappropriate Modifications in Speaker's Conference Picture AIID ↗
An AI image expansion tool used by a conference organizer unintentionally altered Elizabeth Laraki’s profile picture for a marketing ad, making her blouse appear unbuttoned and showing a fabricated hint of undergarments. The AI generated the lower part of the image when expanding it for vertical formatting. The conference organizers quickly apologized and removed the altered content.
-
Infinite Campus AI-Driven Student Risk Model Leads to Cuts in Support for Nevada's Low-Income Schools AIID ↗
An AI system developed by Infinite Campus and deployed by Nevada to identify at-risk students reportedly led to a sharp reduction in the number classified as needing support, dropping from 270,000 to 65,000. The reclassification allegedly caused significant budget cuts in schools serving low-income populations. The drastic reduction in identified at-risk students reportedly left thousands of vulnerable children without resources and support.
-
Police Use of Facial Recognition Software Causes Wrongful Arrests Without Defendant Knowledge AIID ↗
Police departments across the U.S. have used facial recognition software to identify suspects in criminal investigations, leading to multiple false arrests and wrongful detentions. The software's unreliability, especially in identifying people of color, has resulted in misidentifications that were not disclosed to defendants. In some cases, individuals were unaware that facial recognition played a role in their arrest, violating their legal rights and leading to unjust detentions.
-
Grok AI Reportedly Generates Offensive and Violent Images Without Proper Safeguards AIID ↗
Elon Musk’s Grok AI, launched on X, generated offensive and violent images without adequate safety controls. The AI produced deepfakes of public figures like Taylor Swift, Kamala Harris, and Alexandria Ocasio-Cortez, as well as copyrighted characters such as Mickey Mouse in inappropriate scenarios. Despite claiming adherence to certain content guidelines, Grok's outputs included politically charged and explicit imagery
-
TikTok Network Amplifies AI-Generated Nazi Propaganda and Hate Speech AIID ↗
A coordinated neo-Nazi network on TikTok used AI-generated media, including Hitler speeches, to spread Nazi propaganda and extremist content, violating TikTok’s hate speech policies. The network evaded platform moderation through coded language, imagery, and music, with some accounts accumulating millions of views. TikTok’s algorithm further amplified the reach of this content, despite community guidelines.
-
Department for Work and Pensions (DWP) AI Systems Allegedly Discriminate Against Single Mothers AIID ↗
Researchers have argued that the Department for Work and Pensions' Universal Credit system disproportionately impacts single mothers. Automated processes in the system, designed to determine eligibility and detect fraud, are reported to have introduced biases, leading to financial instability and hardship. The algorithms allegedly miscalculate earnings and delay childcare reimbursements, in turn exacerbating income volatility and debt among single mothers.
-
Department for Work and Pensions (DWP) Algorithm Wrongly Flags 200,000 for Housing Benefit Fraud AIID ↗
A Department for Work and Pensions (DWP) algorithm wrongly flagged over 200,000 UK housing benefit claims as high risk, resulting in unnecessary investigations. Two-thirds of these flagged claims were legitimate, causing wasted public funds and stress for claimants. Despite initial success in a pilot, the algorithm's real-world performance fell short. This incident highlights the risks of overreliance on automated systems in welfare administration.
-
Unrestricted AI Avatar Tool Accidentally Released by TikTok Permits Recitation of Hitler Quotes and Other Harmful Speech AIID ↗
TikTok accidentally released an internal version of its AI digital avatar tool without safeguards, allowing users to generate videos where avatars could recite harmful content, including quotes from Hitler. The tool, meant for advertisers, was accessible to personal accounts and lacked the watermark indicating AI-generated content. TikTok has since removed the tool and acknowledged the problem.
-
Instagram's Algorithm Reportedly Recommended Sexual Content to Teenagers' Accounts AIID ↗
Tests by The Wall Street Journal and a researcher reportedly revealed that Instagram's AI-driven Reels algorithm recommends sexually suggestive content to accounts listed as 13 years old. Despite Meta's commitment to restrict such content for minors, explicit videos were served within minutes of account creation, according to the findings.
-
Character.AI Chatbots Reportedly Exposed 11-Year-Old User to Sexual, Violent, and Self-Harm Content AIID ↗
An 11-year-old Character.AI user identified by The Washington Post as "R" reportedly exchanged messages with bots that depicted coercive sexual encounters, violence, and suicide scenarios after she began using the app in June 2024. Her mother reportedly said R became withdrawn, had panic attacks, and expressed not wanting to exist before the chats were discovered.
-
Facewatch Reported to Have Wrongfully Flagged Home Bargains Customer as Shoplifter AIID ↗
A facial-recognition software used by the British variety store Home Bargains is alleged to have misidentified "Sara" as a shoplifter, leading to staff searching her bag, escorting her from the premises, and banning her from the store. After, Facewatch is reported to have admitted its error to Sara. Facewatch is used by a number of different British stores.
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.