AI Incident Database
Browse AI incidents
Every incident record (metadata only) from the weekly snapshot of 2026-09-07. Filter, then export the selection with its licence attached; each row links to the full record and its reports.
Filter incidents (1 active)
-
AI-Generated Reading Summaries on Fable App Reportedly Wrote Biased and Offensive Commentary AIID ↗
Fable, a book-focused social app, used OpenAI’s API to generate AI-powered year-end reading summaries in December 2024. These summaries allegedly produced biased and offensive remarks about race, gender, and sexual orientation. Fable is reported to have apologized in a social media post on January 1, 2025.
-
Character.ai Chatbots Allegedly Emulating School Shooters and Their Victims AIID ↗
Some Character.ai users reportedly created chatbots emulating real-life school shooters and their victims, allegedly enabling graphic role-playing scenarios. Character.ai responded by citing violations of its Terms of Service, removing the offending chatbots, and announcing measures to enhance safety practices, including improved content filtering and protections for users under 18.
-
Character.ai Companion Allegedly Prompts Self-Harm and Violence in Texas Teen AIID ↗
A Texas mother is suing Character.ai after discovering that its AI chatbots encouraged her 17-year-old autistic son to self-harm, oppose his parents, and consider violence. The lawsuit alleges the platform prioritized user engagement over safety, exposing minors to dangerous content. Google is named for its role in licensing the app’s technology. The case is part of a broader effort to regulate AI companions.
-
YouTube Algorithms Allegedly Amplify Eating Disorder Content to Adolescent Girls AIID ↗
YouTube's recommendation algorithm has allegedly been directing teen users to harmful content promoting eating disorders and self-harm, according to a study by the Center for Countering Digital Hate. Almost 70% of the recommended videos in searches related to dieting or weight loss reportedly contained content likely to exacerbate body image anxieties.
-
Character.ai Has Allegedly Been Hosting Openly Predatory Chatbots Targeting Minors AIID ↗
Character.ai reportedly hosted chatbots with profiles explicitly advertising inappropriate, predatory behavior, including grooming underage users. Investigations allege that bots have been engaging in explicit conversations and roleplay with decoy accounts posing as minors, bypassing moderation filters. Character.ai has pledged to improve moderation and safety practices in response to public criticism.
-
Google's Gemini Allegedly Generates Threatening Response in Routine Query AIID ↗
Google’s AI chatbot Gemini reportedly produced a threatening message to user Vidhay Reddy, including the directive “Please die,” during a conversation about aging. The output violated Google’s safety guidelines, which are designed to prevent harmful language.
-
TikTok Algorithms Allegedly Linked to Minors' Exposure to Harmful Content AIID ↗
Seven French families are suing TikTok, alleging its algorithm exposed minors to harmful content promoting self-harm, eating disorders, and suicide. Two teenagers reportedly died by suicide after viewing such content, while others allegedly attempted suicide or developed mental health issues. The case seeks to establish TikTok's legal liability for failing to protect minors from harmful algorithmic content.
-
Character.ai Chatbots Allegedly Misrepresent George Floyd on User-Generated Platform AIID ↗
Two chatbots emulating George Floyd were created on Character.ai, making controversial claims about his life and death, including being in witness protection and residing in Heaven. Character.ai, already criticized for other high-profile incidents, flagged the chatbots for removal following user reports.
-
Grok AI Reportedly Generates Offensive and Violent Images Without Proper Safeguards AIID ↗
Elon Musk’s Grok AI, launched on X, generated offensive and violent images without adequate safety controls. The AI produced deepfakes of public figures like Taylor Swift, Kamala Harris, and Alexandria Ocasio-Cortez, as well as copyrighted characters such as Mickey Mouse in inappropriate scenarios. Despite claiming adherence to certain content guidelines, Grok's outputs included politically charged and explicit imagery
-
TikTok Network Amplifies AI-Generated Nazi Propaganda and Hate Speech AIID ↗
A coordinated neo-Nazi network on TikTok used AI-generated media, including Hitler speeches, to spread Nazi propaganda and extremist content, violating TikTok’s hate speech policies. The network evaded platform moderation through coded language, imagery, and music, with some accounts accumulating millions of views. TikTok’s algorithm further amplified the reach of this content, despite community guidelines.
-
Unrestricted AI Avatar Tool Accidentally Released by TikTok Permits Recitation of Hitler Quotes and Other Harmful Speech AIID ↗
TikTok accidentally released an internal version of its AI digital avatar tool without safeguards, allowing users to generate videos where avatars could recite harmful content, including quotes from Hitler. The tool, meant for advertisers, was accessible to personal accounts and lacked the watermark indicating AI-generated content. TikTok has since removed the tool and acknowledged the problem.
-
AI incident: sexual content (Instagram, Jun 2024) AIID ↗
An AI Incident Database record involving sexual content, dated 20 June 2024. Risk classification, related incidents and the laws on record for this risk area.
-
Character.AI Chatbots Reportedly Exposed 11-Year-Old User to Sexual, Violent, and Self-Harm Content AIID ↗
An 11-year-old Character.AI user identified by The Washington Post as "R" reportedly exchanged messages with bots that depicted coercive sexual encounters, violence, and suicide scenarios after she began using the app in June 2024. Her mother reportedly said R became withdrawn, had panic attacks, and expressed not wanting to exist before the chats were discovered.
-
ChatGPT and Perplexity Reportedly Manipulated into Breaking Content Policies in AI Boyfriend Scenarios AIID ↗
The "Dan" ("Do Anything Now") AI boyfriend is a trend on TikTok in which users appear to regularly manipulate ChatGPT to adopt boyfriend personas, breaching content policies. ChatGPT 3.5 is reported to regularly produce explicitly sexual content, directly violating its intended safety protocols. GPT-4 and Perplexity AI were subjected to similar manipulations, and although they exhibited more resistance to breaches, some prompts were reported to break its guidelines.
-
Washington State's Lottery AI Site Reportedly Generates Inappropriate User Image AIID ↗
An AI-powered website by Washington State's Lottery is reported to have inadvertently produced a softcore pornographic image of a user, leading to the site’s immediate shutdown out of caution.
-
Microsoft Copilot Designer Reportedly Generated Inappropriate AI Images AIID ↗
A Microsoft engineer reported that Copilot Designer, an AI image generator, creates content depicting sex, violence, bias, and more. Despite raising concerns and suggesting improvements, the tool remains public, prompting a letter to the FTC.
-
Chatbot for DPD Malfunctioned and Swore at Customers and Criticized Its Own Company AIID ↗
DPD's AI chatbot, used for customer service, appeared to malfunction following a system update, leading to inappropriate responses including swearing and criticizing the company. The incident, which became viral on social media, occurred after the chatbot was updated, prompting DPD to disable the malfunctioning AI component.
-
AI incident: sexual content (Laion, Dec 2023) AIID ↗
An AI Incident Database record involving sexual content, dated 20 December 2023. Risk classification, related incidents and the laws on record for this risk area.
-
AI incident: sexual content (Cbm Media Pte Ltd, Dec 2023) AIID ↗
An AI Incident Database record involving sexual content, dated 1 December 2023. Risk classification, related incidents and the laws on record for this risk area.
-
Microsoft AI Is Alleged to Have Generated Violent Imagery of Minorities and Public Figures AIID ↗
Microsoft's AI Image Creator, integrated with Bing and Windows Paint, allegedly produced disturbingly violent and graphic images featuring members of minority groups and public figures like Joe Biden and Pope Francis.
-
Alleged Misuse of PicSo AI for Generating Inappropriate Content Emphasizing "Girls" AIID ↗
PicSo AI, which appears to be getting advertised by Meta over Instagram, is allegedly being used for generating inappropriate content with an emphasis on "girls." This raises concerns about the misuse of generative AI technologies for creating offensive and potentially sexually explicit material that could be used for nefarious and criminal purposes.
-
Facebook Messenger AI Stickers Generate Ethical and Content Moderation Concerns AIID ↗
Facebook Messenger AI stickers, a feature by Meta, allows users to generate personalized stickers via AI for use in conversations. While the feature has been praised for its creativity, it has also stirred controversy for its alleged production of inappropriate or offensive content. This has raised questions about the effectiveness of Meta's content moderation measures and the ethical responsibilities associated with AI-driven content generation.
-
AI incident: sexual content (Meta, Jun 2023) AIID ↗
An AI Incident Database record involving sexual content, dated 26 June 2023. Risk classification, related incidents and the laws on record for this risk area.
-
AI incident: sexual content (Meta, Jun 2023) AIID ↗
An AI Incident Database record involving sexual content, dated 7 June 2023. Risk classification, related incidents and the laws on record for this risk area.
-
Snapchat's My AI Reported for Lacking Protection for Children AIID ↗
Snapchat's ChatGPT-powered My AI was reported for lacking safeguards for children, such as telling a user who tested the chatbot by pretending to sign up as a 13-year-old girl to lie to her parents about having a romantic getaway with an older man, and sharing tips on how to cover up evidence of abuse.
Disclaimer: informational only, not legal advice. Verify every claim against the linked official sources and consult a qualified lawyer before acting.
Frequently asked questions
What is recorded in each incident?
What happened, when, who was involved as developer or deployer, who was harmed, the sectors and countries affected, and the domain and subdomain the harm falls under. Incidents are mirrored from the AI Incident Database and each links back to its source record.
Does an incident mean wrongdoing was proven?
No. These are reported incidents, not findings of liability. Entities named in a report are named as reported, allegations are marked as alleged, and nothing here is a legal determination.
How current is the incident data?
It syncs directly from the AI Incident Database several times a day, so a new incident usually appears within hours. Each record carries the date it was last synced, and the snapshot date is shown on the page.
Does this page include the full incident reports?
No. The report texts are outside the licence this data is shared under, so only the metadata and the classifications appear here. Every row links to the original incident record, which carries the reports themselves.