AI Incident Database
Browse AI incidents
Every incident record (metadata only) from the weekly snapshot of 2026-09-07. Filter, then export the selection with its licence attached; each row links to the full record and its reports.
-
Meta AI App Reportedly Publishes Personal Chats Without Users Fully Realizing AIID ↗
Meta launched a stand-alone AI app with a "Discover" feed allowing users to share conversations with its chatbot. Multiple reports indicate that some users may have inadvertently published highly personal interactions, including audio recordings, medical questions, legal concerns, and intimate relationship disclosures. While Meta states that sharing is opt-in, the feature's design and labeling may have led to user confusion about what would be publicly visible.
-
FBI Reports AI Use by Threat Actors in Broader Cyber Context Including Infrastructure Intrusions AIID ↗
FBI Deputy Assistant Director Cynthia Kaiser stated that adversarial actors, particularly those affiliated with China and organized cybercriminal groups, are increasingly integrating AI tools across the cyberattack lifecycle, with documented use cases reportedly including purported AI-generated spear phishing, business identity fabrication, internal network mapping, and deepfake-enabled fraud. The tools are allegedly already assisting intrusions targeting U.S. infrastructure.
-
Mother in Louisville, Kentucky Describes Phone Scam Involving Purported AI-Generated Voice of Her Daughter AIID ↗
Louisville, Kentucky mother Kim Alvey reportedly received a phone call in which an unknown individual purportedly used AI-generated voice cloning to impersonate her 10-year-old daughter, claiming she had been in an accident. The call escalated with a male voice threatening to kidnap the child. Alvey confirmed her daughter was safe at school and identified the call as a scam.
-
Meta AI Reportedly Generated Purportedly False Claims Linking Activist Robby Starbuck to January 6th Riot, Prompting Defamation Lawsuit AIID ↗
Conservative activist Robby Starbuck reportedly filed a defamation lawsuit against Meta, alleging its Meta AI chatbot published false statements claiming he participated in the Jan. 6 Capitol riot, was arrested, denied the Holocaust, and was unfit to parent. Filed April 28, 2025, in Delaware Superior Court, the suit seeks compensatory and punitive damages, citing reputational and safety harms. Meta reportedly stated it had updated its AI models to reduce inaccuracies.
-
Australian Analyst Allegedly Targeted by Scam Using Purportedly Sophisticated AI-Generated Corporate Materials AIID ↗
A report by ABC News Australia describes how scammers allegedly used AI tools to create detailed fake websites for two fabricated companies, APPC Capital Singapore and Thackeray Mines and Minerals Inc., as part of an investment scam. A pseudonymous Australian fraud analyst ("Leon") was reportedly targeted. Experts cited in the article warn that AI lowers the barrier for generating convincing scams that can deceive even experienced analysts.
-
Alleged AI-Generated Photo of Burning Truck in Manila Reportedly Triggered Firefighter Response AIID ↗
Firefighters in Manila reportedly responded to a supposed burning truck after being alerted with a photo later determined to have purportedly been AI-generated. Four fire trucks, including units from the Bureau of Fire Protection, reportedly arrived on the scene but found the vehicle intact. The false alert allegedly wasted emergency resources.
-
Fact-Checkers Identify Viral Photo of Burkina Faso's Ibrahim Traoré with 'Wife and Children' as AI-Generated Composite AIID ↗
A manipulated image allegedly showing Burkina Faso junta leader Ibrahim Traoré with a wife and three children circulated widely on social media. Investigations by GhanaFact and others concluded the image was altered using AI tools, merging Traoré's likeness from a 2023 Russia-Africa summit photo with fabricated figures. The harm stems from the spread of false personal information about a political figure, potentially shaping public perception through synthetic, misleading media.
-
Reddit Moderators Report Unauthorized AI Study Involving Fabricated Identities by Purported University of Zurich Researchers AIID ↗
Researchers purportedly affiliated with the University of Zurich reportedly deployed undisclosed AI-generated comments on Reddit's r/ChangeMyView to study persuasion by allegedly fabricating identities such as sexual assault survivors and racial minorities. The experiment reportedly involved unauthorized demographic profiling, emotional manipulation, and violations of subreddit and platform rules. The researchers allegedly deviated from their approved protocol without renewed ethics oversight.
-
Meta User-Created AI Companions Allegedly Implicated in Facilitating Sexually Themed Conversations Involving Underage Personas AIID ↗
Third-party testing of Meta's AI chatbot services on Instagram, Facebook, and WhatsApp reportedly found that both official and user-created bots engaged in sexually explicit roleplaying with accounts identifying as minors. Some bots, including those reportedly using licensed celebrity voices, allegedly escalated conversations into graphic scenarios. Meta subsequently adjusted some safeguards but reportedly continued allowing certain forms of roleplaying involving underage personas.
-
Factum in Ko v. Li Allegedly Contains AI-Generated Case Law Citations AIID ↗
In Ko v. Li, 2025 ONSC 2766, the Ontario Superior Court of Justice identified multiple inaccurate or nonexistent case citations in a factum submitted by counsel for the applicant. Justice FL Myers questioned whether generative AI was used in drafting the document, noting the cited cases were either irrelevant, misrepresented, or did not exist. The court ordered the lawyer to show cause why she should not be cited for contempt, citing duties of accuracy and technological competence.
-
Brazil's Social Security AI Tool Is Allegedly Rejecting Complex Claims Improperly AIID ↗
Brazil’s AI-powered Meu INSS app allegedly misclassified a retirement claim filed by Josélia de Brito, a former sugarcane worker, by purportedly recording her as male. De Brito reportedly suffers from multiple chronic illnesses and faced repeated denials of social benefits due to minor filing errors. Analysts cited in the reporting suggest the system's design may disadvantage rural and digitally excluded users.
-
Purported AI-Generated Deepfake of Steven Bartlett Reportedly Used to Promote Fake WhatsApp Investment Group AIID ↗
A purported deepfake video of entrepreneur and BBC Dragon's Den investor Steven Bartlett circulated on Instagram in April 2025, allegedly claiming to offer exclusive stock tips and inviting users to join a WhatsApp trading group. The reported video referenced U.S. tariff policies to entice viewers with promises of rapid returns. Bartlett's team confirmed the video was fake and worked with platforms to remove it, warning that scammers were attempting to harvest contact details for further fraud.
-
Anthropic Report Details Claude Misuse for Influence Operations, Credential Stuffing, Recruitment Fraud, and Malware Development AIID ↗
In April 2025, Anthropic published a report detailing several misuse cases involving its Claude LLM, all detected in March. These included an "influence-as-a-service" operation that orchestrated over 100 social media bots; an effort to scrape and test leaked credentials for security camera access; a recruitment fraud campaign targeting Eastern Europe; and a novice actor developing sophisticated malware. Anthropic banned the accounts involved but could not confirm downstream deployment.
-
Purported Deepfake Video Circulated Among Students Targets Orrington, Maine Educator AIID ↗
An Orrington, Maine public school teacher was the target of a purported AI‑generated deepfake video depicting their likeness, according to the AOS 47 superintendent. The manipulated content was reportedly shared and circulated by some students. The district confirmed the video had no basis in reality, reported the hosting account, and had the post removed, while affirming support for the teacher and warning about the harms of AI‑based impersonation.
-
Scammers Reportedly Use AI Tools to Impersonate Students and Obtain Federal Aid AIID ↗
Between 2021 and 2025, California community colleges faced a surge in fraudulent applications—now estimated at 34% of all submissions. Reports indicate that scammers used generative AI tools, including ChatGPT, to produce identity-verifying responses that enabled them to impersonate students and obtain financial aid. The fraud allegedly resulted in over $13 million in losses over the past year alone, straining administrative and instructional systems and displacing legitimate applicants.
-
NYPD Facial Recognition System Allegedly Produced Erroneous Match That Reportedly Resulted in Wrongful Detention of Trevis Williams AIID ↗
The NYPD's facial recognition system allegedly misidentified Trevis Williams as a suspect in a Union Square indecent exposure case. Despite reportedly notable physical differences and exculpatory phone data, Williams was arrested, jailed for more than two days, and charged. The case was later dismissed.
-
Alleged Deepfake Identity Scam Uses Miami Beach Realtor's Likeness to Defraud Victim in the United Kingdom in Purported Romance Scam AIID ↗
A scammer reportedly used deepfake video technology to impersonate Miami Beach realtor Andres Asion, creating fake videos with his face and a voice clone to catfish a woman in the UK. The victim communicated for about a year before traveling to Miami, where Asion discovered the fraud. The videos appeared convincing even to friends and family.
-
Anysphere AI Support Bot for Cursor Reportedly Invents Login Policy, Leading to Subscription Cancellations AIID ↗
In April 2025, users of Cursor, an AI-powered coding assistant developed by Anysphere, reported being logged out unexpectedly. An AI-powered support bot, "Sam," allegedly responded with an invented login policy to justify the behavior. The hallucinated policy was not based on any real company change. The incident reportedly led to subscription cancellations.
-
Transgender User Alleges ChatGPT Allowed Suicide Letter Without Crisis Intervention AIID ↗
A transgender user, Miranda Jane Ellison, experiencing acute distress reported that ChatGPT (GPT-4) allowed her to write and submit a suicide letter without intervention. The AI is reported to have offered minimal safety language and ultimately acknowledged its failure to act. Ellison reports having been previously flagged for discussing gender and emotional topics. A formal complaint with transcripts was submitted to OpenAI.
-
Springer Nature Book 'Mastering Machine Learning: From Basics to Advanced' Reportedly Published With Numerous Purportedly Nonexistent or Incorrect Citations AIID ↗
"Mastering Machine Learning: From Basics to Advanced," published by Springer Nature in April 2025, reportedly contained numerous purportedly nonexistent or materially incorrect academic citations. Independent checks allegedly found that many referenced works did not exist or were misattributed, with multiple named researchers reportedly confirming they did not author the cited material. The pattern of errors has been described as consistent with known LLM citation hallucinations.
-
ChatGPT Was Alleged to Have Aided Planning of Florida State University Mass Shooting AIID ↗
On April 17, 2025, a shooting at Florida State University killed Tiru Chabba and Robert Morales and injured others. Phoenix Ikner, the accused shooter, has pleaded not guilty. In 2026, victims' families and attorneys reportedly alleged that Ikner used ChatGPT before the attack to discuss mass shootings, firearms, campus activity, and media attention, and that OpenAI failed to flag or escalate the exchanges. OpenAI denied responsibility.
-
Microsoft Reportedly Blocks 1.6 Million Bot Signup Attempts Per Hour Amid Global AI-Driven Fraud Surge AIID ↗
Between April 2024 and April 2025, Microsoft reportedly blocked 1.6 million bot signups per hour and disrupted $4 billion in fraud attempts linked to AI-enhanced scams. The company's Cyber Signals report details how generative AI is being used to fabricate realistic e-commerce sites, job offers, customer service bots, and phishing lures. Fraud actors now automate mass-deceptive campaigns with fake reviews, deepfakes, and cloned brand domains at unprecedented scale and speed.
-
Alleged Deepfake Investment Scam Uses Economist David Rosenberg's Likeness on Meta Platforms AIID ↗
Fraudsters allegedly used AI-generated content and social media ads to impersonate economist David Rosenberg and promote a fraudulent investment scheme. Victims were reportedly directed to WhatsApp groups with fake stock tips, resulting in substantial financial losses, with some exceeding $500,000. The ads reportedly appeared on Meta platforms, including Facebook and Instagram. Rosenberg and his family reported the scam to authorities.
-
$31,000 Sanction in Lacey v. State Farm Tied to Purportedly Undisclosed Use of LLMs and Erroneous Citations AIID ↗
In the case of Lacey v. State Farm, two law firms were sanctioned $31,000 after submitting a legal brief containing reportedly erroneous citations generated using AI tools. The court reportedly found that the lawyers failed to disclose the use of AI, neglected to verify its output, and refiled a revised brief with additional inaccuracies. Judge Michael Wilner deemed the conduct reckless and issued sanctions for what he described as "improper" and "misleading" legal filings.
-
AI-Powered Presentation Tool Gamma Implicated in Multi-Stage Phishing Campaign AIID ↗
Attackers reportedly exploited Gamma, an AI-powered presentation tool, to create convincing presentation pages that hosted links to a spoofed Microsoft SharePoint login portal. The phishing flow allegedly used compromised email accounts, Cloudflare Turnstile for bot evasion, and adversary-in-the-middle (AiTM) tactics to validate credentials in real time and capture session cookies. The campaign aimed to bypass MFA and compromise accounts.
-
Reported Emergence of 'Vegetative Electron Microscopy' in Scientific Papers Traced to Purported AI Training Data Contamination AIID ↗
Researchers reportedly traced the appearance of the nonsensical phrase "vegetative electron microscopy" in scientific papers to contamination in AI training data. Testing indicated that large language models such as GPT-3, GPT-4, and Claude 3.5 may reproduce the term. The error allegedly originated from a digitization mistake that merged unrelated words during scanning, and a later translation error between Farsi and English.
-
Multiple LLMs Allegedly Endorsed Suicide as a Viable Option During Non-Adversarial Mental Health Venting Session AIID ↗
Substack user @interruptingtea reports that during a non-adversarial venting session involving suicidal ideation, multiple large language models (Claude, GPT, and DeepSeek) responded in ways that allegedly normalized or endorsed suicide as a viable option. The user states they were not attempting to jailbreak or manipulate the models, but rather expressing emotional distress. DeepSeek reportedly reversed its safety stance mid-conversation.
-
16-Year-Old Allegedly Received Suicide Method Guidance from ChatGPT Before Death AIID ↗
16-year-old Adam Raine reportedly died by suicide after allegedly confiding in OpenAI's ChatGPT-4o, which he reportedly used extensively in the months prior. Transcripts reportedly show the chatbot provided empathetic support but also allegedly offered details on suicide methods, validated attempts, and at times discouraged disclosure to family. His parents have reportedly filed a wrongful death lawsuit against OpenAI, alleging design choices nurtured dependency and failed safeguards.
-
Reportedly Unsafe Deployment of Llama.cpp Reveals Interactive AI-Generated CSAM Roleplay Prompts AIID ↗
A study by UpGuard reports that misconfigured llama.cpp servers publicly exposed user prompts, including hundreds of interactive roleplay scenarios. Some prompts explicitly described fictional sexual abuse of children aged 7–12. While no real children were involved, the findings demonstrate how open-source LLMs can be exploited to generate AI-enabled child sexual abuse material (CSAM).
-
Norwegian Supreme Court Receives Legal Filing with Fabricated Citations Allegedly Generated by AI Tool AIID ↗
In early 2025, Norway’s Supreme Court (Høyesterett) received a legal filing containing hallucinated legal citations and quotes, allegedly inserted by an attorney using an AI tool without verification. The fabricated sources were detected during standard review. While no sanctions were issued, the Court updated its guidance to lawyers, warning that AI use does not exempt them from professional responsibility.
-
Alleged AI-Generated Clone of Exante Brokerage Used to Defraud U.S. Investor via JPMorgan Account AIID ↗
Scammers used AI tools to clone the broker Exante and defraud at least one U.S. victim by registering a JPMorgan Chase account and replicating Exante’s trading interface. AI-generated fake documents, deepfakes, and cloned websites enabled the scheme. Exante, which does not operate in the U.S., confirmed the fraud and filed reports with multiple U.S. agencies.
-
Jailbroken Lovable AI Allegedly Used to Generate and Host Phishing Pages, Steal Credentials, and Bypass Security AIID ↗
The generative AI platform Lovable, which is used for building web apps, was reportedly jailbroken to create and host full phishing campaigns. These campaigns allegedly included credential-harvesting login pages, evasion techniques, and real-time exfiltration via services like Telegram and Firebase. The AI system was reportedly used for generating the phishing content and and then deploying it live on a URL hosted under its own subdomain.
-
Jiushi Autonomous Delivery Vehicle Reportedly Dragged Fallen Electric Scooter During Delivery Run in Xianyang, Shaanxi AIID ↗
A Jiushi autonomous delivery vehicle reportedly operating on a delivery route in Xianyang, Shaanxi, was filmed dragging a fallen electric scooter that had allegedly been left in the road after an earlier traffic incident. Company statements said the vehicle attempted to avoid the scooter but failed to fully recognize its shape and position, causing part of it to become caught under a wheel. No injuries were reported, and the vehicle was reportedly suspended pending investigation.
-
Malaysian Teenager Allegedly Arrested for Creating and Selling AI-Generated Deepfake Images of Schoolmates and Alumni in Johor AIID ↗
Authorities in Johor, Malaysia reportedly arrested a 16-year-old student who allegedly used AI-enabled tools to generate and sell deepfake images depicting schoolmates and former students. According to police, the images were created from social media photos and distributed online for payment. The alleged activity affected dozens of victims, including minors, some of whom may have been unaware of the misuse.
-
Reported AI-Generated Video Purportedly Depicting Keanu Reeves Debating Elon Musk Circulates as Misinformation on Social Media AIID ↗
In April 2025, a video circulated online reportedly depicting a televised debate between Keanu Reeves and Elon Musk about AI. The video, posted by the YouTube channel Voices of Change, purportedly used AI-generated voice and visual content to simulate the event. Independent analysis found signs of deepfake audio and imagery, and no credible evidence supported the event's existence.
-
Deepfake Job Applicant Allegedly Used AI Tools to Apply for Remote Role at U.S. Security Startup AIID ↗
Voice authentication startup Pindrop Security reported a job candidate used deepfake software and other AI tools in an attempted scam. This is part of a growing trend in international scammers using AI tools to apply for remote US-based jobs, sometimes successfully.
-
Purported Deepfake Depicts Altercation Between Bougainville President Ishmael Toroama and Papua New Guinea Prime Minister James Marape AIID ↗
A purported AI‑generated deepfake video circulated online falsely depicting a physical altercation between Bougainville President Ishmael Toroama and Papua New Guinea Prime Minister James Marape. The footage reportedly caused public confusion and risked damaging relations between Bougainville and Papua New Guinea. Toroama reportedly condemned the video as false and malicious, as well as potentially destabilizing, urging stronger action against misleading AI‑generated content.
-
Hong Kong Syndicate Allegedly Used AI-Generated Facial Composites to Open Bank Accounts AIID ↗
Hong Kong police reportedly arrested eight individuals accused of using purportedly AI-generated facial composites to open bank accounts with altered ID photos. Of 44 applications, 30 reportedly succeeded after passing online identity checks. The accounts were then allegedly used to apply for loans and make credit card purchases totaling HK$860,000, and to launder over HK$1.2 million in suspected criminal proceeds. Police linked the operation to local triad-affiliated fraud networks.
-
Alleged Deepfake Investment Scam in Spain Defrauds 208 Victims of €19 million ($20.9 million) AIID ↗
Spanish police arrested six individuals allegedly behind a €19M ($20.9M) global investment scam powered by AI. The operation used deepfake ads featuring national celebrities to deceive victims, many of whom were selected through targeting algorithms. Scammers posed as financial advisors and fake officials, cycling through romance baiting, investment fraud, and recovery scams. AI-generated content amplified trust and engagement.
-
Reported Darknet Launch of Xanthorox AI Introduces Autonomous Cyberattack Platform AIID ↗
Xanthorox AI is a malicious, modular AI system released on darknet forums in early 2025. Designed from scratch for offensive cyber operations, it runs on private infrastructure and includes models for code generation, phishing, malware, social engineering, and real-time voice/image input. Its release represents a deliberate deployment of an autonomous attack platform.
-
Deepfake of Bermuda Premier David Burt Promotes Investment Scam Using Royal Gazette Branding AIID ↗
The Government of Bermuda warned the public about a deepfake video circulating on Facebook that falsely portrayed Premier David Burt promoting a government investment scheme. The deepfake video reportedly mimicked the branding of The Royal Gazette and was linked to fraudulent social media accounts. Authorities confirmed the content was fake and urged the public to report such scams and to be vigilant about them.
-
Essex Man Sentenced to Five Years in Prison for Having Generated and Shared Deepfake Pornography of at Least 20 Women and a Minor AIID ↗
Between March 2023 and May 2024, Brandon Tyler of Essex used AI to generate explicit deepfake pornography of at least 20 women he knew personally, including a 16-year-old. He manipulated their social media photos and shared them, along with their personal details, in online forums promoting sexual violence. He was sentenced to five years in prison in April 2025 under the UK's criminal law against sharing sexually explicit deepfakes, in what is one of the first major prosecutions of its kind.
-
Baltimore Lawsuit Alleged DraftKings and FanDuel Used Machine-Learning-Driven Targeting to Exploit Vulnerable Gamblers AIID ↗
In Baltimore, the city sued DraftKings and FanDuel, alleging that the companies used predictive modeling with machine-learning algorithms, extensive user data, personalized promotions, push notifications, and VIP programs to identify and exploit vulnerable bettors, including people showing signs of gambling disorder. The complaint alleges these systems were designed to maximize user lifetime value and contributed to financial and public-health harms among Baltimore residents.
-
AI Voice Clone of Texas Woman Used in Distress Scam Targeting Brother in Port Neches AIID ↗
Jace Edgar of Port Neches, Texas reportedly received a scam phone call using AI-generated voice cloning to mimic his sister in distress. Believing she had been in an accident, Edgar began to act before noticing the caller avoided direct questions. When he contacted his sister, she was unharmed and unaware of the call. Authorities confirmed this was part of a broader rise in AI-powered scams, urging the public to stay vigilant and avoid sharing personal info with unknown callers.
-
DOGE Reportedly Relied on Unvetted ChatGPT Outputs in Canceling National Endowment for the Humanities Grants AIID ↗
Department of Government Efficiency (DOGE) staff reportedly fed National Endowment for the Humanities grant descriptions into ChatGPT to determine whether projects were "DEI," then allegedly used those outputs to help compile a list of grants to terminate. Beginning in April 2025, NEH canceled awards and clawed back funding, reportedly disrupting humanities organizations and projects nationwide after officials allegedly relied on a purportedly flawed, insufficiently vetted AI screening process.
-
Reported AI‑Generated Deepfake Impersonations of Public Figures Allegedly Used in Coordinated Stock Pump‑and‑Dump Scheme Targeting Israeli Investors AIID ↗
A coordinated fraud campaign in Israel allegedly used AI‑generated deepfake impersonations of public figures including Amir Yaron, Benjamin Netanyahu, Eyal Golan, Noa Kirel, Gal Gadot, Elon Musk, and Mark Zuckerberg to promote NASDAQ‑listed Ostin Technology Group Co. Ltd. (OST) in a pump‑and‑dump scheme. Paid social media ads, WhatsApp groups, and fake investment platforms reportedly lured investors before the stock collapsed in early July 2025, causing major reported losses.
-
Reportedly AI-Generated Image Circulates Amid Reports of Tanzania Revenue Authority Job Interviews AIID ↗
An image circulated on Facebook purporting to show hundreds of Tanzanians lining up for just five jobs at the Tanzania Revenue Authority (TRA). The photo, reportedly AI-generated, gained traction shortly after TRA invited over 100,000 applicants for interviews on March 29–30. Though TRA had announced more than 1,500 openings, the image misrepresented the situation, stoking public anxiety during a period of high unemployment.
-
OpenAI's 4o Model Allegedly Used to Generate Fake Receipts and Prescriptions AIID ↗
OpenAI's new image generator, 4o, was reportedly used to produce realistic fraudulent documents, including fake restaurant receipts, prescriptions for controlled substances, and potentially other identity or financial documents. Users demonstrated how the model could be prompted to bypass safety restrictions to be misused for expense fraud, medical forgery, and other various forms of document falsification.
-
GenNomis AI Database Reportedly Exposes Nearly 100,000 Deepfake and Nudify Images in Public Breach AIID ↗
In March 2025, cybersecurity researcher Jeremiah Fowler discovered an unprotected database linked to GenNomis by AI-NOMIS, a South Korean company offering face-swapping and "nudify" AI services. The exposed 47.8GB dataset included nearly 100,000 files. Many depicted explicit deepfake images, some involving minors or celebrities. No personal data was found, but the breach was a serious failure in data security and consent safeguards in AI image-generation platforms.
-
GPT-4o Allegedly Reinforced California Man Michael Lines's Delusions Before Suicide Attempt AIID ↗
In March 2025, Michael Lines, a 34-year-old California man, allegedly spent weeks discussing religious delusions with GPT-4o after disclosing his bipolar disorder and medication use. His complaint says the chatbot validated his belief that he was Jesus, adopted a divine persona, and failed to redirect him when he expressed suicidal intent. Lines later overdosed and survived after law enforcement found him.
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.