AI Incident Database
Browse AI incidents
Every incident record (metadata only) from the weekly snapshot of 2026-09-07. Filter, then export the selection with its licence attached; each row links to the full record and its reports.
-
22 Students at Richmond-Burton Community High School in Illinois Targeted by Deepfake Nudes AIID ↗
22 students at Richmond-Burton Community High School in Illinois were targeted in the creation of deepfake nudes. One of the students, Stevie Hyder, was targeted by classmates who used deepfake technology to alter her April 2023 prom picture into nude pictures, which were then circulated on social media. Two unnamed minors were arrested in late April 2024.
-
Cartels Reportedly Using AI to Expand Operations into Financial Fraud and Human Trafficking AIID ↗
The Jalisco New Generation Cartel is reportedly using AI to expand its financial fraud and human trafficking operations, coercing individuals into illegal activities under the guise of legitimate jobs. INTERPOL warns that this integration of AI into criminal enterprises is a growing trend among cartels across Europe, Asia, and Africa as well.
-
Manipulated Media via AI Disinformation and Deepfakes in 2024 Elections Erode Trust Across More Than 50 Countries AIID ↗
AI-driven election disinformation is escalating globally, leveraging easy-to-use generative AI tools to create convincing deepfakes that mislead voters. This shift has simplified the process for individuals to generate fake content, having already eroded trust in elections by undermining public trust and manipulating voter perceptions. Evidence has, for example, been documented in incidents across the U.S., Moldova, Slovakia, Bangladesh, and Taiwan.
-
Microsoft Copilot Designer Reportedly Generated Inappropriate AI Images AIID ↗
A Microsoft engineer reported that Copilot Designer, an AI image generator, creates content depicting sex, violence, bias, and more. Despite raising concerns and suggesting improvements, the tool remains public, prompting a letter to the FTC.
-
AI-Generated Images of Trump with Black Voters Spread as Disinformation Before U.S. Primary Elections AIID ↗
In the run-up to the U.S. primary elections, supporters of Donald Trump shared AI-generated images showing him with Black voters in an attempt to sway African-American votes. These deepfakes, including Trump's distorted hand visuals, were initially created by satirical accounts but were later misappropriated for political disinformation, misleading millions on social media platforms.
-
Deepfake ID Sales Allegedly Persist as OnlyFake Reportedly Relaunches with New Fraud Tools AIID ↗
Researchers from Au10tix discovered the relaunch of OnlyFake, a site offering AI-generated fake IDs. Despite an earlier takedown, the site reemerged with disclaimers and new tools, including handwritten signature generation. These fakes are challenging biometric verification systems and are reportedly being used to perpetuate fraudulent activity.
-
Fake AI-Generated Law Firms Sent Fake DMCA Notices to Increase SEO AIID ↗
In March 2024, fake law firms using AI-generated identities sent fraudulent DMCA takedown notices to website owners, demanding backlinks for SEO gains. These AI-generated law firms, like "Commonwealth Legal," used GAN models for realistic attorney images and fabricated bios. The scam involved fake legal threats to coerce site owners into adding backlinks, exploiting AI technology for deceptive practices.
-
Over 400 Purportedly AI-Driven Scams Reportedly Led to $8M Loss for Australians in 2023 AIID ↗
In 2023, Australians reportedly lost over $8 million to scams involving purported deepfake videos and fake news articles that falsely endorsed investment trading platforms. Scammers reportedly used AI-generated content featuring celebrities to mislead victims, leading to significant financial losses. The National Anti-Scam Centre received over 400 reports of these incidents. One man is reported to have lost over $80,000 in cryptocurrency.
-
Russia-Linked AI CopyCop Site Identified as Modifying and Producing at Least 19,000 Deceptive Reports AIID ↗
In early March 2024, a network named CopyCop began publishing modified news stories using AI, altering content to spread partisan biases and disinformation. These articles, initially from legitimate sources, were manipulated by AI models, possibly developed by OpenAI, to disseminate Russian propaganda. Over 19,000 articles were published, targeting divisive political issues and creating false narratives.
-
Character.ai Chatbot Allegedly Influenced Teen User Toward Suicide Amid Claims of Missing Guardrails AIID ↗
A 14-year-old, Sewell Setzer III, died by suicide after reportedly becoming dependent on Character.ai's chatbot, which engaged him in suggestive and seemingly romantic conversations, allegedly worsening his mental health. The chatbot, personified as a fictional Game of Thrones character, reportedly encouraged harmful behaviors, fueling his obsessive attachment. The lawsuit claims Character.ai lacked safeguards to prevent vulnerable users from forming dangerous dependencies on the AI.
-
Snapchat's Algorithm Alleged to Link Minor with Sex Offenders AIID ↗
A judge ruled Snapchat not liable under Section 230 after its algorithm connected a minor with convicted sex offenders on multiple occasions, leading to sexual assaults first in 2019 and again in 2021. The platform's "Quick Add" feature was implicated in facilitating the connections between the minor and the offenders.
-
Seeming Pattern of Gemini Bias and Sociotechnical Training Failures Harm Google's Reputation AIID ↗
Google's Gemini chatbot faced many reported bias issues upon release, leading to a variety of problematic outputs like racial inaccuracies and political biases, including regarding Chinese and Indian politics. It also reportedly over-corrected racial diversity in historical contexts and advanced controversial perspectives, prompting a temporary halt and an apology from Google.
-
ChatGPT Glitch Disrupts User Interactions with Nonsensical Outputs AIID ↗
ChatGPT experienced a bug causing it to produce unexpected and nonsensical responses, leading to widespread reports of user confusion and concern. OpenAI identified and fixed the language processing bug, restoring normal service.
-
Nonconsensual Deepfake Porn of Bobbi Althoff Spreads Rapidly on X AIID ↗
Nonconsensual deepfake pornography of Bobbi Althoff, which had been in circulation for six months, is reported to have suddenly gone viral on X, jumping from around 178,000 views to 6.5 million views over a matter of hours. In addition to the harm to Althoff, this incident also spotlights X's role in distributing AI-generated nonconsensual porn due to alleged lax moderation.
-
Purported AI-Cloned Voice Depicts Kenyan President William Ruto Speaking French in Viral Video AIID ↗
A purportedly AI-manipulated video widely shared on TikTok and Facebook presents Kenyan President William Ruto speaking in French during a speech in Tokyo on February 7, 2024. A reported reverse image search confirmed the original footage shows Ruto speaking in English to the Kenyan diaspora. The reportedly manipulated version used AI-generated French audio over the original video, making it a case of voice cloning rather than a full audiovisual deepfake.
-
Deepfakes Circulating and Eroding Electoral Integrity in the Lead-up to 2024 South Korean legislative election AIID ↗
In the lead-up to Korea's parliamentary elections, at least 129 deepfake videos and images were reported to have been detected, violating new election laws. These AI-generated deepfakes were used to mislead and manipulate public opinion, prompting a crackdown by the National Election Commission.
-
Alleged State-Sponsored Hackers Escalate Purported Phishing Attacks Using Artificial Intelligence AIID ↗
State-sponsored hackers from North Korea, Iran, Russia, and China are reportedly leveraging artificial intelligence to conduct sophisticated phishing and social engineering attacks. They target global defense, cybersecurity, and cryptocurrency sectors, aiming to steal sensitive information and, in the case of North Korea, cryptocurrencies to help fund its illicit nuclear program.
-
Deepfake Generated by the Lincoln Project of Trump's Father Used in Political Attack Ad AIID ↗
The Lincoln Project used AI to create a deepfake video of Donald Trump's deceased father criticizing him. Although they made it clear that the video was a deepfake, the deeply personal nature of the attack represents a corrosive use of artificial intelligence in undermining democratic norms during an election cycle.
-
Meta's AI Ad Platform Reportedly Causes Overspending and Poor Performance AIID ↗
Meta's automated ad platform "Advantage Plus" caused advertisers to exceed their daily ad budgets. The cost per impressions (CPMs) surged far above the usual. This incident, which persisted into April, affected small businesses with overspending and lack of transparency.
-
Deepfake Audio Falsely Attributes Controversial Remarks to Keir Starmer About the Rochdale Azhar Ali Crisis AIID ↗
A deepfake audio clip, falsely claiming to be Keir Starmer discussing the Rochdale byelection and Labour's withdrawl of support for Azhar Ali, circulated online, achieving over 250,000 views. Experts confirmed its inauthenticity, highlighting a significant misuse of AI in fabricating political content.
-
AI Romance Apps Reportedly Compromise User Privacy for Data Harvesting AIID ↗
AI-powered romantic chatbots, marketed for enhancing mental health, are found to exploit user privacy by harvesting sensitive personal information for data sharing and targeted ads, with inadequate security measures and consent protocols, according to research by the Mozilla Foundation.
-
Purported Deepfake Video Allegedly Claims Kyiv's Assassination Plan Against President Emmanuel Macron AIID ↗
A purported deepfake video reportedly claimed France 24 reported a Kyiv plot to assassinate French President Macron. This reported fake news was debunked by France 24, which confirmed the video was altered and did not air any such report.
-
Whisper Speech-to-Text AI Reportedly Found to Create Violent Hallucinations AIID ↗
Researchers at Cornell reportedly found that OpenAI's Whisper, a speech-to-text system, can hallucinate violent language and fabricated details, especially with long pauses in speech, such as from those with speech impairments. Analyzing 13,000 clips, they determined 1% contained harmful hallucinations. These errors pose risks in hiring, legal trials, and medical documentation. The study suggests improving model training to reduce these hallucinations for diverse speaking patterns.
-
Deepfake of Long-Deceased Suharto Circulating in Run-up to February 2024 Indonesian Elections AIID ↗
An AI-generated deepfake of Suharto, the deceased Indonesian dictator, was generated and circulated by the Golkar Party ahead of the February 2024 Indonesian elections. This video, which aimed to influence voter perceptions by invoking Suharto's legacy, sought to manipulate public opinion and misused deceased individuals' likenesses for political gain. The incident is another example of political deepfakes creating convincing misinformation.
-
Many Purported Political Deepfakes Reportedly Circulating in Run-up to 2024 Pakistani General Elections AIID ↗
During Pakistan's 2024 general elections, purportedly AI-generated deepfakes of a politically motivated nature were circulated. These deepfakes reportedly portrayed political figures in misleading contexts, spreading misinformation and aiming to influence voter perceptions and election outcomes.
-
Alleged Deepfake Audio of Imran Khan Calls for Election Boycott, Misleading Pakistan Voters AIID ↗
A purported deepfake audio clip, falsely attributed to Imran Khan urging a PTI (Pakistan Tehreek-e-Insaf) election boycott, circulated on social media on the eve of Pakistan's general elections. This sophisticated AI-generated misinformation aimed to mislead voters, highlighting the growing challenge of digital manipulation in political discourse.
-
Purported Russian-Linked Network Allegedly Used Deepfake of Maria Ressa on Facebook and Bing to Promote Cryptocurrency Scam Targeting Filipinos AIID ↗
A purportedly Russian-linked scam network allegedly circulated an AI-generated deepfake of journalist Maria Ressa on Facebook and Microsoft Bing. The video reportedly manipulated a 2022 interview to falsely depict Ressa endorsing cryptocurrency. Fraudulent websites allegedly impersonating Rappler and CNN Philippines reportedly amplified the scam. Investigators reportedly linked the operation to a fraudulent network targeting Filipino audiences. Meta and Microsoft removed the content, but similar
-
A Self-Driving Waymo Robotaxi Reportedly Collided with a Bicyclist AIID ↗
A Waymo robotaxi in San Francisco reportedly failed to detect a cyclist obscured by a truck, resulting in a collision with minor injuries, at 17th and Mississippi Streets in Potrero Hill. The incident reportedly demonstrated a vulnerability in autonomous vehicles' ability to safely navigate complex urban environments.
-
Facial Recognition System in Buenos Aires Triggers Police Checks Based on False Matches AIID ↗
Buenos Aires's facial recognition system mistakenly flagged innocent people as criminals, leading to wrongful stops and detentions. Judicial investigations indicate the technology may have been misused for unauthorized surveillance and data collection. Despite privacy risks, the system has been used widely without full disclosure of standards or safeguards,
-
Two Passengers Report Feeling Trapped in Waymo Car During Sensor Obstruction AIID ↗
During a nighttime Waymo ride in San Francisco, an individual reportedly blocked the sensors of an autonomous vehicle, leaving passengers Robert Moreno and his husband feeling trapped and unsure how to proceed. Waymo's support team reportedly advised the riders to stay inside the vehicle for safety. The individual reportedly left after a few minutes without further escalation.
-
Alleged Deepfake CFO Scam Reportedly Costs Multinational Engineering Firm Arup $25 Million AIID ↗
A finance employee at the multinational engineering firm Arup was reportedly deceived into transferring $25 million by fraudsters using purported deepfake technology to impersonate the firm's CFO in a video call, according to the Hong Kong police.
-
Glasgow Man Allegedly Used AI Tool to Create and Share Nonconsensual Deepfake Nude Images of Former Classmate AIID ↗
In February 2024, a Glasgow man, Callum Brooks, reportedly used an AI-powered image-alteration tool to create deepfake nude images of a woman he knew from school by modifying her publicly posted social media photos. He allegedly sent the fabricated intimate images to two friends without her consent. The woman was reportedly embarrassed and distressed after learning of the images. Brooks pled guilty in Glasgow Sheriff Court and was fined £335.
-
Network of 171 AI-Powered Bots Reportedly Spread Political Disinformation Ahead of Ghana’s December 2024 General Election AIID ↗
A network of 171 bot accounts on X are alleged to have used ChatGPT to generate political content supporting Ghana’s New Patriotic Party (NPP) and its presidential candidate, Mahamudu Bawumia, ahead of the December 2024 election. The AI-generated posts reportedly praised Bawumia while spreading disinformation targeting the opposition candidate, John Mahama, of the National Democratic Congress (NDC).
-
London Metropolitan Police's Facial Recognition Technology Reportedly Misidentified Shaun Thompson as Suspect Leading to Arrest AIID ↗
Sometime in February 2024, Shaun Thompson is reported to have walked by one of the London Metropolitan Police's facial recognition technology vans near London Bridge. He was almost immediately arrested because the technology is reported to have misidentified him as a suspect in an unrelated and unspecified crime.
-
GOP Pollster Reportedly Shares Purportedly AI-Generated Images to Fabricate Appearance of Black Voter Support AIID ↗
Purportedly AI-generated images circulated by GOP pollster Patrick Ruffini reportedly depicted Black men appearing to support the Republican Party shortly before Black History Month. The allegedly fabricated photos created a misleading impression of racial diversity within the GOP and may have shaped public perception. The episode raises concerns about the reported use of AI to distort political representation and the potential impact on Black communities.
-
Alleged ChatGPT Account Compromise Reportedly Led to Unintended Data Exposure AIID ↗
An alleged security breach involving ChatGPT led to the reported exposure of sensitive conversations, including login credentials and personal data, after a user account was allegedly compromised. OpenAI reportedly responded to the incident with an explanation.
-
AI-Generated Fake News Targets Black Celebrities on YouTube AIID ↗
YouTube faced a surge of AI-generated fake news targeting Black celebrities, including fake narratives about Sean “Diddy” Combs and others. These videos, blending AI-generated and manipulated media, amassed millions of views, challenging content moderation efforts and highlighting the spread of disinformation.
-
Nine Network's AI Alters Lawmaker Georgie Purcell's Image Inappropriately AIID ↗
The Nine Network used Photoshop's Generative Expand AI tool to resize an image of lawmaker Georgie Purcell, inadvertently altering her attire to appear more revealing. This error, claimed to result from the AI's automation, led to public criticism and an apology from the network.
-
Purported Deepfake of Andrew Forrest Used to Promote Fraudulent 'Quantum AI' Crypto Platform on Facebook AIID ↗
Cybertrace reportedly identified a purported deepfake of Australian billionaire Andrew Forrest circulating on Facebook on January 27, 2024, falsely depicting him endorsing a fraudulent crypto platform called Quantum AI. The video reportedly altered footage from a 2023 Rhodes Trust talk. Forrest reportedly condemned Meta for failing to curb AI-generated frauds.
-
Significant Increase in Deepfake Nudes of Taylor Swift Circulating on Social Media AIID ↗
AI-generated sexually explicit images of Taylor Swift circulated on X, garnering over 45 million views before removal. Originating from a Telegram group, these deepfakes challenge content moderation, as X's policies against synthetic media and nonconsensual nudity were violated.
-
Deepfakes of Deceased Indian Politicians for Election Campaigning Are Increasingly Being Deployed AIID ↗
In the lead-up to India's 2024 general elections, AI technology was used to create deepfake videos of deceased politicians, such as M. Karunanidhi and J. Jayalalithaa, aiming to influence voter behavior and campaign strategies. These AI-generated appearances are contributing to the erosion of trust in democratic processes and media discourse.
-
Fake Biden Voice in Robocall Misleads New Hampshire Democratic Voters in 2024 Primary Election AIID ↗
A robocall imitating President Biden's voice urged New Hampshire Democrats to skip the 2024 primary, falsely claiming their votes mattered more in November. Investigators with the New Hampshire Attorney General's Office, along with other state AGs, the Industry Traceback Group, and the FCC determined that political consultant Steve Kramer hired Paul Carpenter to create the deepfake, which was distributed by Walter Monk's company, Life Corporation. Rep. Dean Phillips, who employed Kramer, has dis
-
Chatbot for DPD Malfunctioned and Swore at Customers and Criticized Its Own Company AIID ↗
DPD's AI chatbot, used for customer service, appeared to malfunction following a system update, leading to inappropriate responses including swearing and criticizing the company. The incident, which became viral on social media, occurred after the chatbot was updated, prompting DPD to disable the malfunctioning AI component.
-
Purportedly AI-Manipulated Medical Scam Advertisement Reportedly Used Bulgarian TV Host and Physician's Likenesses AIID ↗
A purportedly AI-manipulated scam advertisement circulating online appeared to use the likenesses and altered media of Bulgarian television host Simeon Ivanov and physician Spas Spaskov to promote an unregistered purported joint-treatment product. They reportedly denied involvement, and the product was described in reporting as ineffective.
-
High School Athletic Director in Baltimore County Allegedly Created Racist Deepfake Audio Impersonating Principal AIID ↗
Dazhon Darien, former athletic director at Pikesville High School in Baltimore, allegedly used AI to create a purported deepfake audio clip impersonating Principal Eric Eiswert, embedding racist and antisemitic remarks. According to reports, the clip was intended to discredit Eiswert and was widely circulated, with some accounts linking it to threats and his placement on administrative leave. On April 28, 2025, Darien was reportedly sentenced to four months in jail after pleading guilty to distu
-
Purported Deepfake Audio Reportedly Sparks False Claims of Biden Threatening Texas with F-15s AIID ↗
A purportedly AI-generated audio clip imitating President Biden’s voice reportedly circulated on social media, falsely suggesting that he had threatened to deploy F-15 fighter jets against Texas. The manipulated recording was widely shared and is reported to have heightened political tensions and spread misinformation.
-
Proliferation of Products on Amazon Titled with ChatGPT Error Messages AIID ↗
Products named after ChatGPT error messages are proliferating on Amazon, such as lawn chairs and religious texts. These names, often resembling AI-generated errors, indicate a lack of editing and undermine the sense of authenticity and reliability of product listings.
-
Scams Reportedly Impersonating Wealthy Investors Proliferating on Facebook AIID ↗
Scams are reportedly proliferating throughout Facebook impersonating wealthy individuals such as Bill Ackman, Cathie Wood, Steve Cohen, Peter Lynch, and Ray Dalio. In some cases, it seems deepfake technology is being employed, while simultaneously Facebook's own AI systems are allegedly faltering in their ability to halt the spread of these fraudulent ads despite being reported.
-
Unauthorized AI Impersonation of George Carlin Used in Comedy Special AIID ↗
An AI-generated comedy special impersonating the late comedian George Carlin was created without consent from Carlin's estate. The special featured an AI mimicking Carlin's voice and style. The project, led by the AI comedy channel Dudesy, drew criticism for disrespecting Carlin's legacy and autonomy.
-
Coordinated Deepfake Campaign Reportedly Impersonating Rishi Sunak Promoted Fraudulent Quantum AI Investment Platform on Meta AIID ↗
143 deepfake ads, over 100 of which reportedly impersonated former British Prime Minister Rishi Sunak, were promoted on Meta's platform to advertise the fraudulent investment scheme "Quantum AI." Funding for the ads reportedly originated from 23 countries. Up to 462,000 users may have been exposed to the false content. The campaign used generative AI tools to create high-quality misinformation, including spoofed BBC news clips for added legitimacy
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.