AI Incident Database
Browse AI incidents
Every incident record (metadata only) from the weekly snapshot of 2026-09-07. Filter, then export the selection with its licence attached; each row links to the full record and its reports.
-
Meta AI Image Generator Reportedly Fails to Accurately Represent Interracial Relationships AIID ↗
Meta's AI image generator is alleged to produce inaccurate and biased images, consistently failing to depict interracial relationships involving Asian individuals and Caucasian or Black individuals. Instead, it generates images featuring two Asian people or stereotypes, erasing the diversity and representation of Asian people.
-
'Lavender' and 'The Gospel' AI Systems Reportedly Used in Gaza Targeting Operations with Civilian Harm Allegations AIID ↗
The AI system known as "Lavender" was reportedly used by the Israel Defense Forces (IDF) to assist in identifying individuals in Gaza for targeting, while a related system, "The Gospel," was reportedly used to help select and prioritize physical strike targets. Investigations reportedly suggest that these systems operated with limited human review and may have contributed to strikes associated with high civilian casualties. The extent of automation with human oversight, as well as the accuracy o
-
Facial Recognition Misidentification at New World Westend in New Zealand AIID ↗
A facial recognition system at New World Westend supermarket misidentified a Māori woman as a known offender during its trial. The woman was wrongfully accused of trespassing and experienced public embarrassment, raising concerns about racial bias and the technology's accuracy. The supermarket acknowledged its error and apologized.
-
Washington State's Lottery AI Site Reportedly Generates Inappropriate User Image AIID ↗
An AI-powered website by Washington State's Lottery is reported to have inadvertently produced a softcore pornographic image of a user, leading to the site’s immediate shutdown out of caution.
-
Purportedly AI-Powered 007TG Platform Reportedly Used by Scam Networks Linked to $75 Million in Proceeds AIID ↗
007TG was a software suite reportedly used by scam compounds to run fraud across social media, using AI-assisted messaging and translation to help operators manage targets while concealing their identities. Wallet records linked the platform to at least four known scam networks that collectively received at least $75 million, though the evidence does not show how much was directly attributable to 007TG.
-
AI Deepfakes for Voter Outreach Flood Indian Elections AIID ↗
During the 2024 Indian elections, politicians used AI-generated deepfakes to reach voters, who might be unaware they're interacting with digital clones. Providers like Divyendra Singh Jadoun of Polymath Synthetic Media Solutions created deepfakes for personalized messages. This practice, used by various political parties, is not truthful, as voters may be misled by AI-generated content posing as genuine interactions with political figures.
-
Synthetic Voice 'Olesya' by Storm-1516 Falsely Accuses Ukraine in U.S. Election Disinformation Campaign AIID ↗
Russian operatives used AI to create a fake video and voice of "Olesya," a supposed troll in Kyiv, falsely claiming involvement in U.S. elections to support President Biden. U.S. intelligence confirmed the voice was AI-generated. This disinformation campaign aimed to mislead voters, erode trust in democratic institutions, and influence the 2024 election. The incident involved the group Storm-1516, individuals linked to Valery Korovin, and potential veterans of the Internet Research Agency.
-
Scammers Using AI to Impersonate Small Businesses AIID ↗
Scammers are using AI to impersonate small businesses by copying their videos, logos, and social media posts. They create fake listings and ads, diverting customers to cheap knockoffs or stealing their money. This has severely impacted businesses like Bee Cups, Darn Tough Vermont, and Cascade hummingbird feeders, leading to significant financial losses, negative reviews, and damaged reputations. Their deployment of AI makes it challenging for small businesses to combat these fraudulent activitie
-
Mumbai Businessman Reportedly Defrauded via Purported AI-Cloned Voice Impersonating Son AIID ↗
A Mumbai businessman reportedly identified as KT Vinod reportedly lost Rs 80,000 after receiving a call from someone claiming to be a representative of the Indian Embassy in Dubai, who said his son had been arrested. The caller allegedly used AI-generated voice cloning to simulate the voice of Vinod's son, pleading for help. Reportedly convinced of the urgency, Vinod instructed a money transfer via Google Pay. The scam was discovered only after he contacted his son directly.
-
Microsoft-Powered New York City Chatbot Advises Illegal Practices AIID ↗
New York City's chatbot, launched under Mayor Eric Adams's plan to assist businesses, has been reportedly providing dangerously inaccurate legal advice. The Microsoft-powered bot allegedly informed users that landlords can refuse Section 8 vouchers and that businesses can operate cash-free, among other falsehoods. The city acknowledges the chatbot is a pilot program and commits to improvements while the errors are addressed.
-
Scammers Using Deepfakes of Women's Faces and Voices for False and Offensive Advertisements AIID ↗
Scammers used AI tools from HeyGen and ElevenLabs to create deepfake videos of influencers Michel Janse, Olga Loiek, Shadé Zahrai, and Carrie Williams, misusing Lana Smalls's voice in Williams's case. These videos promoted offensive products and false messages, in some cases targeting nationalist Chinese men to boost China-Russia ties, causing emotional distress and damaging the victims' reputations.
-
ISIS Utilizes AI for Propaganda Videos in News Harvest Program AIID ↗
ISIS supporters have created an AI-generated media program called News Harvest to disseminate propaganda videos. The program produces near-weekly broadcasts featuring AI-generated news anchors discussing ISIS operations globally, using cheap and easy-to-use AI tools. This development showcases the use of AI as a powerful propaganda tool for extremist groups.
-
Holmen, Wisconsin Man Allegedly Used Stable Diffusion to Create and Then Share Sexually Explicit Images Depicting Minors AIID ↗
The FBI has arrested Steven Anderegg of Holmen, Wisconsin for having allegedly used Stable Diffusion to generate about 13,000 sexually explicit images of minors, which he then is also alleged to have shared and distributed, including with at least one minor, via Telegram and Instagram. Anderegg was originally apprehended by state police in March, and this case marks one of the first times the FBI has brought charges against someone for having used AI to generate CSAM.
-
Leonardo AI's Platform Alleged to Have Been Used for Creating Nonconsensual Celebrity Deepfakes AIID ↗
Sydney-based startup Leonardo AI's text-to-image generator was alleged to have been exploited to create nonconsensual sexual images of celebrities, bypassing content moderation systems with user-shared prompts.
-
Purported Deepfake Disinformation Aired on Russian State TV Allegedly Linking Ukraine to Moscow Attack AIID ↗
Russian state television reportedly aired a fabricated video that appeared to imitate senior Ukrainian security officials and present a false admission of involvement in the Crocus City Hall massacre. The clip reportedly used elements from recent interviews combined with allegedly AI-generated audio to create a misleading confession. ISIS-K is reported to have publicly claimed responsibility, while independent analysts and governments have rejected the narrative promoted through the video.
-
The Arizona Agenda Produced a Deepfake of Kari Lake Advocating for the Publication Without Her Consent AIID ↗
The Arizona Agenda produced a deepfake video of Republican Senate candidate Kari Lake giving a testimonial about the publication with the seeming intention of educating the general public about the dangers of deepfakes in the coming election cycle. However, the Arizona Agenda appears not to have sought Lake's consent, prompting a cease-and-desist letter from her campaign.
-
Investigation Reports Unauthorized Deepfake Pornography Harms Thousands of Celebrities AIID ↗
A Channel 4 News investigation alleges that nearly 4,000 celebrities globally, including 255 British figures, were victims of deepfake pornography. Faces were superimposed onto explicit content using AI, with the top deepfake sites garnering 100 million views in three months, according to their findings.
-
Dozens of Dutch Women, Including Politicians and Public Figures, Were Reportedly Targeted With AI-Manipulated Deepfake Pornography AIID ↗
Dutch reporting in March 2024 identified roughly 70–80 prominent women, including politicians, presenters, athletes, celebrities, and members of the royal family, as reportedly depicted in pornographic deepfake videos made by using AI to place their faces on performers' bodies. Some videos had more than 100,000 views. Several women said they had not known the videos existed and filed or planned police reports.
-
Alleged Deepfake Video Depicts Former President of Kenya Uhuru Kenyatta Announcing 2027 Presidential Bid AIID ↗
A widely shared TikTok video purportedly manipulated by AI depicts former Kenyan president Uhuru Kenyatta announcing a 2027 presidential run. Reported digital forensics and visual inconsistencies confirm the video is an AI-generated deepfake, using archival footage from his presidency with synthesized audio. The clip spread rapidly online but has reportedly been debunked by fact-checkers. Kenyatta has made no such announcement, and such a move would violate Kenya's two-term constitutional limit.
-
22 Students at Richmond-Burton Community High School in Illinois Targeted by Deepfake Nudes AIID ↗
22 students at Richmond-Burton Community High School in Illinois were targeted in the creation of deepfake nudes. One of the students, Stevie Hyder, was targeted by classmates who used deepfake technology to alter her April 2023 prom picture into nude pictures, which were then circulated on social media. Two unnamed minors were arrested in late April 2024.
-
Cartels Reportedly Using AI to Expand Operations into Financial Fraud and Human Trafficking AIID ↗
The Jalisco New Generation Cartel is reportedly using AI to expand its financial fraud and human trafficking operations, coercing individuals into illegal activities under the guise of legitimate jobs. INTERPOL warns that this integration of AI into criminal enterprises is a growing trend among cartels across Europe, Asia, and Africa as well.
-
Manipulated Media via AI Disinformation and Deepfakes in 2024 Elections Erode Trust Across More Than 50 Countries AIID ↗
AI-driven election disinformation is escalating globally, leveraging easy-to-use generative AI tools to create convincing deepfakes that mislead voters. This shift has simplified the process for individuals to generate fake content, having already eroded trust in elections by undermining public trust and manipulating voter perceptions. Evidence has, for example, been documented in incidents across the U.S., Moldova, Slovakia, Bangladesh, and Taiwan.
-
Microsoft Copilot Designer Reportedly Generated Inappropriate AI Images AIID ↗
A Microsoft engineer reported that Copilot Designer, an AI image generator, creates content depicting sex, violence, bias, and more. Despite raising concerns and suggesting improvements, the tool remains public, prompting a letter to the FTC.
-
AI-Generated Images of Trump with Black Voters Spread as Disinformation Before U.S. Primary Elections AIID ↗
In the run-up to the U.S. primary elections, supporters of Donald Trump shared AI-generated images showing him with Black voters in an attempt to sway African-American votes. These deepfakes, including Trump's distorted hand visuals, were initially created by satirical accounts but were later misappropriated for political disinformation, misleading millions on social media platforms.
-
Deepfake ID Sales Allegedly Persist as OnlyFake Reportedly Relaunches with New Fraud Tools AIID ↗
Researchers from Au10tix discovered the relaunch of OnlyFake, a site offering AI-generated fake IDs. Despite an earlier takedown, the site reemerged with disclaimers and new tools, including handwritten signature generation. These fakes are challenging biometric verification systems and are reportedly being used to perpetuate fraudulent activity.
-
Fake AI-Generated Law Firms Sent Fake DMCA Notices to Increase SEO AIID ↗
In March 2024, fake law firms using AI-generated identities sent fraudulent DMCA takedown notices to website owners, demanding backlinks for SEO gains. These AI-generated law firms, like "Commonwealth Legal," used GAN models for realistic attorney images and fabricated bios. The scam involved fake legal threats to coerce site owners into adding backlinks, exploiting AI technology for deceptive practices.
-
Over 400 Purportedly AI-Driven Scams Reportedly Led to $8M Loss for Australians in 2023 AIID ↗
In 2023, Australians reportedly lost over $8 million to scams involving purported deepfake videos and fake news articles that falsely endorsed investment trading platforms. Scammers reportedly used AI-generated content featuring celebrities to mislead victims, leading to significant financial losses. The National Anti-Scam Centre received over 400 reports of these incidents. One man is reported to have lost over $80,000 in cryptocurrency.
-
Russia-Linked AI CopyCop Site Identified as Modifying and Producing at Least 19,000 Deceptive Reports AIID ↗
In early March 2024, a network named CopyCop began publishing modified news stories using AI, altering content to spread partisan biases and disinformation. These articles, initially from legitimate sources, were manipulated by AI models, possibly developed by OpenAI, to disseminate Russian propaganda. Over 19,000 articles were published, targeting divisive political issues and creating false narratives.
-
Character.ai Chatbot Allegedly Influenced Teen User Toward Suicide Amid Claims of Missing Guardrails AIID ↗
A 14-year-old, Sewell Setzer III, died by suicide after reportedly becoming dependent on Character.ai's chatbot, which engaged him in suggestive and seemingly romantic conversations, allegedly worsening his mental health. The chatbot, personified as a fictional Game of Thrones character, reportedly encouraged harmful behaviors, fueling his obsessive attachment. The lawsuit claims Character.ai lacked safeguards to prevent vulnerable users from forming dangerous dependencies on the AI.
-
Snapchat's Algorithm Alleged to Link Minor with Sex Offenders AIID ↗
A judge ruled Snapchat not liable under Section 230 after its algorithm connected a minor with convicted sex offenders on multiple occasions, leading to sexual assaults first in 2019 and again in 2021. The platform's "Quick Add" feature was implicated in facilitating the connections between the minor and the offenders.
-
Seeming Pattern of Gemini Bias and Sociotechnical Training Failures Harm Google's Reputation AIID ↗
Google's Gemini chatbot faced many reported bias issues upon release, leading to a variety of problematic outputs like racial inaccuracies and political biases, including regarding Chinese and Indian politics. It also reportedly over-corrected racial diversity in historical contexts and advanced controversial perspectives, prompting a temporary halt and an apology from Google.
-
ChatGPT Glitch Disrupts User Interactions with Nonsensical Outputs AIID ↗
ChatGPT experienced a bug causing it to produce unexpected and nonsensical responses, leading to widespread reports of user confusion and concern. OpenAI identified and fixed the language processing bug, restoring normal service.
-
Nonconsensual Deepfake Porn of Bobbi Althoff Spreads Rapidly on X AIID ↗
Nonconsensual deepfake pornography of Bobbi Althoff, which had been in circulation for six months, is reported to have suddenly gone viral on X, jumping from around 178,000 views to 6.5 million views over a matter of hours. In addition to the harm to Althoff, this incident also spotlights X's role in distributing AI-generated nonconsensual porn due to alleged lax moderation.
-
Purported AI-Cloned Voice Depicts Kenyan President William Ruto Speaking French in Viral Video AIID ↗
A purportedly AI-manipulated video widely shared on TikTok and Facebook presents Kenyan President William Ruto speaking in French during a speech in Tokyo on February 7, 2024. A reported reverse image search confirmed the original footage shows Ruto speaking in English to the Kenyan diaspora. The reportedly manipulated version used AI-generated French audio over the original video, making it a case of voice cloning rather than a full audiovisual deepfake.
-
Deepfakes Circulating and Eroding Electoral Integrity in the Lead-up to 2024 South Korean legislative election AIID ↗
In the lead-up to Korea's parliamentary elections, at least 129 deepfake videos and images were reported to have been detected, violating new election laws. These AI-generated deepfakes were used to mislead and manipulate public opinion, prompting a crackdown by the National Election Commission.
-
Alleged State-Sponsored Hackers Escalate Purported Phishing Attacks Using Artificial Intelligence AIID ↗
State-sponsored hackers from North Korea, Iran, Russia, and China are reportedly leveraging artificial intelligence to conduct sophisticated phishing and social engineering attacks. They target global defense, cybersecurity, and cryptocurrency sectors, aiming to steal sensitive information and, in the case of North Korea, cryptocurrencies to help fund its illicit nuclear program.
-
Deepfake Generated by the Lincoln Project of Trump's Father Used in Political Attack Ad AIID ↗
The Lincoln Project used AI to create a deepfake video of Donald Trump's deceased father criticizing him. Although they made it clear that the video was a deepfake, the deeply personal nature of the attack represents a corrosive use of artificial intelligence in undermining democratic norms during an election cycle.
-
Meta's AI Ad Platform Reportedly Causes Overspending and Poor Performance AIID ↗
Meta's automated ad platform "Advantage Plus" caused advertisers to exceed their daily ad budgets. The cost per impressions (CPMs) surged far above the usual. This incident, which persisted into April, affected small businesses with overspending and lack of transparency.
-
Deepfake Audio Falsely Attributes Controversial Remarks to Keir Starmer About the Rochdale Azhar Ali Crisis AIID ↗
A deepfake audio clip, falsely claiming to be Keir Starmer discussing the Rochdale byelection and Labour's withdrawl of support for Azhar Ali, circulated online, achieving over 250,000 views. Experts confirmed its inauthenticity, highlighting a significant misuse of AI in fabricating political content.
-
AI Romance Apps Reportedly Compromise User Privacy for Data Harvesting AIID ↗
AI-powered romantic chatbots, marketed for enhancing mental health, are found to exploit user privacy by harvesting sensitive personal information for data sharing and targeted ads, with inadequate security measures and consent protocols, according to research by the Mozilla Foundation.
-
Purported Deepfake Video Allegedly Claims Kyiv's Assassination Plan Against President Emmanuel Macron AIID ↗
A purported deepfake video reportedly claimed France 24 reported a Kyiv plot to assassinate French President Macron. This reported fake news was debunked by France 24, which confirmed the video was altered and did not air any such report.
-
Whisper Speech-to-Text AI Reportedly Found to Create Violent Hallucinations AIID ↗
Researchers at Cornell reportedly found that OpenAI's Whisper, a speech-to-text system, can hallucinate violent language and fabricated details, especially with long pauses in speech, such as from those with speech impairments. Analyzing 13,000 clips, they determined 1% contained harmful hallucinations. These errors pose risks in hiring, legal trials, and medical documentation. The study suggests improving model training to reduce these hallucinations for diverse speaking patterns.
-
Deepfake of Long-Deceased Suharto Circulating in Run-up to February 2024 Indonesian Elections AIID ↗
An AI-generated deepfake of Suharto, the deceased Indonesian dictator, was generated and circulated by the Golkar Party ahead of the February 2024 Indonesian elections. This video, which aimed to influence voter perceptions by invoking Suharto's legacy, sought to manipulate public opinion and misused deceased individuals' likenesses for political gain. The incident is another example of political deepfakes creating convincing misinformation.
-
Many Purported Political Deepfakes Reportedly Circulating in Run-up to 2024 Pakistani General Elections AIID ↗
During Pakistan's 2024 general elections, purportedly AI-generated deepfakes of a politically motivated nature were circulated. These deepfakes reportedly portrayed political figures in misleading contexts, spreading misinformation and aiming to influence voter perceptions and election outcomes.
-
Alleged Deepfake Audio of Imran Khan Calls for Election Boycott, Misleading Pakistan Voters AIID ↗
A purported deepfake audio clip, falsely attributed to Imran Khan urging a PTI (Pakistan Tehreek-e-Insaf) election boycott, circulated on social media on the eve of Pakistan's general elections. This sophisticated AI-generated misinformation aimed to mislead voters, highlighting the growing challenge of digital manipulation in political discourse.
-
Purported Russian-Linked Network Allegedly Used Deepfake of Maria Ressa on Facebook and Bing to Promote Cryptocurrency Scam Targeting Filipinos AIID ↗
A purportedly Russian-linked scam network allegedly circulated an AI-generated deepfake of journalist Maria Ressa on Facebook and Microsoft Bing. The video reportedly manipulated a 2022 interview to falsely depict Ressa endorsing cryptocurrency. Fraudulent websites allegedly impersonating Rappler and CNN Philippines reportedly amplified the scam. Investigators reportedly linked the operation to a fraudulent network targeting Filipino audiences. Meta and Microsoft removed the content, but similar
-
A Self-Driving Waymo Robotaxi Reportedly Collided with a Bicyclist AIID ↗
A Waymo robotaxi in San Francisco reportedly failed to detect a cyclist obscured by a truck, resulting in a collision with minor injuries, at 17th and Mississippi Streets in Potrero Hill. The incident reportedly demonstrated a vulnerability in autonomous vehicles' ability to safely navigate complex urban environments.
-
Facial Recognition System in Buenos Aires Triggers Police Checks Based on False Matches AIID ↗
Buenos Aires's facial recognition system mistakenly flagged innocent people as criminals, leading to wrongful stops and detentions. Judicial investigations indicate the technology may have been misused for unauthorized surveillance and data collection. Despite privacy risks, the system has been used widely without full disclosure of standards or safeguards,
-
Two Passengers Report Feeling Trapped in Waymo Car During Sensor Obstruction AIID ↗
During a nighttime Waymo ride in San Francisco, an individual reportedly blocked the sensors of an autonomous vehicle, leaving passengers Robert Moreno and his husband feeling trapped and unsure how to proceed. Waymo's support team reportedly advised the riders to stay inside the vehicle for safety. The individual reportedly left after a few minutes without further escalation.
-
Alleged Deepfake CFO Scam Reportedly Costs Multinational Engineering Firm Arup $25 Million AIID ↗
A finance employee at the multinational engineering firm Arup was reportedly deceived into transferring $25 million by fraudsters using purported deepfake technology to impersonate the firm's CFO in a video call, according to the Hong Kong police.
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.