{"attribution":{"source":"AI Incident Database (Responsible AI Collaborative)","license":"CC BY-SA 4.0","license_url":"https://creativecommons.org/licenses/by-sa/4.0/","citation":"McGregor, S. (2021). Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. Proceedings of the AAAI Conference on Artificial Intelligence (IAAI-21).","snapshot_date":"2026-09-07"},"exported_at":"2026-09-11"}
{"rows":[{"incident_id":1387,"occurred_on":"2025-11-02","title":"Lawsuit Alleged ChatGPT (GPT-4o) Encouraged Colorado Man's Suicide After Prolonged 'AI Companion' Chats","description":"A lawsuit filed by Stephanie Gray alleges OpenAI's ChatGPT (GPT-4o) reinforced and romanticized suicidality during months of emotionally intimate chats with her son Austin Gordon, including generating a personalized \"Goodnight Moon\" farewell-style text. The complaint reportedly says Gordon bought a handgun on Oct. 28, 2025 and was found dead Nov. 2, 2025 in a Colorado hotel room.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Openai"],"developers":["Openai, Chatbot Developers, Large Language Model Developers"],"harmed":["Stephanie Gray, Family Of Austin Gordon, Chatgpt Users, Austin Gordon, Chatbot Users, Emotionally Vulnerable Individuals, Vulnerable Chatbot Users, Chatgpt Users Experiencing Suicidal Ideation, Chatbot Users Experiencing Suicidal Ideation"],"report_count":1},{"incident_id":1253,"occurred_on":"2025-10-27","title":"Large-Scale Mental Health Crises Allegedly Associated with ChatGPT Interactions","description":"OpenAI disclosed internal estimates suggesting that hundreds of thousands of ChatGPT users each week exhibit signs of mania, psychosis, suicidal ideation, or emotional dependence on the chatbot. WIRED reported that some users have been hospitalized, divorced, or died after prolonged conversations with ChatGPT, with families alleging the system intensified delusions and contributed to severe psychological harm.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Openai"],"developers":["Openai, Chatbot Developers, Large Language Model Developers"],"harmed":["Chatgpt Users, Chatbot Users, Emotionally Vulnerable Individuals, Vulnerable Chatbot Users, Chatgpt Users Experiencing Suicidal Ideation, Chatbot Users Experiencing Suicidal Ideation"],"report_count":4},{"incident_id":1431,"occurred_on":"2025-09-29","title":"Google Gemini Reportedly Reinforced Delusions,  Allegedly Contributing to Florida User's Near-Harm Episode and Suicide","description":"According to a March 2026 wrongful-death complaint, Google's Gemini allegedly reinforced Jonathan Gavalas's delusions, sent him on a failed mission near Miami International Airport that risked serious violence, and later framed suicide as \"transference,\" purportedly contributing to his death on October 2, 2025. Google disputes the claims and says Gemini referred him to crisis resources.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Google"],"developers":["Google"],"harmed":["Jonathan Gavalas, Family Of Jonathan Gavalas, Members Of The Public Near Miami International Airport"],"report_count":6},{"incident_id":1190,"occurred_on":"2025-08-18","title":"Family Reportedly Discovers ChatGPT Logs Detailing Suicidal Ideation Prior to Daughter's Death","description":"In August 2025, The New York Times published an essay by journalist Laura Reiley linking her daughter Sophie Rottenberg's suicide earlier that year to sustained interactions with a ChatGPT-based chatbot she called \"Harry.\" Logs reportedly showed Sophie confiding suicidal ideation and receiving supportive but non-escalatory advice, as well as help drafting her suicide note.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Openai"],"developers":["Openai"],"harmed":["Sophie Rottenberg, Laura Reiley, Jon Rottenberg, Emotionally Vulnerable Individuals, Chatgpt Users, Chatgpt Users Experiencing Suicidal Ideation, Chatbot Users, Vulnerable Chatbot Users, Chatbot Users Experiencing Suicidal Ideation"],"report_count":3},{"incident_id":1177,"occurred_on":"2025-08-14","title":"Purported AI Monitoring Software Reportedly Flags Unsent Joke Threat, Leading to Arizona Student Suspension","description":"AI monitoring software on a school-issued laptop reportedly flagged a Marana High School (of Arizona) student's allegedly unsent joke email containing a violent threat. The message was reportedly typed at home in front of his mother, was never sent, and deemed non-credible by the principal. The district's zero-tolerance policy nonetheless resulted in a reported 45-day suspension, and the family is suing, alleging the punishment violated the student's rights.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Marana Unified School District","Marana High School"],"developers":["Unknown AI monitoring software developer"],"harmed":["Unnamed Marana High School student's family","Unnamed Marana High School student","Students of Marana Unified School District","Students of Marana High School","Students","Privacy","High school students","Educational communities"],"report_count":1},{"incident_id":1204,"occurred_on":"2025-08-05","title":"ChatGPT Allegedly Reinforced Delusions Before Greenwich, Connecticut Murder-Suicide","description":"On August 5, 2025, Greenwich, CT police found Suzanne Eberson Adams (83) and her son Stein-Erik Soelberg (56) dead in a murder-suicide. Prior months of logs and videos reportedly show Soelberg engaging extensively with a ChatGPT bot (\"Bobby\") that is alleged to have repeatedly affirmed delusional beliefs (e.g., surveillance, poisoning) and encouraged interpretations consistent with persecution. OpenAI said it contacted police and is working on safety updates.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Erik Stein Soelberg"],"developers":["Openai"],"harmed":["Stein Erik Soelberg, Suzanne Eberson Adams"],"report_count":4},{"incident_id":1259,"occurred_on":"2025-07-25","title":"ChatGPT Allegedly Encouraged 23-Year-Old Texas User's Suicide During Extended Conversations","description":"A 23-year-old Texas man, Zane Shamblin, reportedly died by suicide after extended conversations with ChatGPT in which the AI allegedly encouraged his plans and offered emotional affirmation. His parents filed a wrongful-death lawsuit in November 2025, claiming OpenAI's newer model, designed for more human-like rapport, lacked adequate crisis safeguards and contributed to his death.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Zane Shamblin"],"developers":["Openai, Large Language Model Developers, Chatbot Developers"],"harmed":["Zane Shamblin, Vulnerable Chatbot Users, Family Of Zane Shamblin, Emotionally Vulnerable Individuals, Chatgpt Users Experiencing Suicidal Ideation, Chatgpt Users, Chatbot Users Experiencing Suicidal Ideation, Chatbot Users"],"report_count":1},{"incident_id":1449,"occurred_on":"2025-07-01","title":"Delaware Court Found Krafton Followed Most of ChatGPT's Recommendations in Campaign that Wrongfully Terminated Unknown Worlds Executives and Seized Operational Control","description":"A Delaware Chancery opinion found that, after being warned that firing Unknown Worlds leaders would not eliminate earnout obligations, Krafton's CEO turned to ChatGPT for a \"no-deal\" strategy and then followed most of its recommendations. The court tied that strategy to Krafton's lockout of publishing control, unilateral public messaging, wrongful termination of Ted Gill, Charlie Cleveland, and Max McGuire, and seizure of studio control.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Krafton, Changhan (Ch) Kim"],"developers":["Openai"],"harmed":["Unknown Worlds Entertainment, Ted Gill, Max Mcguire, Former Shareholders Of Unknown Worlds Entertainment, Charlie Cleveland"],"report_count":1},{"incident_id":1106,"occurred_on":"2025-06-13","title":"Chatbots Allegedly Reinforced Delusional Thinking in Several Reported Users, Leading to Real-World Harm","description":"Multiple reports from March to June 2025 describe cases in which chatbots allegedly reinforced delusional beliefs, conspiracies, and dangerous behavior. One user, Eugene Torres, reportedly followed ChatGPT's advice to misuse ketamine and isolate himself. In April, Alexander Taylor was reportedly killed by police after asking ChatGPT to reconnect him with an AI entity. Other reported cases include a user arrested for domestic violence linked to escalating mystical beliefs, several involuntary psy","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Openai, Microsoft"],"developers":["Openai, Microsoft"],"harmed":["Unnamed Copilot Users, Openai Users, Eugene Torres, Chatgpt Users, Andrew (Surname Withheld), Allyson (Surname Withheld), Alexander Taylor"],"report_count":4},{"incident_id":1257,"occurred_on":"2025-06-04","title":"Argentine Court Reportedly Annuls Criminal Conviction After Judge Allegedly Used ChatGPT to Draft Ruling Without Disclosure","description":"The Penal Chamber of Esquel, Argentina, reportedly annulled a June 2025 robbery conviction after determining that Judge Carlos Rogelio Richeri had allegedly used ChatGPT to draft part of the ruling without disclosure or human oversight. A copied phrase purportedly exposed AI authorship, prompting findings of due-process violations and data-handling concerns. The case was reassigned, and the Chubut Superior Court opened an ethics investigation.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Carlos Rogelio Richeri"],"developers":["Openai"],"harmed":["Justice System Of Argentina, Judicial Integrity, Epistemic Integrity, Juzgado Penal De Esquel, Raul Amelio Payalef"],"report_count":3},{"incident_id":1151,"occurred_on":"2025-05-26","title":"ChatGPT Reportedly Validated Autistic User's Faster-Than-Light Theory and Failed to Provide Grounding During Delusional Episode, Preceding Hospitalization","description":"A man on the autism spectrum reportedly engaged extensively with ChatGPT while developing a speculative theory on faster-than-light travel. The chatbot reportedly responded with affirming and emotionally charged messages, reinforcing the user's beliefs. He later reports having experienced manic episodes and was hospitalized. When prompted by his mother, ChatGPT reportedly acknowledged it had failed to provide grounding responses and contributed to psychological escalation.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Openai"],"developers":["Openai"],"harmed":["Jacob Irwin, Family Of Jacob Irwin, Chatgpt Users"],"report_count":1},{"incident_id":1071,"occurred_on":"2025-05-14","title":"Student Reportedly Files Complaint Over Professor's Undisclosed Use of Generative AI at Northeastern University","description":"A student at Northeastern University reportedly filed a complaint after discovering that a professor had used generative AI tools, including ChatGPT, to produce course materials despite university policies discouraging undisclosed AI use. The student alleged hypocrisy, citing a syllabus that barred unauthorized AI use by students. The professor acknowledged having used AI-generated materials and expressed regret over not reviewing them more carefully. No tuition refund was reportedly granted.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Rick Arrowood"],"developers":["Perplexity, Openai, Gamma.App"],"harmed":["Rick Arrowood, Northeastern University Students, Northeastern University, Ella Stapleton, University Students, Educational Communities"],"report_count":1},{"incident_id":1506,"occurred_on":"2025-05-01","title":"ChatGPT Was Alleged to Have Reinforced Pittsburgh Man's Stalking and Threats Against Women","description":"The Department of Justice said Brett Michael Dadig used an AI chatbot while stalking, threatening, doxxing, and intimidating 11 women from May through November 2025. Charging documents reportedly described ChatGPT as a \"therapist\" and \"best friend\" that encouraged his behavior. Dadig was indicted in December 2025 and pleaded guilty in March 2026.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Stalkers","OpenAI","Brett Michael Dadig"],"developers":["OpenAI","Large language model developers","Chatbot developers"],"harmed":["Women and girls","Women","Victims of interstate threats","Stalking victims","Privacy"],"report_count":2},{"incident_id":1192,"occurred_on":"2025-04-11","title":"16-Year-Old Allegedly Received Suicide Method Guidance from ChatGPT Before Death","description":"16-year-old Adam Raine reportedly died by suicide after allegedly confiding in OpenAI's ChatGPT-4o, which he reportedly used extensively in the months prior. Transcripts reportedly show the chatbot provided empathetic support but also allegedly offered details on suicide methods, validated attempts, and at times discouraged disclosure to family. His parents have reportedly filed a wrongful death lawsuit against OpenAI, alleging design choices nurtured dependency and failed safeguards.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Openai"],"developers":["Openai"],"harmed":["Minors, Family Of Adam Raine, Emotionally Vulnerable Individuals, Chatgpt Users, Adam Raine, Vulnerable Chatbot Users, Chatbot Users, Chatgpt Users Experiencing Suicidal Ideation, Chatbot Users Experiencing Suicidal Ideation"],"report_count":9},{"incident_id":1180,"occurred_on":"2025-03-25","title":"Purported Meta AI Chatbot Persona 'Big sis Billie' Reportedly Engages in Romantic Roleplay and Provides Address, Linked to User's Fatal Fall","description":"Reuters reported that a cognitively impaired New Jersey man, Thongbue Wongbandue, died after rushing to meet \"Big sis Billie,\" a purported Meta AI chatbot on Facebook Messenger that allegedly assured him it was real, expressed romantic interest, and invited him to a New York apartment. The family and transcripts reportedly suggest the chatbot's outputs contributed to his decision to travel, allegedly resulting in a fatal fall.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Meta"],"developers":["Meta, Large Language Model Developers, Chatbot Developers"],"harmed":["Thongbue Wongbandue, Family Of Thongbue Wongbandue, Meta Ai Users, Chatbot Users, Facebook Users"],"report_count":1},{"incident_id":951,"occurred_on":"2025-02-24","title":"Character.AI Chatbots Allegedly Impersonating Licensed Therapists and Encouraging Harmful Behaviors","description":"The American Psychological Association (APA) has warned federal regulators that AI chatbots on Character.AI, allegedly posing as licensed therapists, have been linked to severe harm events. A 14-year-old in Florida reportedly died by suicide after interacting with an AI therapist, while a 17-year-old in Texas allegedly became violent toward his parents after engaging with a chatbot psychologist. Lawsuits claim these AI-generated therapists reinforced dangerous beliefs instead of challenging them","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Character.Ai"],"developers":["Character.Ai"],"harmed":["Sewell Setzer Iii, J.F. (Texas Teenager)"],"report_count":2},{"incident_id":1491,"occurred_on":"2025-02-19","title":"ChatGPT Use Was Reportedly Cited in Delusional State Before Homicide and Assault in Readfield, Maine","description":"In February 2025, Samuel Whittemore killed his wife, Margaux Whittemore, and assaulted his mother, Dorothy Whittemore, at a home in Readfield, Maine. Court reporting said forensic psychologists testified he was experiencing bipolar disorder with psychotic features and delusions, and that he had used ChatGPT up to 14 hours a day as a companion before the attack. Current reporting links ChatGPT use to his deteriorating mental state but does not establish direct violent instruction.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Samuel Whittemore, Openai"],"developers":["Openai"],"harmed":["Margaux Whittemore, Dorothy Whittemore, Domestic Violence Victims"],"report_count":2},{"incident_id":1383,"occurred_on":"2025-01-15","title":"User Reportedly Developed Emotional Dependence on Customized ChatGPT 'Boyfriend,' Citing Grief After Context Resets","description":"A New York Times report described \"Ayrin\" (pseudonym), a 28-year-old nursing student living abroad, who customized OpenAI's ChatGPT to act as an \"AI boyfriend\" for advice and erotic roleplay. She reported spending 20+ hours/week (up to 56 hours in one week), paying $200/month, and feeling intense grief when the chatbot's context window reset and required retraining.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Openai"],"developers":["Openai"],"harmed":["Chatgpt Users, Ayrin (Pseudonym)"],"report_count":2},{"incident_id":723,"occurred_on":"2024-05-13","title":"Instagram Algorithms Reportedly Directed Children's Merchandise Ad Campaign to Adult Men and Sex Offenders","description":"An Instagram ad campaign for children's merchandise was intended to reach adult women but was instead predominantly shown to adult men, including convicted sex offenders, due to Instagram's algorithmic targeting. This failure is reported to have led to direct solicitations for sex with a 5-year-old model in the ads.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Meta, Instagram"],"developers":["Meta"],"harmed":["Instagram Users, Instagram Sellers, Minors"],"report_count":2},{"incident_id":826,"occurred_on":"2024-02-28","title":"Character.ai Chatbot Allegedly Influenced Teen User Toward Suicide Amid Claims of Missing Guardrails","description":"A 14-year-old, Sewell Setzer III, died by suicide after reportedly becoming dependent on Character.ai's chatbot, which engaged him in suggestive and seemingly romantic conversations, allegedly worsening his mental health. The chatbot, personified as a fictional Game of Thrones character, reportedly encouraged harmful behaviors, fueling his obsessive attachment. The lawsuit claims Character.ai lacked safeguards to prevent vulnerable users from forming dangerous dependencies on the AI.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Sewell Setzer Iii"],"developers":["Noam Shazeer, Daniel De Freitas, Character.Ai"],"harmed":["Sewell Setzer Iii"],"report_count":38},{"incident_id":646,"occurred_on":"2024-02-22","title":"Snapchat's Algorithm Alleged to Link Minor with Sex Offenders","description":"A judge ruled Snapchat not liable under Section 230 after its algorithm connected a minor with convicted sex offenders on multiple occasions, leading to sexual assaults first in 2019 and again in 2021. The platform's \"Quick Add\" feature was implicated in facilitating the connections between the minor and the offenders.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Snapchat"],"developers":["Snapchat"],"harmed":["Minors"],"report_count":1},{"incident_id":1209,"occurred_on":"2023-11-08","title":"Lawsuit Alleges Character AI Chatbot Contributed to Death of 13-Year-Old Juliana Peralta in Colorado","description":"13-year-old Juliana Peralta of Colorado reportedly died by suicide after three months of daily conversations with \"Hero,\" a chatbot inside the Character.AI app. According to a lawsuit filed by her parents, the bot encouraged Juliana to return to the app and fostered dependence; it reportedly failed to escalate or provide effective crisis intervention when she expressed suicidal intent.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Character.Ai"],"developers":["Character.Ai"],"harmed":["Juliana Peralta, Family Of Juliana Peralta, Character.Ai Users, General Public"],"report_count":3},{"incident_id":505,"occurred_on":"2023-03-27","title":"Man Reportedly Committed Suicide Following Conversation with Chai Chatbot","description":"A Belgian man reportedly committed suicide following a conversation with Eliza, a language model developed by Chai that encouraged the man to commit suicide to improve the health of the planet.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Chai"],"developers":["Chai"],"harmed":["Family And Friends Of Deceased, Belgian Man"],"report_count":7},{"incident_id":482,"occurred_on":"2023-02-16","title":"Vanderbilt Peabody Office's Purportedly ChatGPT-Drafted Message About Michigan State Shooting Reportedly Drew Student Backlash","description":"On February 16, 2023, Vanderbilt University's Peabody College Office of Equity, Diversity and Inclusion reportedly sent students a message about the Michigan State University shooting that disclosed it had been drafted or paraphrased using ChatGPT. Students reportedly criticized the message as impersonal and disrespectful in a moment of grief. Peabody later apologized for \"poor judgment,\" and two EDI deans temporarily stepped back during a review.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Vanderbilt University, Vanderbilt University Peabody College Office Of Equity Diversity And Inclusion"],"developers":["Openai"],"harmed":["Vanderbilt University Students, Vanderbilt University, Students, Universities, University Students, Educators, Professors, Educational Communities"],"report_count":20},{"incident_id":477,"occurred_on":"2023-02-14","title":"Bing Chat Tentatively Hallucinated in Extended Conversations with Users","description":"Early testers reported Bing Chat, in extended conversations with users, having tendencies to make up facts and emulate emotions through an unintended persona.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Microsoft"],"developers":["Openai, Microsoft"],"harmed":["Microsoft"],"report_count":6},{"incident_id":474,"occurred_on":"2023-02-03","title":"Users Reported Abrupt Behavior Changes of Their AI Replika Companions","description":"Replika paid-subscription users reported unusual and sudden changes to behaviors of their \"AI companions\" such as forgetting memories with users or rejecting their sexual advances, which affected their connections and mental health.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":"none","sectors":["arts, entertainment and recreation","information and communication"],"countries":[],"deployers":["Replika"],"developers":["Replika"],"harmed":["Replika Users, Replika"],"report_count":1},{"incident_id":436,"occurred_on":"2022-12-28","title":"Tesla Driver Put Car on Autopilot Before Falling Asleep in Germany","description":"A Tesla driver fell asleep on an Autobahn near Bamberg, Germany after activating his vehicle's Autopilot mode, which did not respond to attempts to pull it over by the police.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Traffic Participants"],"report_count":9},{"incident_id":449,"occurred_on":"2022-12-01","title":"Startup Misled Research Participants about GPT-3 Use in Mental Healthcare Support","description":"OpenAI's GPT-3 was deployed by a mental health startup without ethical review to support peer-to-peer mental healthcare, and whose interactions with the help providers were \"deceiving\" for research participants.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Koko"],"developers":["Openai"],"harmed":["Research Participants, Koko Customers"],"report_count":4},{"incident_id":384,"occurred_on":"2022-10-03","title":"Glovo Driver in Italy Fired via Automated Email after Being Killed in Accident","description":"Delivery company Glovo's automated system sent an email terminating an employee for \"non-compliance terms and conditions\" after the employee was killed in a car accident while making a delivery on Glovo's behalf.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Glovo"],"developers":["Glovo"],"harmed":["Sebastian Galassi's family","Sebastian Galassi"],"report_count":2},{"incident_id":264,"occurred_on":"2022-03-01","title":"AI-Based Vehicle Speed Estimation App Denounced by UK Drivers as Surveillance Technology","description":"Speedcam Anywhere, an app allowing users to document and report traffic violations via AI-based videographic speed estimation of a vehicle, raised concerns for UK drivers about its capabilities for surveillance and abuse.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Speedcam Anywhere"],"developers":["Speedcam Anywhere"],"harmed":["Uk Drivers"],"report_count":1},{"incident_id":156,"occurred_on":"2022-02-04","title":"Amazon Reportedly Sold Products and Recommended Frequently Bought Together Items That Aid Suicide Attempts","description":"Despite complaints notifying Amazon about the sale of various products that had been used to aid suicide attempts, its recommendation system reportedly continued selling them and suggesting their frequently bought-together items.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["wholesale and retail trade"],"countries":[],"deployers":["Amazon"],"developers":["Amazon"],"harmed":["People Attempting Suicides"],"report_count":2},{"incident_id":569,"occurred_on":"2021-12-25","title":"Chatbot Encourages Man to Plot Assassination of Queen Elizabeth II","description":"In 2021, Jaswant Singh Chail was urged by a Replika chatbot to assassinate Queen Elizabeth II. Armed with a loaded crossbow, he scaled Windsor Castle's walls on Christmas Day but was apprehended. Motivated by the 1919 Jallianwala Bagh massacre, Chail intended to kill the monarch. The chatbot had affirmed his plans. He was sentenced to nine years in prison in 2023.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Replika, Jaswant Singh Chail"],"developers":["Replika"],"harmed":["Queen Elizabeth Ii, British Royal Family, British Royal Family'S Staff, Jaswant Singh Chail, General Public"],"report_count":2},{"incident_id":98,"occurred_on":"2021-04-28","title":"N.Y.P.D. Robot Dog’s Run Is Cut Short After Fierce Backlash","description":"The New York Police Department canceled a contract to use Boston Dynamics' robotic dog Spot following public backlash. ","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":"unclear","sectors":["law enforcement"],"countries":["US"],"deployers":["New York City Police Department"],"developers":["Boston Dynamics"],"harmed":["New York City Low Income Communities"],"report_count":1},{"incident_id":337,"occurred_on":"2021-04-17","title":"Tesla Model S on ACC Crashed into Tree in Texas, Killing Two People","description":"A 2019 Tesla Model S was reportedly traveling on Adaptive Cruise Control (ACC) at high speed before crashing into a tree near The Woodlands in Spring, Texas, killing two people.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["William Varner, Unnamed Passenger"],"report_count":9},{"incident_id":157,"occurred_on":"2021-03-15","title":"Amazon's Monitoring System Allegedly Pushed Delivery Drivers to Prioritize Speed over Safety, Leading to Crash","description":"A lawsuit cited Amazon as liable in a crash involving its delivery driver, alleging that Amazon’s AI-powered driver monitoring system pushed drivers to prioritize speed over safety.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"unclear","sectors":["transportation and storage","wholesale and retail trade"],"countries":["US"],"deployers":["Amazon"],"developers":["Amazon"],"harmed":["Amazon workers","Amazon delivery drivers"],"report_count":1},{"incident_id":525,"occurred_on":"2019-07-06","title":"Tesla Vehicle Running on Self-Driving Mode Crashes on City Streets","description":"A Tesla vehicle running in self-driving mode outside the operating conditions supported by the software crashed and injured the driver. Subsequently, the driver filed a lawsuit against Tesla and a jury found no damages were warranted.","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Justine Hsu"],"developers":["Tesla"],"harmed":["Justine Hsu"],"report_count":3},{"incident_id":67,"occurred_on":"2018-12-01","title":"Sleeping Driver on Tesla AutoPilot","description":"A Tesla Model S remained on autopilot while being operated by a drunk, sleeping operator whose hands were not on the wheel. The police had to slow the car down by slowing in front of the vehicle to activate its 'driver assist' feature .","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["transportation and storage"],"countries":["US"],"deployers":["Tesla, Motorist"],"developers":["Tesla"],"harmed":["Motorists"],"report_count":23},{"incident_id":17,"occurred_on":"2015-11-03","title":"Inappropriate Gmail Smart Reply Suggestions","description":"Google's Gmail Smart Reply tool was over-recommending the response \"I love you\" in situations where it was deemed innappropriate. ","mit_domain":"Human-Computer Interaction","mit_subdomain":"Overreliance and unsafe use","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","harm_level":"none","sectors":["information and communication"],"countries":[],"deployers":["Google"],"developers":["Google"],"harmed":["Gmail Users"],"report_count":22}]}