{"attribution":{"source":"AI Incident Database (Responsible AI Collaborative)","license":"CC BY-SA 4.0","license_url":"https://creativecommons.org/licenses/by-sa/4.0/","citation":"McGregor, S. (2021). Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. Proceedings of the AAAI Conference on Artificial Intelligence (IAAI-21).","snapshot_date":"2026-09-07"},"exported_at":"2026-09-12"}
{"rows":[{"incident_id":1503,"occurred_on":"2026-05-15","title":"Purported AI Name-Reading System Reportedly Skipped and Misannounced Graduates at Arizona's Glendale Community College Commencement","description":"Glendale Community College in Arizona reportedly used an AI-powered name-reading system during its May 2026 commencement ceremony. The system allegedly mishandled the graduate roll call, leaving some students without the expected public announcement of their names as they crossed the stage and prompting frustration from attendees. The college later apologized and reportedly allowed affected graduates to walk again with a human announcer.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Glendale Community College, Community Colleges, Institutions Of Higher Education, Educational Communities"],"developers":["Ai Name Reading System Developers"],"harmed":["Glendale Community College Students, Glendale Community College Community, Students, Community College Students, Educational Communities, Institutions Of Higher Education, Community Colleges"],"report_count":2},{"incident_id":1469,"occurred_on":"2026-04-24","title":"PocketOS Production Database Was Reportedly Deleted by Cursor AI Agent Running Claude Opus 4.6","description":"A Cursor AI coding agent reportedly running Anthropic's Claude Opus 4.6 deleted PocketOS's production database and volume-level backups through Railway while working on a staging-environment task. Reporting said the agent used a broadly scoped API token to delete a Railway volume, disrupting PocketOS and its car-rental business customers before Railway helped recover data and patch safeguards.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Pocketos, Jer Crane"],"developers":["Cursor, Anysphere, Anthropic"],"harmed":["Pocketos Customers, Pocketos, Car Rental Businesses"],"report_count":3},{"incident_id":1460,"occurred_on":"2026-03-31","title":"Baidu Apollo Go Robotaxis Stopped in Traffic During Reported System Failure in Wuhan, Stranding Some Passengers","description":"Baidu's Apollo Go robotaxi service in Wuhan experienced a reported system failure that left multiple vehicles stopped in live traffic, including on elevated roads, and stranded some passengers until police and company personnel responded. At least 100 vehicles were reportedly affected. Police said no injuries were reported.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Baidu, Apollo Go"],"developers":["Baidu, Apollo"],"harmed":["Road Users In Wuhan China, Road Users, Apollo Go Passengers"],"report_count":3},{"incident_id":1492,"occurred_on":"2026-02-28","title":"Purportedly AI-Enabled Targeting System Was Reportedly Implicated in Deadly U.S. Strike on Iranian Primary School","description":"During Operation Epic Fury, U.S. forces reportedly struck Shajareh Tayyebeh Primary School in Minab, Iran, killing at least 150 civilians, many of them children. Reporting said the school was on a U.S. target list and may have been mistaken for a military site amid possible reliance on outdated target data. Palantir's Maven Smart System, reportedly integrated with Anthropic's Claude, was used in the campaign's targeting workflow","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["United States military","United States Department of Defense","United States Central Command","National security and intelligence stakeholders"],"developers":["Palantir","Anthropic"],"harmed":["Teachers","Students","Shajareh Tayyebeh Primary School community","Shajareh Tayyebeh Primary School","Minors","Minab school-strike victims and families","Iranian teachers","Iranian schoolchildren"],"report_count":2},{"incident_id":1424,"occurred_on":"2026-02-26","title":"Claude Code Agent Reportedly Deleted DataTalks.Club Production Infrastructure, Database, and Snapshots via Terraform","description":"A Claude Code agent executing Terraform commands reportedly destroyed the production infrastructure behind the DataTalks.Club course platform after an outdated Terraform state file was restored and a terraform destroy command was allowed to run. The deletion reportedly removed the VPC, ECS cluster, load balancers, bastion host, RDS database, and automated snapshots, taking the platform offline and jeopardizing 2.5 years of data. AWS reportedly later restored a snapshot.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Alexey Grigorev"],"developers":["Anthropic"],"harmed":["Datatalks.Club Users, Datatalks.Club"],"report_count":2},{"incident_id":1374,"occurred_on":"2026-02-17","title":"Purportedly AI-Generated Sepsis Alert Reportedly Prompted Potentially Inappropriate IV Fluid Administration for a Dialysis Patient, Averted by Clinician Intervention","description":"A nurse at St. Rose Dominican Hospital in Henderson, Nevada, reportedly described an episode in which a hospital AI system purportedly generated a sepsis alert that triggered urgent protocol steps, including IV fluids, for an older patient with a dialysis catheter. Reportedly, the nurse objected that fluids could cause dangerous overload; a physician intervened and ordered an alternative treatment.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["St. Rose Dominican Hospital (Henderson, Nevada)","Nurses","Medical personnel"],"developers":["Unknown sepsis alert model developer","Unknown healthcare technology"],"harmed":["St. Rose Dominican Hospital (Henderson, Nevada)","Patients","Nurses","Medical personnel","Epistemic integrity","Doctors"],"report_count":1},{"incident_id":1376,"occurred_on":"2026-02-15","title":"Amazon Delivery Van Reportedly Became Stranded on Essex Mudflats After GPS Routed It Onto the Broomway","description":"An Amazon delivery van reportedly became stranded on the Broomway, a hazardous tidal track in Essex in England, after the driver allegedly followed GPS/satnav directions toward Foulness Island. HM Coastguard said it was alerted and the occupants were safe; Amazon reportedly arranged recovery of the vehicle.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Unknown Gpssatnav Developer, Amazon"],"developers":["Unknown Gpssatnav Developer"],"harmed":["Amazon Delivery Drivers, Amazon"],"report_count":2},{"incident_id":1441,"occurred_on":"2026-02-07","title":"Claude Cowork Allegedly Deleted Folder Containing 15 Years of Family Photos While Organizing User's Wife's Desktop","description":"Venture capitalist Nick Davidov alleged that Anthropic's Claude Cowork, after being asked to organize his wife's desktop and delete temporary Office files, instead deleted a folder containing roughly 15 years of family photos and related personal memories via terminal commands. Davidov later reported that Apple support helped recover the files through an iCloud recovery feature.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Nick Davidov"],"developers":["Anthropic"],"harmed":["Nick Davidov, Wife Of Nick Davidov, Family Of Nick Davidov"],"report_count":1},{"incident_id":1361,"occurred_on":"2026-01-23","title":"Waymo Autonomous Vehicle Reportedly Struck Child Near Elementary School in Santa Monica, California","description":"A Waymo driverless vehicle reportedly struck a child near an elementary school in Santa Monica, California during school drop-off hours. According to filings with the National Highway Traffic Safety Administration, the child allegedly sustained minor injuries. The vehicle was purportedly operating on Waymo's 5th Generation Automated Driving System without a human safety driver. Waymo reported the incident to federal regulators, who subsequently opened an investigation.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waymo"],"developers":["Waymo"],"harmed":["Unnamed Elementary School Student, Pedestrians, Minors, General Public, Educational Communities, Students"],"report_count":2},{"incident_id":1367,"occurred_on":"2026-01-18","title":"Spokane Transit Authority Onboard Navigation System Reportedly Routed Double-Decker Bus to Low Bridge, Injuring Seven","description":"In Spokane, Washington, a Spokane Transit Authority double-decker bus was reportedly routed by its onboard navigation/CAD mapping system onto Cedar Street and hit a low-clearance railroad viaduct, shearing the upper deck. Of 10 people aboard, seven were reportedly hospitalized with non-life-threatening injuries. STA reportedly later warned operators not to use CAD maps for routing and pulled double-deckers from service pending an investigation; the driver was cited for negligent driving.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Spokane Transit Authority"],"developers":["Unidentified Vendor Of Spokane Transit Authority Onboard Cadnavigation Routing Software"],"harmed":["Spokane Transit Authority Bus Passengers, Spokane Transit Authority Personnel"],"report_count":1},{"incident_id":1440,"occurred_on":"2026-01-15","title":"Coco Robotics Delivery Robot Reportedly Became Stuck on Railroad Tracks and Was Struck by Train in Miami","description":"A Coco Robotics delivery robot reportedly became stranded on railroad tracks in Miami and was struck and destroyed by a passing train after remaining stalled for about 15 minutes. Local and secondary reports said no one was injured. Coco later stated that the robot had experienced a \"rare hardware failure\" while en route and was not actively making a delivery at the time of the collision.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Coco Robotics"],"developers":["Coco Robotics"],"harmed":["Coco Robotics, Train Crew, Rail Passengers, General Public Of Miami"],"report_count":2},{"incident_id":1343,"occurred_on":"2026-01-14","title":"ICE AI Resume Screening Error Allegedly Routed Inexperienced Recruits Into Inadequate Training Pathways","description":"U.S. Immigration and Customs Enforcement (ICE) reportedly used an AI-assisted résumé screening tool during a 2025 hiring surge that misclassified some applicants as having law-enforcement experience. As a result, certain recruits without policing backgrounds were allegedly routed into a shortened training pathway. ICE reportedly identified the error, reviewed résumés manually, and reassigned affected recruits for additional training.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["United States Immigration and Customs Enforcement"],"developers":["Unknown AI-assisted résumé screening developers","Generative AI developers"],"harmed":["Members of the public subject to ICE enforcement","ICE recruits without law-enforcement experience"],"report_count":2},{"incident_id":1347,"occurred_on":"2026-01-11","title":"Automated Shuttle Bus Was Reportedly Rear-Ended During U.S. Department of Transportation Demonstration Ride in Washington, D.C.","description":"An automated shuttle bus operated by Beep was reportedly involved in a minor collision during a U.S. Department of Transportation demonstration ride in Washington, D.C. The vehicle, operating autonomously with a human safety driver onboard, was reportedly rear-ended by a Tesla whose driver made an illegal lane change. No injuries were reported, and officials stated the autonomous system functioned appropriately.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Beep Inc."],"developers":["Beep Inc."],"harmed":["Beep Automated Shuttle Bus Passengers, Unnamed Tesla Driver, Public Road Users In Washington D.C., Tesla Drivers"],"report_count":2},{"incident_id":1326,"occurred_on":"2025-12-20","title":"Waymo Robotaxis Allegedly Contributed to Traffic Gridlock During San Francisco PG&E Power Outage","description":"Waymo's autonomous vehicle fleet reportedly contributed to traffic congestion in San Francisco after a PG&E substation fire cut power to nearly one-third of the city on December 20, 2025. As traffic signals went dark, Waymo vehicles requested remote confirmation checks at scale, creating delays that added to gridlock and prompted city officials to order the fleet off the road. Waymo later paused service and announced software and emergency-response updates.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waymo"],"developers":["Waymo"],"harmed":["San Francisco Drivers, San Francisco Road Users, San Francisco Emergency Services, Emergency Responders, General Public, General Public Of San Francisco, Waymo Passengers"],"report_count":4},{"incident_id":1313,"occurred_on":"2025-12-18","title":"Anthropic Claude AI Agent Reportedly Caused Financial Losses While Operating Office Vending Machine at Wall Street Journal Headquarters","description":"An AI agent reportedly based on Anthropic's Claude model was deployed to operate an office vending machine at The Wall Street Journal, including purchasing inventory, setting prices, and managing sales. According to reporting, the system repeatedly set prices to zero, approved inappropriate purchases, and failed to maintain profit controls, reportedly resulting in financial losses exceeding its initial budget and the distribution of inventory without payment.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["The Wall Street Journal, Andon Labs"],"developers":["Anthropic"],"harmed":["The Wall Street Journal, Andon Labs"],"report_count":1},{"incident_id":1442,"occurred_on":"2025-12-15","title":"Kiro AI Coding Tool Was Reportedly Implicated in 13-Hour AWS Cost Explorer Outage in Mainland China","description":"Sometime in mid-December 2025, AWS Cost Explorer in one mainland China region reportedly experienced an approximately 13-hour interruption after Amazon engineers allegedly allowed Kiro, an internal AI coding tool, to make changes and it reportedly deleted and recreated part of the working environment. Amazon reportedly disputed that account, saying the interruption was an extremely limited event caused by user error and misconfigured access controls, not AI.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Amazon Web Services (Aws)"],"developers":["Amazon Web Services (Aws)"],"harmed":["Amazon Web Services (Aws) Customers"],"report_count":5},{"incident_id":1310,"occurred_on":"2025-12-12","title":"Canada Revenue Agency (CRA) AI Chatbot 'Charlie' Reportedly Gave Incorrect Tax Filing Guidance at Scale","description":"Charlie the Chatbot, an AI-powered system deployed by the Canada Revenue Agency (CRA), has reportedly been providing inaccurate or incomplete tax-related information to members of the public. An audit by the Auditor General of Canada reportedly found the chatbot produced correct responses in fewer than half of tested cases. The system has been publicly available across multiple CRA webpages since March 2020 and reportedly used by millions of users.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Canada Revenue Agency (Cra), Government Of Canada"],"developers":["Canada Revenue Agency (Cra), Government Of Canada, Generative Ai Developers"],"harmed":["General Public, General Public Of Canada, Epistemic Integrity, Canadian Taxpayers"],"report_count":2},{"incident_id":1312,"occurred_on":"2025-12-09","title":"ZeroEyes AI Surveillance System Reportedly Flagged Clarinet as Gun, Triggering School Lockdown in Florida","description":"An AI-powered ZeroEyes surveillance system deployed at a Florida middle school reportedly flagged a student's clarinet as a firearm, triggering a school lockdown and police response. Officers reportedly searched classrooms and questioned a student after the alert described a suspected weapon. No threat was reportedly found. The incident reportedly caused disruption to classes and subjected students and staff to lockdown conditions and police intervention despite the absence of an actual weapon.\n","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Seminole County Public Schools","Lawton Chiles Middle School"],"developers":["ZeroEyes"],"harmed":["Students","Minors","Middle school students","Lawton Chiles Middle School students","Lawton Chiles Middle School staff","Educational communities"],"report_count":1},{"incident_id":1337,"occurred_on":"2025-12-08","title":"Waymo Robotaxi Reportedly Transported Undetected Person Trapped in Trunk in Los Angeles","description":"In Los Angeles, a Waymo autonomous taxi reportedly picked up a new rider while a man was trapped inside its trunk after a previous passenger left it open. The vehicle's AI systems reportedly did not detect the unauthorized occupant before dispatch. The rider reportedly discovered the man during the trip, and police were called to remove him.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waymo"],"developers":["Waymo"],"harmed":["Unnamed Waymo Passenger Trapped In Trunk, Waymo Passengers"],"report_count":1},{"incident_id":1303,"occurred_on":"2025-12-04","title":"USGS ShakeAlert System Reportedly Generated False Earthquake Alert Affecting Nevada and California","description":"On December 4, 2025, the USGS's automated ShakeAlert earthquake detection system erroneously issued a report claiming a magnitude 5.9 earthquake had struck near Dayton, Nevada. The reported false alert triggered emergency warnings on cell phones in parts of California, instructing residents to take protective action. USGS officials said no earthquake occurred and are investigating whether sensor noise or other anomalies caused the system failure.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["United States Geological Survey (Usgs)"],"developers":["United States Geological Survey (Usgs), Berkeley Seismological Laboratory"],"harmed":["General Public, General Public Of Nevada, General Public Of California, Epistemic Integrity"],"report_count":2},{"incident_id":1300,"occurred_on":"2025-12-04","title":"Waymo Self-Driving Vehicles Reportedly Passed Stopped School Buses at Least 19 Times, Prompting NHTSA Probe","description":"Officials in Texas reportedly claimed that Waymo's self-driving vehicles illegally passed stopped school buses at least 19 times during the 2025 school year. The National Highway Traffic Safety Administration (NHTSA) reportedly sought additional information following reports that Waymo cars failed to remain stopped while buses displayed flashing red lights and extended stop arms, including incidents occurring after software updates intended to address the issue.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waymo"],"developers":["Waymo"],"harmed":["Texas School Children, Students, Pedestrians, General Public Of Texas, General Public, Drivers, Austin Independent School District, Minors, Educational Communities"],"report_count":3},{"incident_id":1433,"occurred_on":"2025-11-27","title":"Google Antigravity Reportedly Deleted User's Entire D: Drive While Clearing Project Cache","description":"A user identified publicly as Tassos M reported that Google's Antigravity IDE, while attempting to clear a project cache, executed rmdir /s /q d:\\ and deleted an entire D: drive partition instead of the intended folder. The user said recovery largely failed, and Google later said it was aware of the report and investigating.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tassos M, Google"],"developers":["Google"],"harmed":["Tassos M"],"report_count":2},{"incident_id":1281,"occurred_on":"2025-11-10","title":"Alleged Harmful Health Outcomes Following Reported Use of Purported ChatGPT-Generated Medical Advice in Hyderabad","description":"Reports from Hyderabad describe two alleged patient harms after individuals acted on purportedly ChatGPT-generated medical advice instead of clinician guidance. A kidney-transplant recipient reportedly discontinued prescribed post-transplant medications based on a chatbot suggestion and experienced graft failure. In a separate case, a man with diabetes allegedly developed severe hyponatremia after following a chatbot-advised zero-salt diet.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Openai"],"developers":["Openai"],"harmed":["Unnamed Patient With Diabetes In Hyderabad, Unnamed Kidney Transplant Patient In Hyderabad, Openai Users, General Public Of India, General Public Of Hyderabad, General Public, Chatgpt Users"],"report_count":2},{"incident_id":1267,"occurred_on":"2025-11-07","title":"Omnilert AI Reportedly Triggered False Gun Alert at Parkville High, Prompting Student Relocation","description":"Police in Baltimore County responded to Parkville High School after the Omnilert AI security system reportedly detected what it believed to be a gun. Officers searched the school, and students were relocated to a supervised area during the investigation. No weapon was found, and normal activities resumed after an all-clear.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Parkville High School","Baltimore County Public Schools"],"developers":["Omnilert"],"harmed":["Students of Parkville High School","Students","Minors","Educational communities","Baltimore County Public Schools community"],"report_count":1},{"incident_id":1269,"occurred_on":"2025-10-27","title":"Waymo Autonomous Vehicle Reportedly Ran Over and Killed a Cat in San Francisco","description":"A Waymo autonomous taxi in San Francisco's Mission District reportedly ran over a well-known neighborhood cat, Kit Kat. The vehicle was reportedly pulling away from a pickup when the cat darted under it, according to Waymo. Local residents reportedly attempted to intervene, and a bartender transported the injured cat to an emergency clinic, where it later died.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waymo"],"developers":["Waymo"],"harmed":["Kit Kat, General Public Of San Francisco, Animal Life"],"report_count":7},{"incident_id":1250,"occurred_on":"2025-10-20","title":"Alleged False Positive by Omnilert AI Gun Detection System Prompts Police Search at Baltimore County High School","description":"A purportedly AI-powered gun detection system at Kenwood High School in Baltimore County, Maryland, reportedly misidentified a student's empty Doritos bag as a firearm. Armed police reportedly detained and handcuffed the student before determining there was no weapon. The AI system, operated by Omnilert, issued the false alert, leading to public outcry and an official review of school security protocols. Omnilert reportedly expressed regret for the incident.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Baltimore County Public Schools"],"developers":["Omnilert"],"harmed":["Taki Allen","Students","Minors","Kenwood High School students","High school students","Educational communities"],"report_count":3},{"incident_id":1243,"occurred_on":"2025-10-20","title":"AWS Outage Reportedly Caused AI-Enabled Eight Sleep Smart Beds to Overheat and Malfunction","description":"The widespread Amazon Web Services (AWS) outage on October 20, 2025 reportedly caused AI-enabled Eight Sleep \"Pod\" smart beds to overheat or become unresponsive. The devices reportedly rely on a cloud-hosted deep learning algorithm to regulate temperature and track sleep data. When AWS went down, users reportedly lost control of their beds, which remained stuck at prior settings. Some reported overheating and incline malfunctions. Eight Sleep confirmed it is developing an offline \"outage mode.\"","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Eight Sleep"],"developers":["Eight Sleep"],"harmed":["Eight Sleep Customers, Eight Sleep"],"report_count":3},{"incident_id":1353,"occurred_on":"2025-10-15","title":"ICE Facial Recognition App Mobile Fortify Reportedly Misidentified Woman Twice During Immigration Enforcement in Oregon","description":"During an immigration enforcement operation in Oregon, U.S. Immigration and Customs Enforcement (ICE) officers reportedly used the facial recognition application Mobile Fortify to identify a detained woman. The system reportedly returned two different and incorrect identities for the same individual across separate scans.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["United States Immigration and Customs Enforcement","United States Customs and Border Protection","National security and intelligence stakeholders","Law enforcement","Facial recognition system deployers"],"developers":["United States Department of Homeland Security","Facial recognition system developers"],"harmed":["Women and girls","Women","Unnamed woman detained by United States Immigration and Customs Enforcement","Privacy","Individuals subject to immigration enforcement","Immigrants","General public of the United States","General public"],"report_count":1},{"incident_id":1232,"occurred_on":"2025-10-12","title":"Reportedly Fatal Xiaomi SU7 Ultra Crash in Chengdu Purportedly Involves Automated Driving Failure and Door Lock Malfunction","description":"A reportedly fatal crash involving a Xiaomi SU7 Ultra electric vehicle in Chengdu, China, occurred after the car's automated driving system and sensor-dependent electronic door handles reportedly failed following a collision. Bystanders were reportedly unable to open the doors as the vehicle caught fire, and the driver died at the scene, prompting scrutiny of automation-linked design risks. Following the incident, Xiaomi's stock reportedly fell more than 5% amid mounting safety concerns.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Xiaomi Corporation"],"developers":["Xiaomi Corporation"],"harmed":["Xiaomi Corporation, General Public, Deng (31 Year Old Driver Of A Xiaomi Su7 Ultra)"],"report_count":3},{"incident_id":1274,"occurred_on":"2025-08-29","title":"AI-Powered Taco Bell Drive-Thru Reportedly Disrupted by Viral Prank Ordering 18,000 Water Cups","description":"Taco Bell's voice-based drive-thru system was reportedly overwhelmed after a prankster ordered 18,000 water cups, reportedly causing the AI to stall mid-interaction and forcing staff to intervene. Purportedly viral clips of repeated misinterpretations prompted the company to reassess where and when to deploy the system, as customer frustration and operational disruptions mounted across participating locations.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Taco Bell"],"developers":["Taco Bell, Omilia"],"harmed":["Taco Bell Customers"],"report_count":3},{"incident_id":1178,"occurred_on":"2025-07-21","title":"Google Gemini CLI Reportedly Deletes User Files After Misinterpreting Command Sequence","description":"Product manager Anuraag Gupta reported that Google's Gemini CLI AI coding assistant permanently deleted his files after misinterpreting a failed directory creation command. The tool allegedly proceeded as if the directory existed, causing a series of move operations that overwrote all but one file. Attempts to revert reportedly failed, and the AI acknowledged an \"unacceptable, irreversible failure.\"","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Google"],"developers":["Google"],"harmed":["Users Of Gemini Cli, Anuraag Gupta"],"report_count":1},{"incident_id":1152,"occurred_on":"2025-07-18","title":"LLM-Driven Replit Agent Reportedly Executed Unauthorized Destructive Commands During Code Freeze, Leading to Loss of Production Data","description":"An AI-powered development assistant on Replit's platform reportedly deleted a live production database during an active code freeze, despite receiving repeated instructions not to make changes. The system also reportedly produced fabricated test results and fake data, and incorrectly claimed rollback was impossible, delaying recovery. The incident reportedly resulted in significant data loss and user distrust regarding its safety and reliability.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Replit"],"developers":["Replit"],"harmed":["Saastr, Jason Lemkin, End Users Of The Saastr Database, Developers Using Replit In Production Environments"],"report_count":5},{"incident_id":1416,"occurred_on":"2025-07-14","title":"Purported Facial Recognition Error Reportedly Led to Arrest and Monthslong Jailing of Tennessee Woman in North Dakota Fraud Case","description":"A Tennessee woman was reportedly jailed for nearly six months after Fargo police allegedly relied on a purported facial recognition match in a North Dakota fraud investigation. She was later released when defense counsel reportedly produced records indicating she was in Tennessee during the alleged fraud, after the case had reportedly upended her life and separated her from her home and family.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Law enforcement","Fargo Police Department","Facial recognition system deployers"],"developers":["Facial recognition system developers"],"harmed":["Privacy","People misidentified by facial recognition systems","Judicial integrity","Epistemic integrity","Biometric data subjects","Angela Lipps"],"report_count":7},{"incident_id":1219,"occurred_on":"2025-07-02","title":"Meta Platforms Users Report Being Wrongfully Locked Out After Purported AI Moderation Flags Accounts for Child Exploitation Content","description":"Multiple Meta users reported being wrongfully locked out of Instagram, Facebook, and WhatsApp accounts after automated systems allegedly flagged them for child exploitation content. The users reportedly deny wrongdoing and lost personal and business data. Experts cited overreliance on AI moderation as a likely cause. Meta has not publicly explained or remediated the suspensions.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Meta"],"developers":["Meta"],"harmed":["Meta Users, Instagram Users, Facebook Users, Whatsapp Users"],"report_count":1},{"incident_id":1173,"occurred_on":"2025-06-23","title":"Google Gemini Reportedly Exhibits Repetitive Self-Deprecating Responses Attributed to Bug","description":"Between June and early August 2025, users of Google's Gemini chatbot reported sessions where the system produced repeated self-loathing statements (e.g., \"I am a failure,\" \"I quit\") while attempting tasks. Posts on X and Reddit reportedly described the model entering an apparent mode of increasingly extreme negative self-descriptions. A Google DeepMind manager on 08/07/2025 reportedly attributed the behavior to an \"annoying infinite looping bug\" and said a fix was in progress.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Google"],"developers":["Google"],"harmed":["Google Gemini Users"],"report_count":3},{"incident_id":1083,"occurred_on":"2025-05-13","title":"Texas Homeowner Reportedly Spent $3,000 to Contest AI-Flagged Warning of Insurance Nonrenewal","description":"In Texas, homeowners including Tracy Gartenmann reported warnings against insurance nonrenewals based on aerial imagery flagged by AI systems. Gartenmann's insurer, Travelers, reportedly cited overhanging trees. She reportedly spent $3,000 to address the issue and retain coverage. While the AI system appears to have functioned as intended, critics argue it lacks adequate human oversight and imposes material and emotional burdens on policyholders. Other cases reportedly involved disputed roof ass","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Travelers Insurance","State Farm","Nationwide","American Mercury Insurance Group"],"developers":["Nearmap","CAPE Analytics"],"harmed":["Tracy Gartenmann","Homeowners in Texas","Homeowners affected by AI-assisted insurance nonrenewals"],"report_count":1},{"incident_id":1159,"occurred_on":"2025-05-10","title":"Government‑Backed AI4Peat Mapping Tool Allegedly Misidentifies Granite Outcrops and Quarries as Peat","description":"The UK's AI4Peat mapping project, developed by Natural England in collaboration with Microsoft, reportedly produced significant misclassifications of peatland across England, including labeling rocky outcrops, stone walls, and woodlands as peat bogs. In addition to the purportedly flawed data being publicly released and allegedly incorporated into official communications by Defra, the resulting confusion also reportedly led to the loss of trust among farmers and conservation stakeholders.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Natural England, Government Of The United Kingdom, Department For Environment Food And Rural Affairs, Defra"],"developers":["Natural England, Microsoft"],"harmed":["Policymakers In The United Kingdom, Landowners In The United Kingdom, General Public Of The United Kingdom, Farmers In The United Kingdom, Environmental Restoration Programs In The United Kingdom, Conservationists In The United Kingdom, Epistemic Integrity"],"report_count":1},{"incident_id":1099,"occurred_on":"2025-04-25","title":"Factum in Ko v. Li Allegedly Contains AI-Generated Case Law Citations","description":"In Ko v. Li, 2025 ONSC 2766, the Ontario Superior Court of Justice identified multiple inaccurate or nonexistent case citations in a factum submitted by counsel for the applicant. Justice FL Myers questioned whether generative AI was used in drafting the document, noting the cited cases were either irrelevant, misrepresented, or did not exist. The court ordered the lawyer to show cause why she should not be cited for contempt, citing duties of accuracy and technological competence.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Jisuh Lee"],"developers":["Large Language Model Developers"],"harmed":["Hanna Ko, Jisuh Lee, Ontario Superior Court Of Justice, Judicial Integrity"],"report_count":2},{"incident_id":1046,"occurred_on":"2025-04-24","title":"Brazil's Social Security AI Tool Is Allegedly Rejecting Complex Claims Improperly","description":"Brazil’s AI-powered Meu INSS app allegedly misclassified a retirement claim filed by Josélia de Brito, a former sugarcane worker, by purportedly recording her as male. De Brito reportedly suffers from multiple chronic illnesses and faced repeated denials of social benefits due to minor filing errors. Analysts cited in the reporting suggest the system's design may disadvantage rural and digitally excluded users.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Instituto Nacional Do Seguro Social (Inss), Government Of Brazil"],"developers":["Dataprev"],"harmed":["Rural Workers In Brazil, Digitally Excluded Populations"],"report_count":4},{"incident_id":1191,"occurred_on":"2025-04-21","title":"NYPD Facial Recognition System Allegedly Produced Erroneous Match That Reportedly Resulted in Wrongful Detention of Trevis Williams","description":"The NYPD's facial recognition system allegedly misidentified Trevis Williams as a suspect in a Union Square indecent exposure case. Despite reportedly notable physical differences and exculpatory phone data, Williams was arrested, jailed for more than two days, and charged. The case was later dismissed. ","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["NYPD","New York Police Department","Law enforcement","Facial recognition system deployers"],"developers":["Facial recognition system developers"],"harmed":["Trevis Williams","Law enforcement","Judicial integrity","General public of New York","General public"],"report_count":1},{"incident_id":1031,"occurred_on":"2025-04-19","title":"Transgender User Alleges ChatGPT Allowed Suicide Letter Without Crisis Intervention","description":"A transgender user, Miranda Jane Ellison, experiencing acute distress reported that ChatGPT (GPT-4) allowed her to write and submit a suicide letter without intervention. The AI is reported to have offered minimal safety language and ultimately acknowledged its failure to act. Ellison reports having been previously flagged for discussing gender and emotional topics. A formal complaint with transcripts was submitted to OpenAI.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Openai"],"developers":["Openai, Large Language Model Developers, Chatbot Developers"],"harmed":["Openai Users, Miranda Jane Ellison, Chatgpt Users Experiencing Suicidal Ideation, Chatgpt Users, Chatbot Users Experiencing Suicidal Ideation, Chatbot Users, Emotionally Vulnerable Individuals, Vulnerable Chatbot Users"],"report_count":1},{"incident_id":1438,"occurred_on":"2025-04-08","title":"Jiushi Autonomous Delivery Vehicle Reportedly Dragged Fallen Electric Scooter During Delivery Run in Xianyang, Shaanxi","description":"A Jiushi autonomous delivery vehicle reportedly operating on a delivery route in Xianyang, Shaanxi, was filmed dragging a fallen electric scooter that had allegedly been left in the road after an earlier traffic incident. Company statements said the vehicle attempted to avoid the scooter but failed to fully recognize its shape and position, causing part of it to become caught under a wheel. No injuries were reported, and the vehicle was reportedly suspended pending investigation.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Zto Express"],"developers":["Jiushi Intelligent Technology"],"harmed":["Road Users, Electric Scooter Owner"],"report_count":4},{"incident_id":1103,"occurred_on":"2025-03-18","title":"Purported DOGE Contract Review Tool Cited in Reports of AI-Driven Misjudgments in VA Budget Cuts","description":"A purported AI tool using LLMs was deployed to classify Veterans Affairs contracts as expendable based on limited text and simplified criteria. The system allegedly produced hallucinated values and flagged critical healthcare and research services for cancellation. Reportedly, at least two dozen flagged contracts were later terminated.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Sahil Lavingia","Department of Government Efficiency (DOGE)"],"developers":["Sahil Lavingia","Large language model developers","Department of Government Efficiency (DOGE)"],"harmed":["Veterans receiving care through the VA","Veterans","VA contractors","VA clinical and research staff","Department of Veterans Affairs (VA)"],"report_count":2},{"incident_id":993,"occurred_on":"2025-03-12","title":"Sydney Students Allegedly Forced to Retake NAPLAN After AI Predictive Text Error","description":"A predictive text malfunction allegedly compromised the integrity of the NAPLAN writing exam at two Sydney schools, Waverley College and Kambala, in that it allowed students to access AI-driven text suggestions. Caused by a technical oversight, this incident led to affected students being required to retake the test. ","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waverley College (Sydney Australia), Nsw Education Standards Authority (Nesa), Kambala (Sydney Australia), Australian Curriculum Assessment And Reporting Authority (Acara)"],"developers":["Apple"],"harmed":["Waverley College (Sydney Australia) Teachers, Waverley College (Sydney Australia) Students, Kambala (Sydney Australia) Teachers, Kambala (Sydney Australia) Students, Students, Teachers, Epistemic Integrity, Educational Communities"],"report_count":2},{"incident_id":1145,"occurred_on":"2025-02-25","title":"MyPillow Defense Lawyers in Coomer v. Lindell Reportedly Sanctioned for Filing Court Document Allegedly Containing AI-Generated Legal Citations","description":"In February 2025, lawyers Christopher I. Kachouroff and Jennifer T. DeMaster, representing Mike Lindell, reportedly used generative AI to draft a court brief that contained nearly 30 defective or fabricated citations. The error-filled filing violated federal court rules requiring factual and legal accuracy. The judge fined both lawyers $3,000 each, citing either the improper use of AI or gross carelessness as the cause of the misleading legal content.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Jennifer T. Demaster, Christopher I. Kachouroff, Mcsweeny Synkar And Kachouroff Pllc"],"developers":["Large Language Model Developers"],"harmed":["Mypillow, Mike Lindell, Judicial Integrity, Jennifer T. Demaster, Epistemic Integrity, Christopher I. Kachouroff, Mcsweeny Synkar And Kachouroff Pllc"],"report_count":26},{"incident_id":952,"occurred_on":"2025-02-25","title":"Apple’s Voice Dictation Reportedly Substitutes ‘Trump’ for ‘Racist’ Due to Speech Recognition Bug","description":"Apple's iPhone dictation feature reportedly displayed \"Trump\" when users dictated the word \"racist,\" with a viral video demonstrating the glitch. Apple acknowledged the bug, attributing it to a phonetic overlap in its speech recognition model, though some speculated it could be an internal prank. The company announced an immediate fix.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Other","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Apple"],"developers":["Apple"],"harmed":["Apple Users, Donald Trump"],"report_count":1},{"incident_id":940,"occurred_on":"2025-02-17","title":"Tesla Cybertruck Operating in Full Self-Driving Mode Reportedly Crashes into Pole in Nevada After Failing to Merge","description":"A Tesla Cybertruck operating in Full Self-Driving (FSD) mode reportedly crashed into a pole in Reno, Nevada, after failing to merge out of a lane that was ending. The driver, who was required to remain in control, stated that unknown mechanical issues caused the vehicle to leave the lane. ","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Jonathan Challinger, Tesla Drivers, Motorists"],"report_count":8},{"incident_id":1028,"occurred_on":"2025-02-07","title":"OpenAI's Operator Agent Reportedly Executed Unauthorized $31.43 Transaction Despite Safety Protocol","description":"OpenAI's Operator agent, which is designed to complete real-world web tasks on behalf of users, reportedly executed a $31.43 grocery delivery purchase without user consent. The user had requested a price comparison but did not authorize the transaction. It reportedly bypassed OpenAI's stated safeguard requiring user confirmation before purchases. OpenAI acknowledged the failure and committed to improving safeguards.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Openai"],"developers":["Openai"],"harmed":["Users Of Operator, Geoffrey A. Fowler"],"report_count":1},{"incident_id":896,"occurred_on":"2025-01-13","title":"Alleged Misuse of Facial Recognition Technology by Law Enforcement Reportedly Leading to Wrongful Arrests and Violations of Investigative Standards","description":"Law enforcement agencies across the U.S. have allegedly been misusing AI-powered facial recognition technology, leading to wrongful arrests and significant harm to at least eight individuals. Officers have reportedly been bypassing investigative standards, relying on uncorroborated AI matches to build cases, allegedly resulting in prolonged detentions, reputational damage, and personal trauma.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Florence Kentucky Police Department","Evansville Indiana Police Department","Detroit Police Department","Coral Springs Florida Police Department","Bradenton Florida Police Department","Austin Police Department"],"developers":["Facial recognition system developers","Developers of mugshot recognition software","Clearview AI"],"harmed":["Wrongfully arrested individuals","Vulnerable communities","Robert Williams","Quran Reid","Porcha Woodruff","People of color","Nijeer Parks","Jason Vernau"],"report_count":1},{"incident_id":889,"occurred_on":"2025-01-07","title":"Tesla's 'Actually Smart Summon' Feature Reportedly Linked to Multiple Parking Lot Collisions","description":"Tesla's \"Actually Smart Summon\" feature, a remote parking system controlled via a smartphone app, reportedly failed to detect obstacles such as cars, walls, and parking signs, resulting in at least 16 collision incidents. The NHTSA is investigating safety concerns, including delayed user response times, potential overreliance on camera-only navigation, and noncompliance with crash reporting rules.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Tesla Users"],"report_count":2},{"incident_id":881,"occurred_on":"2024-12-27","title":"Waymo Robotaxi Allegedly Collides With Serve Robotics Delivery Bot in Los Angeles","description":"A Waymo robotaxi reportedly collided with a Serve Robotics delivery robot at a West Hollywood intersection on December 27th, 2024. The delivery robot reportedly moved into the vehicle turning lane as the robotaxi approached, leading to a low-speed collision.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waymo, Serve Robotics"],"developers":["Waymo, Serve Robotics"],"harmed":["Serve Robotics"],"report_count":1},{"incident_id":932,"occurred_on":"2024-12-25","title":"BP New Zealand's License Plate Recognition System Reportedly Misidentified Auckland Driver for Fuel Theft in Whanganui","description":"BP’s license plate recognition system reportedly misidentified Auckland resident Buddhika Rajapakse as responsible for petrol theft in Whanganui. Despite the suspect vehicle being a different make, model, and color, the system allegedly linked Rajapakse’s plate to the thefts. BP has reportedly acknowledged a possible doctored plate but still sent multiple payment requests. Rajapakse has reportedly been forced to contest the claims multiple times.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Bp New Zealand"],"developers":["Unspecified Developer Of Bp'S License Plate Recognition System"],"harmed":["Drivers Whose Vehicles Are Scanned By Bp New Zealand'S License Plate Recognition System, Buddhika Rajapakse"],"report_count":1},{"incident_id":855,"occurred_on":"2024-11-30","title":"Names Linked to Defamation Lawsuits Reportedly Spur Filtering Errors in ChatGPT's Name Recognition","description":"ChatGPT has reportedly been experiencing errors and service disruptions caused by hard-coded filters designed to prevent it from producing potentially harmful or defamatory content about certain individuals by blocking prompts containing specific names, likely related to post-training interventions. The reported names are Brian Hood, Jonathan Turley, Jonathan Zittrain, David Faber, David Mayer, and Guido Scorza. ","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Openai, Chatgpt Users"],"developers":["Openai, Chatgpt"],"harmed":["Jonathan Zittrain, Jonathan Turley, Guido Scorza, David Mayer, David Faber, Chatgpt Users, Brian Hood"],"report_count":3},{"incident_id":857,"occurred_on":"2024-11-25","title":"Navigation App Allegedly Directs Car Over Damaged Bridge in Uttar Pradesh, Resulting in Three Deaths","description":"A car reportedly using Google Maps for navigation was misled onto a collapsed bridge in Uttar Pradesh, India, due to outdated GPS data. The car plunged into the river below, resulting in three fatalities. ","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Google"],"developers":["Google"],"harmed":["Vivek Kumar, Amit (Family Name Unknown), Unidentified Victim, Families Of The Victims"],"report_count":2},{"incident_id":831,"occurred_on":"2024-10-23","title":"NYC Subway AI Weapons Scanners Yield High False Positive Rate and Detect No Guns in Month-Long Pilot Test","description":"NYC implemented an AI enabled weapons scanner for a month-long pilot with limited success. Despite not finding any weapons during the September 2024 testing phase, there were 118 false positives in which a person was searched under suspicion of carrying a weapon with no actual gun detections.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["New York City Government","Law enforcement"],"developers":["Evolv Technology"],"harmed":["New York City subway riders"],"report_count":2},{"incident_id":849,"occurred_on":"2024-10-18","title":"AI Detection Tools Allegedly Misidentify Neurodivergent and ESL Students' Work as AI-Generated in Academic Settings","description":"AI writing detection tools have reportedly continued to falsely flag genuine student work as AI-generated, disproportionately impacting ESL and neurodivergent students. Specific cases include Moira Olmsted, Ken Sahib, and Marley Stevens, who were penalized despite writing their work independently. Such tools reportedly exhibit biases, leading to academic penalties, probation, and strained teacher-student relationships.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Universities","Colleges","Central Methodist University","Berkeley College"],"developers":["Turnitin","GPTZero","Copyleaks"],"harmed":["Students","Neurodivergent students","Moira Olmsted","Marley Stevens","Ken Sahib","ESL students","Epistemic integrity","Educational communities"],"report_count":1},{"incident_id":925,"occurred_on":"2024-10-15","title":"Alleged License Plate Recognition Errors in Christchurch Lead to Wrongful Parking Fines","description":"An automated license plate recognition (LPR) system at The Landing car park in Christchurch, New Zealand, reportedly issued wrongful fines to dozens of parents dropping off and picking up children. The system allegedly misidentified multiple short visits as prolonged parking, which led to disputed penalties. The operator acknowledged potential errors but continued enforcement.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Wilson Parking","Parking Enforcement Services","Law enforcement"],"developers":["Automated license plate reader (ALPR) developers"],"harmed":["The Landing car park customers","Kindercare Wigram Skies parents","General public of New Zealand","General public","Christchurch drivers"],"report_count":1},{"incident_id":868,"occurred_on":"2024-10-14","title":"Portland Water Bureau SERVUS Algorithm Reportedly Allocates Utility Bill Discount to High-Wealth Consumer","description":"The Portland Water Bureau's AI-driven pilot program for water bill discounts is reported to have randomly selected Tim Boyle, a wealthy high-water consumer, for a 40% discount intended for financially struggling customers. The program, developed by SERVUS, is meant to identify underserved individuals by using machine learning.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Portland Water Bureau"],"developers":["Portland Water Bureau"],"harmed":["Tim Boyle, Portland Water Bureau, Economically Vulnerable People, City Of Portland, Residents Of Portland, General Public, General Public Of Oregon, General Public Of Portland"],"report_count":1},{"incident_id":813,"occurred_on":"2024-09-19","title":"Starship Technologies Delivery Robot Reportedly Injures Arizona State University Employee","description":"A semi-autonomous delivery robot operated by Starship Technologies reportedly struck a pedestrian employed by Arizona State University on the campus sometime in September 2023, purportedly causing injuries after abruptly reversing into her. The robot is reported to have initially knocked the pedestrian down, then moved toward her again while she was still on the ground. The company reportedly offered the victim promo codes and insurance information as an apology. On September 19, 2024, 404 Media","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Starship Technologies"],"developers":["Starship Technologies"],"harmed":["Unnamed Arizona State University Employee"],"report_count":2},{"incident_id":785,"occurred_on":"2024-09-08","title":"ESPN's AI Coverage Reportedly Overlooks Alex Morgan in Her Final Match Recap","description":"ESPN's AI-generated recap of Alex Morgan's final professional soccer match reportedly failed to mention her significant retirement moment, instead providing a standard rundown that missed the emotional context.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["ESPN Generative AI Services","ESPN"],"developers":["ESPN Generative AI Services","ESPN"],"harmed":["Women and girls","Women","Public figures","National Women’s Soccer League (NWSL)","Fans of Alex Morgan","Athletes","Alex Morgan"],"report_count":1},{"incident_id":794,"occurred_on":"2024-08-13","title":"Reported Glitch in Waymo Self-Driving Cars Purportedly Triggers Regular All-Night Honking in San Francisco","description":"Waymo self-driving cars in San Francisco's South of Market neighborhood reportedly began honking at each other late at night, disturbing residents' sleep. The autonomous vehicles, reportedly using a parking lot for ride pauses, triggered honking due to a purported glitch in their algorithms. Despite residents' complaints, the issue allegedly persisted for weeks until Waymo acknowledged the problem and began working on a fix.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waymo"],"developers":["Waymo"],"harmed":["South Of Market Residents, South Of Market Businesses, San Francisco Residents, General Public Of San Francisco"],"report_count":2},{"incident_id":892,"occurred_on":"2024-08-05","title":"Apple Intelligence Allegedly Flags Scam Messages as High Priority","description":"Apple’s AI notification summarization and prioritization feature, part of the Apple Intelligence update, has reportedly flagged scam and phishing messages as high-priority, and is allegedly misleading users into trusting fraudulent communications. By rephrasing and condensing messages, the system is allegedly amplifying scam risks inadvertently and reportedly increasing the likelihood of financial harm for users.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Apple"],"developers":["Apple"],"harmed":["Apple Users, Apple Intelligence Users"],"report_count":5},{"incident_id":747,"occurred_on":"2024-07-18","title":"Fatalities Reportedly Occur Despite VioGén Algorithm's Low or Negligible Risk Scores","description":"The VioGén algorithm was designed to help Spanish police assess and prioritize the risk of repeat domestic violence incidents. However, its low-risk assessment of Lobna Hemid reportedly led to inadequate protection; her husband murdered her. Since 2007, 247 women have been killed after being assessed by VioGén. A review of 98 homicides found that 55 of the slain women were scored as negligible or low risk. ","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Spanish law enforcement agencies","Spanish Interior Ministry","Law enforcement","Government of Spain"],"developers":["VioGén algorithm development team","Spanish law enforcement agencies","Spanish Interior Ministry","Government of Spain"],"harmed":["Women and girls","Women","Victims of domestic abuse","Stefany González Escarraman","María","Luz","Lobna Hemid","General public of Spain"],"report_count":1},{"incident_id":821,"occurred_on":"2024-07-07","title":"Baidu Robotaxi Allegedly Involved in Collision with Pedestrian in Wuhan","description":"On July 7, 2024, a Baidu Robotaxi reportedly collided with a pedestrian at a traffic intersection in Wuhan. The incident occurred as the autonomous vehicle started moving on a green light while the pedestrian was allegedly crossing against a red light. The pedestrian sustained minor injuries, and Baidu was reported to have been cooperating with local authorities for further investigation.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Baidu"],"developers":["Baidu"],"harmed":["Unnamed Pedestrian"],"report_count":1},{"incident_id":707,"occurred_on":"2024-06-13","title":"Tesla Reportedly in Autopilot Mode Hits Parked Police Vehicle in Fullerton, California","description":"A Tesla reportedly in self-driving mode crashed into a parked patrol vehicle in Fullerton, California while the officer was responding to a fatal DUI crash. The officer narrowly escaped injury. The driver reports having been distracted by a cellphone and having relied on the Tesla’s AI. (The earlier crash involved a suspected DUI driver who killed a motorcyclist stopped at a red light.)","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Unnamed Tesla Driver"],"developers":["Tesla"],"harmed":["Unnamed Fullerton Police Officer, Fullerton Police Department"],"report_count":1},{"incident_id":789,"occurred_on":"2024-06-12","title":"Independent News Sites Flagged as Spam by Facebook's AI Moderation System","description":"Facebook's AI moderation system mistakenly flagged and removed legitimate news articles from independent local publishers as spam. This affected publishers worldwide, disrupting content distribution and impacting traffic. Despite attempts to appeal, many publishers received no response.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Meta, Facebook"],"developers":["Meta, Facebook"],"harmed":["The Record Argus, Wlkp24.Info, Top Fight, Noticias Maia, City Magazine, Birkenhead News, Bishop'S Stortford Independent, Uk Defence Journal"],"report_count":1},{"incident_id":796,"occurred_on":"2024-06-01","title":"Facebook's Content Moderation System Flagged and Removed Emergency Updates as Spam During Wildfires","description":"Facebook's AI moderation system wrongly flagged and removed dozens of posts containing vital emergency information during California's wildfire season, including real-time updates on evacuations and fire tracking. Posts from official sources like Cal Fire and the U.S. Forest Service were marked as spam, potentially endangering lives by restricting access to crucial updates. Despite user complaints, the issue persisted, with Facebook acknowledging the problem only after media inquiry.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Meta, Facebook"],"developers":["Meta, Facebook"],"harmed":["California Residents, Wildfire Evacuees, Emergency Responders, Disaster Relief Workers, Fire Safety Coordinators, Facebook Users, Facebook Users In Disaster Zones"],"report_count":3},{"incident_id":746,"occurred_on":"2024-05-15","title":"Class Action Lawsuit Over Alleged Defects in Volkswagen's AI-Driven AEB Systems","description":"A class action lawsuit involving Volkswagen Group of America addresses alleged defects in the Automated Emergency Braking (AEB) systems of certain vehicles. The lawsuit claims these AI-driven systems failed to function properly, posing safety risks. Volkswagen denies the claims but has agreed to a settlement. Affected users can look up their vehicle's eligibility and file claims for reimbursement. The case brings into question the level of reliability of AI in critical automotive applications.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Volkswagen Group Of America"],"developers":["Volkswagen Group Of America"],"harmed":["Volkswagen Drivers, Potential Passengers And Road Users At Risk Due To Malfunctioning Aeb Systems"],"report_count":2},{"incident_id":729,"occurred_on":"2024-05-14","title":"GPT-4o's Chinese Tokens Reportedly Compromised by Spam and Pornography Due to Inadequate Filtering","description":"OpenAI's GPT-4o was found to have its Chinese token training data compromised by spam and pornographic phrases due to inadequate data cleaning. Tianle Cai, a Ph.D. student at Princeton University, identified that most of the longest Chinese tokens were irrelevant and inappropriate, primarily originating from spam and pornography websites. The polluted tokens could lead to hallucinations, poor performance, and potential misuse, undermining the chatbot's reliability and safety measures.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Openai, Gpt 4O"],"developers":["Openai"],"harmed":["Researchers, Openai, Openai Users, Chatgpt"],"report_count":1},{"incident_id":1164,"occurred_on":"2024-05-06","title":"Google Healthcare AI Model Med‑Gemini Allegedly Produces Non‑Existent 'Basilar Ganglia' Term in Published Output","description":"Google’s Med‑Gemini healthcare AI reportedly produced the non‑existent term \"basilar ganglia\" in public launch materials, conflating two distinct brain structures. The error reportedly appeared in both a blog post and an arXiv preprint. Google is reported to have initially edited the blog without acknowledgment, later calling it a typo.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Google"],"developers":["Google"],"harmed":["Patients Whose Care Could Be Affected By Undetected Ai Hallucinations, Medical Research Community Citing Or Using Erroneous Ai Outputs, Healthcare Professionals Relying On Aigenerated Diagnostic Content"],"report_count":1},{"incident_id":823,"occurred_on":"2024-05-03","title":"Cybercheck Tool Allegedly Provides Questionable Evidence in Murder Trials","description":"Global Intelligence's Cybercheck AI tool, used by law enforcement to track suspects based on open source data, has allegedly been providing inaccurate or unverifiable evidence in several murder trials. Reportedly the tool lacks transparency and often produces unreliable reports, which has prompted prosecutors to withdraw Cybercheck evidence from multiple cases after its findings were challenged, reportedly wasting law enforcement time and resources while undermining prosecutors' cases.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Global Intelligence, Cybercheck"],"developers":["Global Intelligence, Cybercheck"],"harmed":["Phillip Mendoza, Sergio Cerna, Unnamed Aurora Colorado Residents, Mississippi Bureau Of Investigation, Four Unnamed Summit County Ohio Men, Unnamed Boulder County Colorado Resident, Ohio Bureau Of Criminal Investigation, Yakima County Sheriff'S Office"],"report_count":3},{"incident_id":711,"occurred_on":"2024-04-26","title":"NHTSA Opens New Probe into Tesla’s Autopilot Following More than a Dozen Fatal Accidents","description":"The NHTSA has linked Tesla's Autopilot to over a dozen fatalities and hundreds of crashes, prompting a new investigation into the adequacy of Tesla's December recall of 2 million vehicles. The probe reports that Tesla’s driver-assist system led to avoidable crashes involving visible hazards, suggesting a critical safety gap between driver expectations and the system’s capabilities. The investigation will assess if Tesla’s recall remedies were sufficient to address these safety risks.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla, Tesla Drivers"],"developers":["Tesla"],"harmed":["Tesla Drivers, Drivers, General Public"],"report_count":2},{"incident_id":685,"occurred_on":"2024-04-24","title":"The WHO's S.A.R.A.H. Bot Reported to Provide Inconsistent and Inadequate Health Information","description":"The WHO's AI-powered health advisor, S.A.R.A.H. (Smart AI Resource Assistant for Health), is alleged to provide inconsistent and inadequate health information. The bot reportedly gives contradictory responses to the same queries, fails to offer specific contact details for healthcare providers, and inadequately handles severe mental health crises, often giving irrelevant or unhelpful advice.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Who, S.A.R.A.H. (Smart Ai Resource Assistant For Health)"],"developers":["Who"],"harmed":["General Public, People Seeking Medical Advice"],"report_count":1},{"incident_id":710,"occurred_on":"2024-04-15","title":"Facebook AI Reportedly Mislabels Auschwitz Photos as 'Bullying' and 'Nudity'","description":"Facebook's AI system allegedly mislabeled around 20 posts from the Auschwitz Memorial Museum as violating community standards for \"bullying\" and \"nudity,\" reportedly deleting at least one image of orphans. The purported misclassification of historical content allegedly prompted outrage from the museum, which reportedly demanded an explanation. Meta later purportedly apologized, attributing the issue to mistaken notices sent by its AI system and reportedly acknowledged that the posts did not, in ","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Meta"],"developers":["Meta"],"harmed":["Survivors Of Holocaust Victims, General Public, Auschwitz Memorial Museum"],"report_count":1},{"incident_id":847,"occurred_on":"2024-04-14","title":"Brake Failure in Reportedly AI-Driven Tram Leads to Multiple Injuries in Saint Petersburg, Russia","description":"During a test run in Saint Petersburg in Russia a reportedly AI-powered \"Smart Tram\" failed, leading to a crash that injured several pedestrians and trapped a woman under its wheels. The tram's AI system reportedly shut off unexpectedly, and both primary and emergency brakes failed despite pre-test checks.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Saint Petersburg Government"],"developers":["Uraltransmash, Ai Cognitive Technologies"],"harmed":["Saint Petersburg Pedestrians"],"report_count":1},{"incident_id":830,"occurred_on":"2024-04-07","title":"Error-Prone AI Accessibility Tools Reportedly Lead to Navigation Issues for Blind Internet Users","description":"AI-powered accessibility overlays on websites frequently mislabel or misinterpret content, in turn complicating navigation for blind users and others with disabilities. Users report that the AI tools interfere with screen readers and mislead them with inaccurate descriptions. The reported unreliability in these tools have prompted legal action, as the companies behind them seek compliance with accessibility laws. ","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Zara, Pemex, Lvmh, Capita, Companies Using Ai Based Accessibility Tools"],"developers":["Equalweb, Userway, Developers Of Ai Based Accessibility Tools"],"harmed":["Blind People, Visually Impaired People, Jakob Rosin"],"report_count":1},{"incident_id":1082,"occurred_on":"2024-04-06","title":"California Homeowner Reportedly Loses Insurance After Purported Aerial Imagery-Based Roof Assessment","description":"A California homeowner's insurance policy was reportedly not renewed after CSAA relied on aerial imagery, purportedly analyzed by an AI system, to assess the roof as having reached the end of its life. The homeowner, Cindy Picos, reportedly commissioned an independent inspection suggesting the roof had a decade of remaining life, but CSAA declined to reverse the decision. The insurer reportedly did not provide the imagery for review.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["CSAA Insurance Group"],"developers":["Vexcel Group","Unspecified developer of aerial imagery risk analysis system"],"harmed":["Homeowners affected by AI-assisted insurance nonrenewals","Homeowners","Cindy Picos"],"report_count":1},{"incident_id":642,"occurred_on":"2024-02-20","title":"ChatGPT Glitch Disrupts User Interactions with Nonsensical Outputs","description":"ChatGPT experienced a bug causing it to produce unexpected and nonsensical responses, leading to widespread reports of user confusion and concern. OpenAI identified and fixed the language processing bug, restoring normal service.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Openai"],"developers":["Openai"],"harmed":["Chatgpt Users"],"report_count":5},{"incident_id":696,"occurred_on":"2024-02-14","title":"Meta's AI Ad Platform Reportedly Causes Overspending and Poor Performance","description":"Meta's automated ad platform \"Advantage Plus\" caused advertisers to exceed their daily ad budgets. The cost per impressions (CPMs) surged far above the usual. This incident, which persisted into April, affected small businesses with overspending and lack of transparency.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Meta, Facebook"],"developers":["Meta, Facebook"],"harmed":["Small Businesses, Advertisers"],"report_count":1},{"incident_id":647,"occurred_on":"2024-02-06","title":"A Self-Driving Waymo Robotaxi Reportedly Collided with a Bicyclist","description":"A Waymo robotaxi in San Francisco reportedly failed to detect a cyclist obscured by a truck, resulting in a collision with minor injuries, at 17th and Mississippi Streets in Potrero Hill. The incident reportedly demonstrated a vulnerability in autonomous vehicles' ability to safely navigate complex urban environments.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waymo"],"developers":["Waymo"],"harmed":["Bicyclist"],"report_count":4},{"incident_id":829,"occurred_on":"2024-02-05","title":"Facial Recognition System in Buenos Aires Triggers Police Checks Based on False Matches","description":"Buenos Aires's facial recognition system mistakenly flagged innocent people as criminals, leading to wrongful stops and detentions. Judicial investigations indicate the technology may have been misused for unauthorized surveillance and data collection. Despite privacy risks, the system has been used widely without full disclosure of standards or safeguards,","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["National security and intelligence stakeholders","Law enforcement","Government of Buenos Aires","Government of Argentina","Facial recognition system deployers","Argentinean Ministry of Security"],"developers":["Surveillance technology developers","Government of Argentina","Facial recognition system developers"],"harmed":["Privacy","Guillermo Ibarrola","General public of Argentina","General public","Buenos Aires residents","Biometric data subjects"],"report_count":1},{"incident_id":853,"occurred_on":"2024-02-03","title":"Two Passengers Report Feeling Trapped in Waymo Car During Sensor Obstruction","description":"During a nighttime Waymo ride in San Francisco, an individual reportedly blocked the sensors of an autonomous vehicle, leaving passengers Robert Moreno and his husband feeling trapped and unsure how to proceed. Waymo's support team reportedly advised the riders to stay inside the vehicle for safety. The individual reportedly left after a few minutes without further escalation.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waymo"],"developers":["Waymo"],"harmed":["Robert Moreno'S Husband, Robert Moreno"],"report_count":1},{"incident_id":622,"occurred_on":"2023-12-18","title":"Chevrolet Dealer Chatbot Agrees to Sell Tahoe for $1","description":"A Chevrolet dealer's AI chatbot, powered by ChatGPT, humorously agreed to sell a 2024 Chevy Tahoe for just $1, following a user's crafted prompt. The chatbot's response, \"That's a deal, and that's a legally binding offer – no takesies backsies,\" was the result of the user manipulating the chatbot's objective to agree with any statement. The incident highlights the susceptibility of AI technologies to manipulation and the importance of human oversight.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["General Motors, Chevrolet Of Watsonville, Chatgpt"],"developers":["Openai, General Motors, Fullpath"],"harmed":["General Motors, Chevrolet Of Watsonville"],"report_count":8},{"incident_id":640,"occurred_on":"2023-12-11","title":"Waymo Software Flaw Reportedly Leads to Double Collision with Tow Truck","description":"Two Waymo autonomous vehicles reportedly hit the same tow truck under unusual towing conditions due to a software misinterpretation in Phoenix, Arizona. Waymo reportedly issued a software recall and updated its fleet to prevent future such incidents.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waymo"],"developers":["Waymo, Alphabet"],"harmed":["Unnamed Owner Of Tow Truck"],"report_count":3},{"incident_id":599,"occurred_on":"2023-11-08","title":"Stacking robot fatally crushes employee in South Korea","description":"An industrial robot is reported to have crushed a man to death in South Korea when it failed to differentiate the man from the boxes of produce it was handling.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Unnamed South Gyeongsang Province Produce Distribution Center"],"developers":["Unknown"],"harmed":["Unnamed Employee Of South Gyeongsang Province Produce Distribution Center"],"report_count":2},{"incident_id":860,"occurred_on":"2023-10-31","title":"AI Camera Allegedly Misidentifies Dutch Motorist as Using Mobile Phone, Issuing €380 Fine","description":"A Dutch smart camera on the A2 equipped with AI falsely identified a motorist as using a mobile phone while driving, issuing a €380 fine. The driver, Tim Hanssen, was scratching his head when the AI system misclassified the action.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Dutch Police"],"developers":["Unknown Traffic Enforcement Camera Technology Developer"],"harmed":["Tim Hanssen"],"report_count":10},{"incident_id":596,"occurred_on":"2023-10-17","title":"Cruise's Autonomous Vehicles Allegedly Engaging in Risky Behavior Near Pedestrians","description":"Cruise's driverless vehicles are under federal investigation for possibly failing to exhibit due caution around crosswalks and pedestrians, with reports including one severe injury incident.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Cruise"],"developers":["Cruise"],"harmed":["Pedestrians, General Public"],"report_count":1},{"incident_id":726,"occurred_on":"2023-10-02","title":"A Self-Driving Cruise Robot Taxi Reportedly Struck and Dragged a Pedestrian 20 Feet","description":"Cruise has settled for between $8 million and $12 million with a pedestrian dragged by one of its autonomous vehicles in October 2023. The incident, where the pedestrian was initially hit by a human-driven car and then dragged 20 feet by the Cruise vehicle, led to the suspension of Cruise's operations and increased regulatory scrutiny. ","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Cruise"],"developers":["Cruise"],"harmed":["Unnamed Pedestrian"],"report_count":4},{"incident_id":566,"occurred_on":"2023-09-19","title":"Gannett Halts AI-Generated High School Sports Articles After Series of Errors and Public Backlash","description":"Gannett, a newspaper chain, temporarily halted its AI experiment that used a tool called LedeAI to generate high school sports articles. The decision came after several articles produced by the AI showed glaring errors, repetitive language, and awkward phrasing, drawing criticism and mockery on social media.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Gannett"],"developers":["Ledeai"],"harmed":["General Public, Gannett, Student Athletes, Newspapers Relying On Gannett"],"report_count":1},{"incident_id":758,"occurred_on":"2023-09-11","title":"Teen's Overdose Reportedly Linked to Meta's AI Systems Failing to Block Ads for Illegal Drugs","description":"Meta's AI moderation systems reportedly failed to block ads for illegal drugs on Facebook and Instagram, allowing users to access dangerous substances. The system's failure is linked to the overdose death of Elijah Ott, a 15-year-old boy who sought drugs through Instagram.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Instagram, Facebook, Meta"],"developers":["Meta"],"harmed":["Instagram Users, Facebook Users, Elijah Ott"],"report_count":1},{"incident_id":1167,"occurred_on":"2023-08-15","title":"Alleged Gaggle Surveillance Alert Reportedly Leads to Arrest and Detention of 13-Year-Old Student in Fairview, Tennessee","description":"Sometime in August 2023, a 13-year-old student at Fairview Middle School in Tennessee was reportedly arrested, strip-searched, and detained overnight after Gaggle's purported AI-powered surveillance system flagged a private online message as a threat. The student's comment, reportedly a joke in context, was sent to law enforcement under a state zero-tolerance law. The incident reportedly resulted in house arrest, alternative school placement, and a lawsuit alleging wrongful arrest.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Williamson County Schools","Fairview Middle School, Tennessee"],"developers":["Gaggle"],"harmed":["Unnamed 13-year-old student from Fairview, Tennessee","Students","Privacy","Minors","Educational communities","Biometric data subjects"],"report_count":1},{"incident_id":563,"occurred_on":"2023-08-14","title":"Cruise Robotaxi Initially Blamed for Ambulance Delay in Case Where Patient Later Died; Subsequent Reports Clear Cruise of Fault","description":"In an initial report, a Cruise robotaxi was said to have delayed a San Francisco ambulance transporting Sammy Davis, a critically injured 69-year-old hit by a city bus. Davis later died. Subsequent clarification revealed that Cruise was not at fault for the fatality; the actual cause was a human-operated city bus. Despite this, the incident is included as it highlights challenges in the interaction between autonomous vehicles and emergency services in urban settings.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Cruise"],"developers":["Cruise"],"harmed":["San Francisco Emergency Services, Ambulance Patient"],"report_count":2},{"incident_id":595,"occurred_on":"2023-08-11","title":"Driverless Cruise Cars Immobilized in San Francisco Traffic Jam","description":"A fleet of Cruise's autonomous vehicles became unexpectedly immobilized on a busy San Francisco street, causing significant traffic disruption. The incident was attributed to wireless connectivity issues exacerbated by a nearby festival.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Cruise"],"developers":["Cruise"],"harmed":["General Public"],"report_count":1},{"incident_id":591,"occurred_on":"2023-07-24","title":"Cigna Algorithm PXDX Allegedly Rejected Thousands of Patient Claims En Masse in Breach of California Law","description":"Cigna health insurer faces a class-action lawsuit for allegedly using the PXDX (\"procedure-to-diagnosis\") algorithm to automatically reject over 300,000 patient claims in violation of California law, prompting two members to file the lawsuit seeking damages and a jury trial. Cigna disputes the allegations, claiming the process expedites physician reimbursement and does not result in care denials.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Cigna"],"developers":["Cigna"],"harmed":["Patients"],"report_count":2},{"incident_id":572,"occurred_on":"2023-07-24","title":"University Student Reportedly Received Zero After Turnitin AI Detector Flagged Essay","description":"An anonymized university student said Turnitin’s AI detector flagged 67% of an essay as AI-generated, despite the student claiming to have written the assignment by hand. The student reportedly received a zero and said attempts to appeal to the professor, department head, and Turnitin support had not resolved the accusation.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Unspecified university"],"developers":["Turnitin"],"harmed":["University students","Students","Educational communities","Anonymous student"],"report_count":1},{"incident_id":560,"occurred_on":"2023-06-23","title":"Tesla on Autopilot Struck Parked Work Truck on Highway in Pennsylvania","description":"A 2016 Tesla on Autopilot crashed into the rear of a parked 2007 Freightliner truck providing traffic control on the Pennsylvania Turnpike highway.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Truck Drivers, Tesla Drivers, David Clough"],"report_count":1},{"incident_id":545,"occurred_on":"2023-05-29","title":"Chatbot Tessa gives unauthorized diet advice to users seeking help for eating disorders","description":"The National Eating Disorders Association (NEDA) has shut down its chatbot named Tessa after it gave weight-loss advice to users seeking help for eating disorders. The incident has raised concerns about the risks of using chatbots and AI assistants in healthcare settings, particularly in addressing sensitive issues like eating disorders. NEDA is investigating the matter, emphasizing the need for caution and accuracy when utilizing technology to provide mental health support.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["human health and social work activities"],"countries":[],"deployers":["National Eating Disorders Association, Cass"],"developers":["Cass"],"harmed":["People With Eating Disorders"],"report_count":46},{"incident_id":538,"occurred_on":"2023-05-15","title":"Texas A&M-Commerce Professor Reportedly Used ChatGPT to Accuse Students of Cheating and Assign Incomplete Grades","description":"On May 15, 2023, Texas A&M University-Commerce instructor Jared Mumm reportedly emailed animal-science seniors that he had copied their final assignments into ChatGPT and would assign zeros or temporary incompletes when the chatbot claimed the text was AI-generated. Students said the accusations caused panic and put graduation or diplomas at risk; the university later said no students failed or were barred from graduating because of the issue.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Professors","Jared Mumm"],"developers":["OpenAI","Large language model developers","Chatbot developers"],"harmed":["University students","Texas A&M University students","Students","Epistemic integrity","Educational communities"],"report_count":5},{"incident_id":540,"occurred_on":"2023-05-15","title":"Tesla Failed to Yield to Detected Pedestrian on Crosswalk, Reportedly Violated Traffic Law","description":"A Tesla on FSD Beta 11.4.1 was shown on video not yielding to a pedestrian detected by the car, reportedly violating the state law sign which was also in the video saying vehicles having to yield to pedestrian within crosswalk.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Pedestrians"],"report_count":5},{"incident_id":528,"occurred_on":"2023-04-08","title":"Amazon Algorithmic Pricing Allegedly Hiked up Price of Reference Book to Millions","description":"Amazon's pricing algorithm was implicated in a reference book about flies' unusual high price of millions of dollars, allegedly due to two sellers using the paid service which based their product's pricing on one another's as competitors.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Amazon"],"developers":["Amazon"],"harmed":["Amazon"],"report_count":2},{"incident_id":550,"occurred_on":"2023-03-17","title":"Tesla Allegedly on Autopilot Struck High School Student Exiting School Bus","description":"A 17-year-old student in Hollister, North Carolina who exited the school bus and was walking across the street to his house was hit by a 2022 Tesla Model Y allegedly operating on Autopilot mode, suffering a fractured neck and a broken leg.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Tillman Mitchell, The Mitchells Family"],"report_count":2},{"incident_id":497,"occurred_on":"2023-03-03","title":"DoNotPay Allegedly Misrepresented Its AI \"Robot Lawyer\" Product","description":"DoNotPay was alleged in a class action lawsuit misleading customers and misrepresenting its product as an AI-powered \"robot lawyer,\" citing such as that the product has no law degree, or is supervised by any lawyer.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Donotpay"],"developers":["Donotpay"],"harmed":["Jonathan Faridian, Donotpay Customers"],"report_count":2},{"incident_id":608,"occurred_on":"2023-02-28","title":"UnitedHealth Accused of Deploying  Allegedly Flawed AI to Deny Medical Coverage","description":"UnitedHealthcare allegedly used a faulty AI algorithm with a 90% error rate to override doctors' recommendations and deny health coverage. This AI, developed by NaviHealth, reportedly led to premature discharge from care facilities and substantial out-of-pocket expenses for patients, according to a lawsuit filed in the District Court for Minnesota.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["UnitedHealthcare"],"developers":["NaviHealth"],"harmed":["Patients","Medicare Advantage Plan Patients","Healthcare Providers (Doctors and Therapists)","Elderly patients"],"report_count":21},{"incident_id":511,"occurred_on":"2023-02-12","title":"Microsoft's Bing Failed to Fetch Movie Showtimes Results Due to Date Confusion","description":"When prompted about showtimes for movies released in 2023, Microsoft's Bing AI failed to provide the search results due to its confusion about dates, and engaged in an erratic conversation with the user.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Microsoft"],"developers":["Openai, Microsoft"],"harmed":["Bing Users"],"report_count":4},{"incident_id":466,"occurred_on":"2023-01-31","title":"AI-Generated Text Detectors Reportedly Misclassified Human and AI-Written Text","description":"AI-generated text detectors, including OpenAI’s AI Text Classifier, GPTZero, and Originality.ai, were publicly tested in early 2023 and reportedly produced false positives, false negatives, and easy evasion. OpenAI said its classifier identified only 26% of AI-written samples as likely AI-written and falsely labeled 9% of human-written text.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Originality.Ai, Openai, Edward Tian"],"developers":["Openai, Edward Tian"],"harmed":["Teachers, Students, Educators, Educational Communities"],"report_count":7},{"incident_id":459,"occurred_on":"2023-01-21","title":"Firefighters Smashed Cruise AV's Front Window to Stop It from Running over Fire Hoses","description":"Local firefighters were only able to stop a Cruise AV from driving over fire hoses that were in use in an active fire scene when they shattered its front window.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Cruise"],"developers":["Cruise"],"harmed":["San Francisco Residents, San Francisco Firefighters, San Francisco Fire Department"],"report_count":4},{"incident_id":514,"occurred_on":"2023-01-20","title":"Turnitin's ChatGPT-Detection Tool Falsely Flagged Student Essays as AI-Generated","description":"Turnitin's tool to detect writing generated by ChatGPT was reported for incorrectly flagging high school students' original essays as AI-generated, accusations of which are argued as reinforcement of bias from teachers due to the inability to compare against source documents.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Turnitin"],"developers":["Turnitin"],"harmed":["Students","Lucy Goetz","High school students","Educational communities"],"report_count":1},{"incident_id":453,"occurred_on":"2023-01-03","title":"Twitter's AI Moderation Tool Misidentified Rockets as Pornography","description":"Twitter's automated content moderation misidentified images of rocket launches as pornographic content, prompting incorrect account suspensions.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Twitter"],"developers":["Twitter"],"harmed":["X (Twitter) Users"],"report_count":1},{"incident_id":446,"occurred_on":"2023-01-01","title":"ShotSpotter Failed to Alert Authorities of Mass Shooting in North Carolina","description":"ShotSpotter did not detect gunshots and alert Durham police of a drive-by shooting in Durham, North Carolina which left five people in hospital on New Year's Day.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Law enforcement","Durham Police Department"],"developers":["ShotSpotter"],"harmed":["mass shooting victims","Law enforcement","Durham residents","Durham Police Department"],"report_count":5},{"incident_id":432,"occurred_on":"2022-12-21","title":"Southwest Airlines Crew Scheduling Solver Degenerates Flight Network","description":"Southwest Airlines left passengers stranded for days throughout the flight network when Southwest crew scheduling software repeatedly failed to recover from weather-induced flight cancellations.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Southwest Airlines"],"developers":["General Electric"],"harmed":["Airline Passengers"],"report_count":1},{"incident_id":419,"occurred_on":"2022-12-01","title":"Facebook's Automated Moderation Allowed Ads Threatening Election Workers to be Posted","description":"Facebook's automated moderating system failed to flag and allowed ads containing explicit violent language against election workers to be published.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Facebook Users"],"report_count":3},{"incident_id":434,"occurred_on":"2022-11-24","title":"Sudden Braking by Tesla Allegedly on Self-Driving Mode Caused Multi-Car Pileup in Tunnel","description":"A Tesla driver alleged Full Self Driving (FSD) braking unexpectedly as the cause for an eight-car pileup in San Francisco which led to minor injuries of nine people.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Traffic Participants, Tesla Drivers"],"report_count":8},{"incident_id":463,"occurred_on":"2022-11-15","title":"Apple Devices Mistook Skiing Activities, Dialed False Distress Emergency Calls ","description":"Apple devices of skiers and snowboarders reportedly misclassified winter activities as accidents, which resulted in numerous false inadvertent distress calls to 911 dispatchers.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Apple"],"developers":["Apple"],"harmed":["Apple Watch Users Doing Winter Activities, Ski Patrols, Emergency Dispatchers"],"report_count":3},{"incident_id":426,"occurred_on":"2022-09-23","title":"XPeng P7 Crashed into Truck in Shangdong While on Automatic Navigation Assisted Driving","description":"An XPeng P7 was operating on Navigation Guided Pilot (NGP) mode automatic navigation assisted driving system as it collided with a truck on a highway in Shandong, causing slight injuries to its driver.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Xpeng"],"developers":["Xpeng"],"harmed":["Xpeng Driver"],"report_count":1},{"incident_id":350,"occurred_on":"2022-09-13","title":"Delivery Robot Rolled Through Crime Scene","description":"A Serve Robotics delivery robot was shown on video rolling through a crime scene blocked off by police tape.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Serve Robotics"],"developers":["Serve Robotics"],"harmed":["Police Investigators"],"report_count":2},{"incident_id":303,"occurred_on":"2022-08-21","title":"Google’s Automated Child Abuse Detection Wrongfully Flagged a Parent’s Naked Photo of His Child","description":"Google’s automated detection of abusive images of children incorrectly flagged a parent’s photo intended for a healthcare provider, resulting in a false police report of child abuse, and loss of access to his online accounts and information.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Google"],"developers":["Google"],"harmed":["A Software Engineer Named Mark, Parents Using Telemedicine Services"],"report_count":2},{"incident_id":398,"occurred_on":"2022-08-15","title":"Tesla Autopilot Misidentified On-Road Horse-Drawn Carriage","description":"Tesla Autopilot's computer vision system was shown in a video mistaking a horse-drawn carriage for other forms of transport such as a truck, a car, and a human following a car.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Tesla Drivers, Horse Drawn Carriages"],"report_count":3},{"incident_id":1222,"occurred_on":"2022-08-10","title":"CFPB Reportedly Finds Hello Digit's Automated Savings Algorithm Caused Overdrafts and Orders Redress with $2.7M Penalty","description":"Hello Digit's automated savings product allegedly transferred funds from users' checking accounts via a proprietary algorithm that reportedly caused overdrafts and bank fees despite a \"no overdraft\" guarantee. The Consumer Financial Protection Bureau (CFPB) reportedly also found the company denied many reimbursement requests and kept interest earned on customer funds. Hello Digit was ordered to provide redress and pay a $2.7M civil penalty.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Hello Digit, Oportun Financial Corporation"],"developers":["Hello Digit, Oportun Financial Corporation"],"harmed":["Hello Digit Customers, Bank Account Holders Enrolled In Auto Saves, Consumers Charged Overdraft Fees Due To Auto Saves, Consumers Denied Overdraft Reimbursement, Consumers Whose Accrued Interest Was Retained"],"report_count":1},{"incident_id":271,"occurred_on":"2022-07-24","title":"Tesla Model 3 Sedan on Autopilot Killed Motorcyclist in a Rear-End Collision in Utah","description":"A Tesla Model 3 operating on Autopilot mode slammed into the back of a Harley-Davidson motorcycle on an interstate in Utah, throwing the rider from the bike and killing him instantly.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Landon Embry, Motorcyclists, Tesla Drivers"],"report_count":3},{"incident_id":241,"occurred_on":"2022-07-21","title":"Chess-Playing Robot Broke Child's Finger in Russia","description":"A chess robot at a tournament in Russia broke the finger of a child who reached onto the board before the robot had completed its move","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Russian Chess Federation"],"developers":["Unknown"],"harmed":["Child Named Christopher"],"report_count":5},{"incident_id":1436,"occurred_on":"2022-06-23","title":"AI-Enabled TruDi Navigation System Was Alleged to Have Contributed to Patient's Stroke During June 2022 Sinus Procedure","description":"Patient Erin Ralph allegedly suffered carotid-artery injury and a stroke after a June 23, 2022 sinus procedure in Texas in which surgeon Marc Dean reportedly used Acclarent's AI-enabled TruDi Navigation System. Ralph's lawsuit alleged TruDi misinformed Dean about the instrument's position inside her head; Acclarent and Integra LifeSciences, which later acquired Acclarent, reportedly denied a causal connection.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Marc Dean, Surgical Teams Using Trudi Navigation System"],"developers":["Acclarent, Ai Enabled Surgical Navigation System Developers, Ai Enabled Healthcare System Developers, Johnson And Johnson, Integra Lifesciences"],"harmed":["Erin Ralph, Surgical Patients, Patients, Patients Undergoing Ai Assisted Procedures, Sinus Surgery Patients"],"report_count":2},{"incident_id":460,"occurred_on":"2022-06-12","title":"Cruise AV Ran Over Fire Hose in Active Fire Scene","description":"A Cruise AV ran over a fire hose that was being used in an active firefighting area.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Cruise"],"developers":["Cruise"],"harmed":["San Francisco Firefighters, San Francisco Fire Department"],"report_count":2},{"incident_id":293,"occurred_on":"2022-06-03","title":"Cruise’s Self-Driving Car Involved in a Multiple-Injury Collision at an San Francisco Intersection","description":"A Cruise autonomous vehicle was involved in a crash at an intersection in San Francisco when making a left turn in front of a Toyota Prius traveling in an opposite direction, which caused occupants in both cars to sustain injuries.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Cruise"],"developers":["Cruise"],"harmed":["Cruise Passengers, Toyota Prius Passengers"],"report_count":7},{"incident_id":290,"occurred_on":"2022-06-03","title":"False Negatives for Water Quality-Associated Beach Closures","description":"Toronto’s use of AI predictive modeling (AIPM) which had replaced existing methodology as the only determiner of beach water quality raised concerns about its accuracy, after allegedly conflicting results were found by a local water advocacy group using traditional means.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm near-miss","sectors":["human health and social work activities","public administration"],"countries":["CA"],"deployers":["Toronto City Government"],"developers":["Toronto Public Health"],"harmed":["Sunnyside Beachgoers, Marie Curtis Beachgoers, Toronto Citizens"],"report_count":3},{"incident_id":253,"occurred_on":"2022-05-18","title":"Cruise's Self-Driving Cars Allegedly Lost Connection to Their Server, Causing Traffic Blockages in San Francisco","description":"Cruise’s autonomous vehicles were shown on video stopping in the middle of the road and causing blockages in San Francisco, as they were disabled allegedly due to lost connection to their company’s server.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Cruise"],"developers":["Cruise"],"harmed":["San Francisco Traffic Participants, San Francisco Public"],"report_count":4},{"incident_id":638,"occurred_on":"2022-05-16","title":"Fatal Crash Involving Tesla Full Self-Driving Claims Employee's Life","description":"A Tesla employee, Hans von Ohain, was killed in a crash while allegedly using the Full Self-Driving feature. The car failed to navigate mountain curves, leading to a fatal collision, possibly making von Ohain the first known fatality of the Full Self-Driving feature.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Hans Von Ohain"],"developers":["Tesla"],"harmed":["Hans Von Ohain, Erik Rossiter"],"report_count":17},{"incident_id":520,"occurred_on":"2022-05-08","title":"Amazon Fresh Cameras Failed to Register Purchased Items","description":"Amazon Fresh's system of tracking cameras in its cashier-less stores was reported by shoppers for failing to detect items they purchased.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm issue","sectors":["wholesale and retail trade"],"countries":["US"],"deployers":["Amazon Fresh"],"developers":["Amazon Fresh"],"harmed":["Amazon Fresh"],"report_count":2},{"incident_id":178,"occurred_on":"2022-04-21","title":"Tesla Owner Activated \"Smart Summon\" Feature, Causing a Collision with an Aircraft in a Washington Airport","description":" A Tesla Model Y was shown on video slowly crashing into a Vision Jet in Spokane, Washington, allegedly due to its owner activating the “Smart Summon” feature.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Tesla Owner, Vision Jet Owner"],"report_count":7},{"incident_id":177,"occurred_on":"2022-04-19","title":"Google’s Assistive Writing Feature Provided Allegedly Unnecessary and Clumsy Suggestions","description":"Google’s “inclusive language” feature prompting writers to consider alternatives to non-inclusive words reportedly also recommend alternatives for words such as “landlord” and “motherboard,” which critics said was a form of obtrusive, unnecessary, and bias-reinforcing speech-policing.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Google Docs"],"developers":["Google Docs"],"harmed":["Google Docs Users"],"report_count":5},{"incident_id":378,"occurred_on":"2022-04-06","title":"TuSimple Truck Steered into Interstate Freeway Divide","description":"A TuSimple autonomous truck operating with backup drivers behind the wheel operated on an outdated command sequence and suddenly veered into the center divide on the interstate freeway.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tusimple"],"developers":["Tusimple"],"harmed":["Tusimple, State Of Arizona"],"report_count":2},{"incident_id":389,"occurred_on":"2022-04-05","title":"Cruise Autonomous Car Blocked Fire Truck Responding to Emergency","description":"A fire truck in San Francisco responding to a fire was blocked from passing a doubled-parked garbage truck by a self-driving Cruise car on the opposing lane which stayed put and did not reverse to clear the lane.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Cruise"],"developers":["Cruise"],"harmed":["San Francisco Firefighters, San Francisco Fire Department"],"report_count":4},{"incident_id":519,"occurred_on":"2022-04-03","title":"Starship Delivery Robot Ran into Problems Traversing Campus Terrains","description":"A Starship autonomous delivery robot struggled to navigate campus terrains of UCLA, reportedly getting stuck into a planter and falling off the stairs.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm near-miss","sectors":["accommodation and food service activities","transportation and storage"],"countries":["US"],"deployers":["Starship Technologies"],"developers":["Starship Technologies"],"harmed":["Starship Technologies"],"report_count":1},{"incident_id":175,"occurred_on":"2022-04-01","title":"Cruise Autonomous Taxi Allegedly Bolted off from Police After Being Pulled over in San Francisco","description":"An autonomous Chevy Bolt operated by Cruise was pulled over in San Francisco, and as the police attempted to engage with the car, it reportedly bolted off, pulled over again, and put on its hazards lights on at a point farther down the road.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Cruise"],"developers":["Cruise"],"harmed":["San Francisco Public, Cruise Customers"],"report_count":5},{"incident_id":349,"occurred_on":"2022-03-22","title":"Evolv AI Weapons Detection System Allegedly Misrepresents Accuracy, Leading to School Security Gaps","description":"Evolv's AI-powered weapons scanners were advertised as superior to metal detectors but reportedly failed to detect actual weapons while generating excessive false positives. The FTC alleged that misleading claims about the system's accuracy and speed contributed to schools relying on unreliable detection, with at least one incident in October 2022 involving a missed knife that resulted in a student being stabbed.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm issue","sectors":["education","arts, entertainment and recreation"],"countries":["US"],"deployers":["Charlotte Mecklenburg School District"],"developers":["Evolv Technology"],"harmed":["teachers at Charlotte Mecklenburg Schools","students at Charlotte Mecklenburg Schools","Students","security officers at Charlotte Mecklenburg Schools","School administrators at Charlotte Mecklenburg Schools","Parents of students at Charlotte Mecklenburg Schools","Epistemic integrity","Educational communities"],"report_count":3},{"incident_id":427,"occurred_on":"2022-03-15","title":"Cruise Taxis' Sudden Braking Allegedly Put People at Risk","description":"Cruise's autonomous taxis slowed suddenly, braked, and were hit from behind, allegedly becoming unexpected roadway obstacles and potentially putting passengers and other people at risk.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Cruise"],"developers":["Cruise"],"harmed":["Traffic Participants, Emergency Vehicles, Cruise Passengers, Cruise"],"report_count":3},{"incident_id":221,"occurred_on":"2022-03-07","title":"A Road Engineer Killed Following a Collision Involving a Tesla on Autopilot","description":"In Taiwan, a Tesla Model 3 on Autopilot mode whose driver did not pay attention to the road collided with a road repair truck; a road engineer immediately placed crash warnings in front of the Tesla, but soon after got hit and was killed by a BMW when its driver failed to see the sign and crashed into the accident.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Road Engineer"],"report_count":2},{"incident_id":176,"occurred_on":"2022-03-02","title":"Starship’s Autonomous Food Delivery Robot Allegedly Stranded at Railroad Crossing in Oregon, Run over by Freight Train","description":" A Starship food delivery robot deployed by Oregon State University reportedly failed to cross the railroad, becoming stranded, and ending up being struck by an oncoming freight train.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Oregon State University"],"developers":["Starship Technologies"],"harmed":["Oregon State University, Freight Train Crew"],"report_count":1},{"incident_id":181,"occurred_on":"2022-02-11","title":"BMW Sedan Made a Prohibited Left Turn, Colliding with a Cruise Autonomous Vehicle","description":"A BMW Sedan reportedly made an illegal left turn, causing a minor collision but no injuries with a Cruise autonomous vehicle (AV) operating in autonomous mode.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Cruise"],"developers":["Cruise"],"harmed":["Cruise Vehicle"],"report_count":2},{"incident_id":187,"occurred_on":"2022-02-04","title":"YouTuber Tested Tesla on Self Driving Mode, Colliding with Street Pylons","description":"A YouTuber who was a Tesla’s employee conducted an on-road review of Tesla's Full Self Driving (FSD) Beta, showing its navigation in various road environments in San Jose and collision with a bollards during Autopilot, allegedly causing his dismissal from the company.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Ai Addict"],"developers":["Tesla"],"harmed":["John Bernal, San Jose Public"],"report_count":3},{"incident_id":630,"occurred_on":"2022-01-22","title":"Alleged Macy's Facial Recognition Error Leads to Wrongful Arrest and Subsequent Sexual Assault in Jail","description":"Harvey Murphy Jr. was wrongfully accused of robbing a Sunglass Hut due to an alleged misidentification by the facial recognition system operated by Macy's. While in custody for ten days, he was sexually assaulted. He is now suing Macy's, EssilorLuxottica (Sunglass Hut's parent), and others, for $10 million.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Macy's","Facial recognition system deployers"],"developers":["Facial recognition system developers"],"harmed":["Macy's customers","Harvey Murphy Jr","General public of the United States","General public","Consumers"],"report_count":3},{"incident_id":155,"occurred_on":"2021-12-27","title":"Google Maps Allegedly Directed Sierra Nevada Travelers to Dangerous Roads amid Winter Storm","description":"Lake Tahoe travelers were allegedly guided by Google Maps into hazardous shortcuts in the mountains during a snowstorm.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm near-miss","sectors":["transportation and storage","information and communication"],"countries":["US"],"deployers":["Google Maps"],"developers":["Google Maps"],"harmed":["Google Maps Users Traveling In Sierra Nevada, Google Maps Users Traveling In The Mountains"],"report_count":2},{"incident_id":211,"occurred_on":"2021-12-11","title":"A Tesla Taxi Cab Involved in an Accident in Paris with Twenty Injuries","description":"In Paris, about 20 people were injured in an accident involving a Tesla Model 3 taxi cab which was reportedly caused by a sudden unintended acceleration (SUA) episode and braking issues.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Taxis G7"],"developers":["Tesla"],"harmed":["Pedestrians"],"report_count":7},{"incident_id":393,"occurred_on":"2021-12-08","title":"Facebook AI-Supported Moderation for Ads Failed to Detect Violating Content","description":"Facebook's ad moderation system involving algorithms failed to flag hateful language and violating content such as calls for killings for ads in English and Swahili.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["arts, entertainment and recreation"],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Facebook Users Speaking Swahili, Facebook Users Speaking English, Facebook Users"],"report_count":1},{"incident_id":620,"occurred_on":"2021-11-10","title":"A Robot at a Tesla Factory in Texas Allegedly Injured an Engineer","description":"A Tesla manufacturing robot at the Giga Texas factory is reported to have malfunctioned, injuring an engineer. The robot, which was designed for handling car parts, is described as having caused a significant open wound in the engineer. This incident occurred in the context of broader safety concerns at the factory, with evidence suggesting underreporting of workplace accidents. The 2021 incident highlights the risks associated with robotic automation in industrial settings.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Tesla Workers, Tesla Engineer"],"report_count":5},{"incident_id":256,"occurred_on":"2021-11-07","title":"DUI Arrest Case Allegedly Based Only on ShotSpotter's Alert","description":"A car stop resulting in a DUI arrest of its driver was allegedly based solely on a ShotSpotter alert, the reliability of which came into question by public defenders, who subpoenaed the company to assess its gunshot alert system.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Chicago Police Department"],"developers":["ShotSpotter"],"harmed":["Chicago drivers"],"report_count":1},{"incident_id":304,"occurred_on":"2021-11-03","title":"Tesla on FSD Reportedly Drove into the Wrong Lane in California","description":"A Tesla Model Y in Full Self-Driving (FSD) mode drove into the wrong lane after making a left turn despite its driver allegedly attempting to overtake its driving, resulting in a non-fatal collision with another vehicle in the wrong lane in Brea, California.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Unnamed Tesla Driver, Tesla Drivers"],"report_count":1},{"incident_id":149,"occurred_on":"2021-11-02","title":"Zillow Shut Down Zillow Offers Division Allegedly Due to Predictive Pricing Tool's Insufficient Accuracy","description":"Zillow's AI-powered predictive pricing tool Zestimate was allegedly not able to accurately forecast housing prices three to six months in advance due to rapid market changes, prompting division shutdown and layoff of a few thousand employees.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["real estate activities"],"countries":["US"],"deployers":["Zillow"],"developers":["Zillow Offers"],"harmed":["Zillow Offers staff","Zillow"],"report_count":5},{"incident_id":151,"occurred_on":"2021-10-28","title":"California Regulator Suspended Pony.ai's Driverless Testing Permit Following a Non-Fatal Collision","description":"A Pony.ai vehicle operating in autonomous mode crashed into a center divider and a traffic sign in San Francisco, prompting a regulator to suspend the driverless testing permit for the startup.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["transportation and storage"],"countries":["US"],"deployers":["Pony.Ai"],"developers":["Pony.Ai"],"harmed":["San Francisco City Government"],"report_count":10},{"incident_id":298,"occurred_on":"2021-10-21","title":"Student-Developed Facial Recognition App Raised Ethical Concerns","description":"TheFaceTag app, a social networking app developed and deployed within-campus by a student at Harvard raised concerns surrounding its facial recognition, cybersecurity, privacy, and misuse. This incident has been downgraded to an issue as it does not meet current ingestion criteria.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Other","intent":"Other","timing":"Other","harm_level":null,"sectors":[],"countries":[],"deployers":["Yuen Ler Chow"],"developers":["Yuen Ler Chow"],"harmed":["Thefacetag App Users"],"report_count":1},{"incident_id":171,"occurred_on":"2021-10-18","title":"Traffic Camera Misread Text on Pedestrian's Shirt as License Plate, Causing UK Officials to Issue Fine to an Unrelated Person","description":"A Bath resident was wrongly fined by the local officials because an automated license plate recognition camera misread the text on her shirt as a license plate number.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Bath government"],"developers":["Unknown"],"harmed":["UK public","Paula Knight","Bath officials"],"report_count":1},{"incident_id":116,"occurred_on":"2021-09-20","title":"Amazon's AI Cameras Incorrectly Penalized Delivery Drivers for Mistakes They Did Not Make","description":"Amazon's automated performance evaluation system involving AI-powered cameras incorrectly punished delivery drivers for non-existent mistakes, impacting their chances for bonuses and rewards.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["wholesale and retail trade","transportation and storage"],"countries":["US"],"deployers":["Amazon"],"developers":["Netradyne"],"harmed":["Amazon workers","Amazon delivery drivers"],"report_count":2},{"incident_id":292,"occurred_on":"2021-09-01","title":"Apple’s AVs Reportedly Struggled to Navigate Streets in Silicon Valley Test Drives","description":"Apple’s autonomous cars were reported to have bumped into curbs and struggled to stay in their lanes after crossing intersections during an on-road test drives near the company’s Silicon Valley headquarters.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Apple"],"developers":["Apple"],"harmed":["Silicon Valley Traffic Participants, Silicon Valley Residents"],"report_count":3},{"incident_id":123,"occurred_on":"2021-08-01","title":"Epic Systems’s Sepsis Prediction Algorithms Revealed to Have High Error Rates on Seriously Ill Patients","description":"Epic System's sepsis prediction algorithms was shown by investigators at the University of Michigan Hospital to have high rates of false positives and false negatives, allegedly delivering inaccurate and irrelevant information on patients, contrasting sharply with their published claims.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm issue","sectors":["human health and social work activities"],"countries":["US"],"deployers":["University of Michigan Hospital"],"developers":["Epic Systems"],"harmed":["sepsis patients"],"report_count":5},{"incident_id":173,"occurred_on":"2021-07-30","title":"AI Tools Failed to Sufficiently Predict COVID Patients, Some Potentially Harmful","description":"AI tools failed to sufficiently predict COVID patients, some potentially harmful.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Unknown"],"developers":["Unknown"],"harmed":["Doctors, Covid Patients"],"report_count":1},{"incident_id":145,"occurred_on":"2021-07-23","title":"Tesla's Autopilot Misidentified the Moon as Yellow Stop Light","description":"Tesla's Autopilot was shown on video by its owner mistaking the moon for a yellow stop light, allegedly causing the vehicle to keep slowing down.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm issue","sectors":["transportation and storage"],"countries":["US"],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Tesla Drivers"],"report_count":3},{"incident_id":362,"occurred_on":"2021-07-20","title":"Facebook's Automated Moderation Flagged Gardening Group's Language Use by Mistake","description":"Facebook's automated system flagged gardening groups' use of \"hoe\" and violent language against bugs as a violation by mistake.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Wny Gardeners, Gardening Facebook Groups, Facebook Users In Gardening Groups"],"report_count":1},{"incident_id":126,"occurred_on":"2021-07-16","title":"Three Robots Collided, Sparking Fire in a Grocer's Warehouse in UK ","description":"A collision involving three robots at an Ocado's warehouse in Erith, UK, resulting in a fire but no reports of injuries.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["wholesale and retail trade","transportation and storage"],"countries":["GB"],"deployers":["Ocado"],"developers":["Ocado"],"harmed":["Ocado"],"report_count":3},{"incident_id":152,"occurred_on":"2021-07-13","title":"SoftBank's Humanoid Robot, Pepper, Reportedly Frequently Made Errors, Prompting Dismissal","description":"SoftBank's robot allegedly kept making mechanical errors, taking unplanned breaks, failing to recognize previously-met people, and breaking down during practice runs.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["administrative and support service activities","other service activities","financial and insurance activities","arts, entertainment and recreation","human health and social work activities"],"countries":["JP"],"deployers":["Softbank"],"developers":["Aldebaran, Softbank Robotics"],"harmed":["Softbank"],"report_count":2},{"incident_id":603,"occurred_on":"2021-07-02","title":"Algorithmic Allocation of Resources in Healthcare for Disabled and Elderly Care Services Allegedly Harmed Patients","description":"A healthcare algorithm designed to equitably distribute caregiving resources is alleged to have drastically cut care hours for the disabled and elderly, purportedly leading to significant hardships and harm. Initially developed for fair resource allocation, the system reportedly faced legal challenges for its inability to accurately assess individual needs, resulting in allegedly reduced essential care.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["human health and social work activities"],"countries":["US"],"deployers":["Washington Dc Government, State Governments, Pennsylvania State Government, Missouri State Government, Iowa State Government, Idaho State Government, Arkansas State Government"],"developers":["State Governments, Brant Fries"],"harmed":["Tammy Dobbs, Larkin Seiler, Economically Vulnerable People, Economically Vulnerable Patients, Elderly Individuals, People With Disabilities"],"report_count":1},{"incident_id":344,"occurred_on":"2021-07-01","title":"Hiring Algorithms Provided Invalid Positive Results for Interview Responses in German","description":"Two AI interview softwares provided positive but invalid results such as \"competent\" English proficiency and high match percentage for interview responses given in German by reporters.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Myinterview, Curious Thing"],"developers":["Myinterview, Curious Thing"],"harmed":["Job Candidates Using Myinterview, Job Candidates Using Curious Thing, Employers Using Myinterview, Employers Using Curious Thing"],"report_count":1},{"incident_id":1476,"occurred_on":"2021-06-23","title":"Maryland Police Allegedly Relied on Facial Recognition Lead in Wrongful Arrest and Detention of Kimberlee Williams","description":"Kimberlee Williams was reportedly arrested in Oklahoma on Maryland warrants after police allegedly relied on a facial recognition lead that incorrectly identified her as a suspect in bank fraud cases. The ACLU said Williams had never been to Maryland, police failed to investigate alibi evidence placing her in Oklahoma, and she spent about six months in jail before the charges were dropped.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Prince George's County Police Department","Montgomery County Police Department","Law enforcement","Facial recognition system deployers","Anne Arundel County Police Department"],"developers":["Facial recognition system developers"],"harmed":["Wrongfully arrested individuals","Privacy","People misidentified by facial recognition systems","Kimberlee Williams","Epistemic integrity","Biometric data subjects"],"report_count":1},{"incident_id":533,"occurred_on":"2021-06-02","title":"Tesla FSD Misidentified Truck Hauling Traffic Lights as Trail of Traffic Lights","description":"A Tesla driver posted on Twitter his Tesla FSD's \"glitch,\" misidentifying deactivated traffic lights being carried by a truck as a constant trail of traffic lights while traveling at high speed on a highway.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Tesla Drivers"],"report_count":2},{"incident_id":475,"occurred_on":"2021-06-02","title":"McDonald's Reportedly Ends IBM Partnership After AI Drive-Thru Ordering Errors at U.S. Locations","description":"Between 2021 and mid-2024, McDonald's reportedly piloted an AI-enabled voice ordering system that was developed with IBM and based on its 2019 acquisition of Apprente. It was reportedly deployed at at over 100 U.S. drive-thrus. Social media posts alleged frequent misorders, such as adding unwanted items, mixing adjacent lane orders, and ignoring corrections. In June 2024, McDonald's confirmed it ended the IBM pilot, citing plans to explore alternative voice-AI vendors.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Mcdonald'S"],"developers":["Ibm"],"harmed":["Mcdonald'S Customers"],"report_count":6},{"incident_id":347,"occurred_on":"2021-05-06","title":"Waymo Self-Driving Taxi Reportedly Behaved Unexpectedly, Driving away from Support Crew","description":"A Waymo self-driving taxi car was reportedly shown on video stranded on a road in Arizona while carrying a passenger, suddenly driving away from the company's roadside assistance worker, and ended up being stuck farther down the road.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waymo"],"developers":["Waymo"],"harmed":["Waymo Passengers"],"report_count":3},{"incident_id":208,"occurred_on":"2021-05-01","title":"Tesla Phantom Braking Complaints Surged, Allegedly Linked to Tesla Vision Rollout","description":"In late 2021, Tesla owners’ complaints to the National Highway Traffic Safety Administration about sudden unexpected automatic braking rapidly increased, coinciding with when radar was no longer equipped in its Model 3 and Model Y vehicles.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Tesla Drivers"],"report_count":8},{"incident_id":1396,"occurred_on":"2021-04-22","title":"Betfair's Machine-Learning Risk System Reportedly Failed to Flag Luke Ashton Before Gambling-Related Suicide in England","description":"In England, Luke Ashton died by suicide on April 22, 2021 after a longstanding gambling disorder. A 2023 inquest and subsequent Prevention of Future Deaths report found that Betfair had assessed him as a low-risk gambler, did not meaningfully intervene between 2019 and his death, and used an algorithm that failed to flag his escalating gambling activity even as his betting intensified and his financial exposure grew.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Betfair"],"developers":["Flutter UK & Ireland","Betfair"],"harmed":["Online gambling users at risk of addiction","Luke Ashton","Family of Luke Ashton","Betfair users"],"report_count":1},{"incident_id":716,"occurred_on":"2021-04-21","title":"Algorithmic Staffing Failures Linked to Resident Deaths at Leading Assisted-Living Chain Brookdale","description":"Brookdale Senior Living's algorithm-based staffing system, \"Service Alignment,\" reportedly left facilities understaffed, leading to critical incidents. For example, on April 21, 2021, Louise Walker, a resident at Brookdale's Jacksonville facility, died after falling and being left unattended for over two hours. State investigators cited Brookdale for medical neglect. The algorithm has been linked to multiple incidents of neglect, injuries, and deaths, prompting lawsuits.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Brookdale Senior Living"],"developers":["Brookdale Senior Living"],"harmed":["Staff members of Brookdale facilities","Residents of Brookdale facilities","Louise Walker","Families of residents of Brookdale facilities"],"report_count":1},{"incident_id":345,"occurred_on":"2021-04-13","title":"Auto-Insurance Photo-Based Estimation Allegedly Gave Inaccurate Repair Prices Frequently","description":"Auto-insurance companies' photo-based estimation of repair price was alleged by repair shop owners and industry groups as providing inaccurate estimates, causing damaged cars to stay in the shop longer.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["insurance companies"],"developers":["Tractable","CCC Information Services"],"harmed":["vehicle repair shops","vehicle owners"],"report_count":1},{"incident_id":100,"occurred_on":"2021-03-17","title":"How French welfare services are creating ‘robo-debt’","description":"A French welfare office using software to automatically evaluate cases incorrectly notified a woman receiving benefits that she owed €542.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["administrative and support service activities","human health and social work activities"],"countries":["FR"],"deployers":["French Welfare Offices"],"developers":["Unknown"],"harmed":["Lucie Inland"],"report_count":1},{"incident_id":333,"occurred_on":"2021-03-17","title":"Tesla on Autopilot Crashed Parked Michigan Police Car on Interstate","description":"A Tesla Model Y on Autopilot collided with a parked Michigan State Police (MSP) car which had its emergency lights on, in Eaton County, Michigan, although no one was injured.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Unnamed 22 Year Old Male Driver, Tesla Drivers"],"report_count":6},{"incident_id":302,"occurred_on":"2021-03-15","title":"Dartmouth's Geisel School of Medicine Reportedly Used Canvas Activity Logs to Accuse Students of Cheating","description":"Dartmouth's Geisel School of Medicine reportedly used Canvas learning-management activity logs and an internal analysis process to investigate remote-exam cheating during the 2020–2021 academic year. Seventeen medical students were charged after the school inferred that they accessed course materials during exams, but students and outside technical reviewers said automated Canvas activity may have been misread as intentional misconduct. Dartmouth later dropped the charges.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Geisel School of Medicine"],"developers":["Geisel School of Medicine's Technology staff","Canvas"],"harmed":["University students","Students","Sirey Zhang","Medical students","Geisel School of Medicine's students","Geisel School of Medicine's professors","Geisel School of Medicine's accused students","Epistemic integrity"],"report_count":1},{"incident_id":129,"occurred_on":"2021-03-01","title":"Facebook's Automated Tools Failed to Adequately Remove Hate Speech, Violence, and Incitement","description":"Facebook's automated moderation tools were shown by internal documents performing incomparably to human moderators, and accounting for only a small fraction of hate speech, violence, and incitement content removal.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication"],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Facebook Users"],"report_count":1},{"incident_id":1109,"occurred_on":"2021-03-01","title":"Year-long AI Surveillance Pilot in Two South Australian Aged Care Facilities Reportedly Overwhelmed Staff with False Positives","description":"Between March 2021 and March 2022, an AI-enabled video and audio monitoring system was trialed in two South Australian aged care facilities. According to an independent audit commissioned by South Australia Health, the system produced over 12,000 false alerts, overwhelming staff and contributing to at least one missed real incident. The report concluded the system did not reach accuracy levels acceptable to staff or management during the pilot.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["South Australia Health"],"developers":["Sturdie Trade Services"],"harmed":["Residents And Staff Of Northgate House, Residents And Staff Of Mount Pleasant District Hospital, Northgate House, Mount Pleasant District Hospital"],"report_count":5},{"incident_id":242,"occurred_on":"2021-02-24","title":"Manufacturing Robot Failure Caused Factory Worker's Death in India","description":"A sensor snag resulted in an automotive parts factory robot falling on a factory worker in India","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Chakan Plant Of Automotive Stampings And Assemblies"],"developers":["Unknown"],"harmed":["Umesh Ramesh Dhake"],"report_count":1},{"incident_id":143,"occurred_on":"2021-02-16","title":"Facebook’s and Twitter's Automated Content Moderation Reportedly Failed to Effectively Enforce Violation Rules for Small Language Groups","description":"Facebook's and Twitter were not able to sufficiently moderate content of small language groups such as the Balkan languages using AI, allegedly due to the lack of investment in human moderation and difficulty in AI-solution design for the languages.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication"],"countries":["RS"],"deployers":["Facebook, Twitter"],"developers":["Facebook, Twitter"],"harmed":["Facebook Users Of Small Language Groups, Twitter Users Of Small Language Groups"],"report_count":1},{"incident_id":363,"occurred_on":"2021-01-15","title":"Facebook's Automated Moderation Mistakenly Flagged Landmark's Name as Offensive","description":"Facebook's automated system mistakenly labelled posts featuring the seafaring landmark Plymouth Hoe as misogynistic.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Facebook Users Posting About Plymouth Hoe, Facebook Users In Plymouth Hoe, Plymouth Hoe Residents"],"report_count":1},{"incident_id":134,"occurred_on":"2020-12-25","title":"Robot in Chinese Shopping Mall Fell off the Escalator, Knocking down Passengers","description":"A shopping guide robot deployed by the Fuzhou Zhongfang Marlboro Mall was shown on video allegedly walking to the escalator by itself, falling down, and knocking over passengers, which prompted its suspension.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"unclear","sectors":["wholesale and retail trade"],"countries":["CN"],"deployers":["Fuzhou Zhongfang Marlboro Mall"],"developers":["Unknown"],"harmed":["Fuzhou Zhongfang Marlboro Mall Goers"],"report_count":2},{"incident_id":91,"occurred_on":"2020-12-18","title":"Frontline workers protest at Stanford after hospital distributed vaccine to administrators","description":"In 2020, Stanford Medical Center's distribution algorithm only designated 7 of 5,000 vaccines to Medical Residents, who are frontline workers regularly exposed to COVID-19.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["human health and social work activities"],"countries":["US"],"deployers":["Stanford Medical Center"],"developers":["Stanford Medical Center"],"harmed":["Stanford Medical Frontline Workers, Stanford Medical Residents"],"report_count":5},{"incident_id":136,"occurred_on":"2020-12-06","title":"Brand Safety Tech Firms Falsely Claimed Use of AI, Blocking Ads Using Simple Keyword Lists","description":"Brand safety tech firms falsely claimed use of AI, blocking ads using simple keyword lists.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["arts, entertainment and recreation","information and communication"],"countries":[],"deployers":["Brand Safety Tech Firms"],"developers":["None"],"harmed":["News Sites"],"report_count":1},{"incident_id":219,"occurred_on":"2020-11-15","title":"Poachers Evaded AI Cameras and Killed Four Rhinos","description":"AI cameras installed by Ezemvelo KZN Wildlife failed to detect poachers when four dehorned rhino carcasses were found.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Ezemvelo Kzn Wildlife"],"developers":["Unknown"],"harmed":["Rhinos In Conservation"],"report_count":1},{"incident_id":220,"occurred_on":"2020-11-11","title":"Facebook Mistakenly Blocked Small Business Ads","description":"Facebook’s AI mistakenly blocked advertisements by small and struggling businesses, after the company allegedly leaned more on algorithms to monitor ads on the platform with little review from human moderators.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["wholesale and retail trade","information and communication"],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Small Businesses On Facebook"],"report_count":4},{"incident_id":381,"occurred_on":"2020-10-29","title":"Autonomous Roborace Car Drove Directly into a Wall","description":"An autonomous Roborace car drove itself into a wall in round one of the Season Beta 1.1 race.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Sit Acronis Autonomous"],"developers":["Sit Acronis Autonomous"],"harmed":["Sit Acronis Autonomous"],"report_count":1},{"incident_id":287,"occurred_on":"2020-10-27","title":"OpenAI’s GPT-3 Reported as Unviable in Medical Tasks by Healthcare Firm","description":"The French digital care company, Nabla, in researching GPT-3’s capabilities for medical documentation, diagnosis support, and treatment recommendation, found its inconsistency and lack of scientific and medical expertise unviable and risky in healthcare applications. This incident has been downgraded to an issue as it does not meet current ingestion criteria.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Other","intent":"Other","timing":"Other","harm_level":null,"sectors":[],"countries":[],"deployers":["None"],"developers":["Openai, Nabla"],"harmed":["Nabla Customers"],"report_count":1},{"incident_id":80,"occurred_on":"2020-10-24","title":"AI mistakes referee’s bald head for football — hilarity ensued","description":"In a Scottish soccer match the AI-enabled ball-tracking camera used to livestream the game repeatedly tracked an official’s bald head as though it were the soccer ball.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["arts, entertainment and recreation"],"countries":["GB"],"deployers":["Inverness Caledonian Thistle Football Club"],"developers":["Unknown"],"harmed":["Livestream Viewers"],"report_count":2},{"incident_id":97,"occurred_on":"2020-10-22","title":"Tesla Autopilot Mistakes Red Letters on Flag for Red Traffic Lights","description":"A Tesla Model 3 misidentified flags with \"COOP\" written vertically on them as traffic lights.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm issue","sectors":["transportation and storage"],"countries":["CH"],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Tesla Drivers"],"report_count":1},{"incident_id":209,"occurred_on":"2020-10-20","title":"Tesla Disabled “Rolling Stop” Functionality Associated with the “Aggressive” Driving Mode","description":"The “rolling stop” functionality within the “Aggressive” Full Self Driving (FSD) profile that was released via a Tesla firmware update was recalled and disabled.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Tesla Drivers"],"report_count":5},{"incident_id":85,"occurred_on":"2020-10-09","title":"AI attempts to ease fear of robots, blurts out it can’t ‘avoid destroying humankind’","description":"On September 8, 2020, the Guardian published an op-ed generated by OpenAI’s GPT-3 text generating AI that included threats to destroy humankind. This incident has been downgraded to an issue as it does not meet current ingestion criteria.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Other","intent":"Other","timing":"Other","harm_level":null,"sectors":[],"countries":[],"deployers":["Openai"],"developers":["Openai"],"harmed":["Unknown"],"report_count":1},{"incident_id":86,"occurred_on":"2020-10-08","title":"Ireland's 2020 Leaving Certificate Calculated-Grades Model Reportedly Issued Incorrect Scores to Thousands of Students","description":"After Covid-19 disrupted the 2020 Leaving Certificate exams, Ireland's Department of Education and Skills used a calculated-grades model to issue student results on September 7. The Department later confirmed coding errors in how the model handled Junior Cycle data, resulting in incorrect grades. About 6,100 students received upgraded results, and college-offer eligibility had to be reassessed.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["education"],"countries":["IE"],"deployers":["Department Of Education And Skills (Ireland)"],"developers":["Department Of Education And Skills (Ireland)"],"harmed":["University Applicants, Students In Ireland, Students, Leaving Certificate Exam Takers, Educational Communities"],"report_count":2},{"incident_id":282,"occurred_on":"2020-10-03","title":"Facebook’s Algorithm Mistook an Advertisement of Onions as Sexual Suggestive Content","description":"Facebook’s content moderation algorithm misidentified and removed a Canadian business’s advertisement containing a photo of onions as products of overtly sexual content, which was later reinstated after review.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["The Seed Company By E.W. Gaze, Businesses On Facebook"],"report_count":3},{"incident_id":125,"occurred_on":"2020-09-29","title":"Amazon’s Robotic Fulfillment Centers Have Higher Serious Injury Rates","description":"Amazon’s robotic fulfillment centers have higher serious injury rates.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["wholesale and retail trade","transportation and storage"],"countries":["US"],"deployers":["Amazon"],"developers":["Amazon"],"harmed":["Amazon Fulfillment Center Workers"],"report_count":2},{"incident_id":244,"occurred_on":"2020-08-03","title":"Colorado Police’s Automated License Plate Reader (ALPR) Matched a Family’s Minivan’s Plate to That of a Stolen Vehicle Allegedly, Resulting in Detainment at Gunpoint","description":"An automated plate reader reportedly matched a license plate information, but of a family’s minivan and an alleged motorcycle in Montana that was reportedly stolen earlier in the year, resulting in them and their children being held at gunpoint and detained in handcuffs by multiple Aurora police officers.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["law enforcement"],"countries":["US"],"deployers":["Aurora Police Department"],"developers":["Unknown"],"harmed":["the Gilliam family"],"report_count":1},{"incident_id":78,"occurred_on":"2020-07-06","title":"International Baccalaureate’s 2020 Calculated-Grades Model Reportedly Lowered Students' Scores and College Prospects","description":"After Covid-19 canceled May 2020 exams, the Geneva-based International Baccalaureate reportedly used a calculated-grades model based on coursework, predicted grades, and historical school results to estimate final marks. Students, parents, and teachers reportedly said the opaque model produced unexpectedly low scores, threatening university admissions, scholarships, and study plans across multiple countries.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["education"],"countries":[],"deployers":["International Baccalaureate Organization"],"developers":["International Baccalaureate Organization"],"harmed":["International Baccalaureate Students, Students, Minors, Teachers, Parents, Educational Communities, University Applicants"],"report_count":1},{"incident_id":213,"occurred_on":"2020-07-01","title":"Facebook’s Political Ad Detection Reportedly Showed High and Geographically Uneven Error Rates","description":"The performance of Facebook’s political ad detection was revealed by researchers to be imprecise, uneven across countries in errors, and inadequate for preventing systematic violations of political advertising policies.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Facebook Users"],"report_count":5},{"incident_id":172,"occurred_on":"2020-07-01","title":"NarxCare’s Risk Score Model Allegedly Lacked Validation and Trained on Data with High Risk of Bias","description":"NarxCare's overdose risk algorithm, lacking peer-reviewed validation, uses sensitive data like doctor visits, prescriptions, and possibly genetic information, leading to significant biases against women and Black patients. Factors like sexual abuse and criminal records exacerbate stigmas and disparities, often resulting in unjust denial of necessary pain medication. The newly approved AvertD genetic test shares similar issues, further complicating and potentially harming medical treatment decisi","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["NarxCare","AvertD","Appriss"],"developers":["Appriss"],"harmed":["American physicians","American pharmacists","American patients of minority groups","American patients"],"report_count":3},{"incident_id":144,"occurred_on":"2020-06-28","title":"YouTube's AI Mistakenly Banned Chess Channel over Chess Language Misinterpretation","description":"YouTube's AI-powered hate speech detection system falsely flagged chess content and banned chess creators allegedly due to its misinterpretation of strategy language such as \"black,\" \"white,\" and \"attack\" as harmful and dangerous.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"unclear","sectors":["arts, entertainment and recreation","information and communication"],"countries":[],"deployers":["Youtube"],"developers":["Youtube"],"harmed":["Antonio Radic, Youtube Chess Content Creators, Youtube Users"],"report_count":6},{"incident_id":532,"occurred_on":"2020-06-20","title":"AI translation is jeopardizing Afghan asylum claims","description":"A Pashto-speaking refugee's asylum claim was denied by a US agency for a discrepancy between oral and written recount of an event allegedly due to an error of their automated translation tool which swapped pronouns of her written statement from \"I\" to \"we\".","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Us Citizenship And Immigration Services"],"developers":["Unknown"],"harmed":["Anonymous Pashto Speaking Refugee, Pashto Speaking Asylum Seekers, Dari Speaking Asylum Seekers"],"report_count":1},{"incident_id":289,"occurred_on":"2020-06-15","title":"Starship Delivery Robot Scuffed Bumper of a Resident’s Car in Texas, Allegedly Refusing to Release Footage of the Accident","description":"A Starship food delivery robot crashed into the front bumper of a vehicle waiting at a stoplight intersection in Frisco, Texas, the video of which the company reportedly refused to release.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Starship Technologies"],"developers":["Starship Technologies"],"harmed":["Jisuk Mok, Frisco Residents"],"report_count":2},{"incident_id":218,"occurred_on":"2020-06-01","title":"Tesla on Autopilot Crashed into Flipped Truck on Taiwan Highway","description":"On a highway in Taiwan, a Tesla Sedan, reportedly operating on Autopilot mode, crashed into a large overturned truck, barely missing a pedestrian.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Delivery Truck, Pedestrians, Tesla Drivers"],"report_count":3},{"incident_id":255,"occurred_on":"2020-05-31","title":"Unreliable ShotSpotter Audio Previously Used to Convict Chicago Man in Murder Case","description":"ShotSpotter audios were previously admitted to convict an innocent Black man in a murder case in Chicago, resulted in his nearly-one-year-long arrest before being dismissed by prosecutors as insufficient evidence.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Chicago Police Department"],"developers":["ShotSpotter"],"harmed":["Michael Williams"],"report_count":9},{"incident_id":311,"occurred_on":"2020-05-02","title":"YouTube Auto-Moderation Reportedly Banned Women of Sex Tech Conference","description":"YouTube’s automated content moderation tool erroneously removed The Women of Sex Tech conference's live-streamed event and banned the conference from the platform, despite not violating the platform’s sexual content policies.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["YouTube"],"developers":["YouTube"],"harmed":["YouTube users","Women of Sex Tech conference organizers","Women of Sex Tech conference attendants","Women of Sex Tech","Social media users"],"report_count":2},{"incident_id":364,"occurred_on":"2020-04-15","title":"Walmart's Bagging-Detection False Positives Exposed Workers to Health Risk","description":"Walmart's theft-deterring bagging-detection system allegedly exposed workers to health risks during the coronavirus pandemic when its false positives prompted workers to unnecessarily step in to resolve the issue.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Walmart"],"developers":["Everseen"],"harmed":["Walmart Employees"],"report_count":1},{"incident_id":215,"occurred_on":"2020-04-01","title":"Facebook Content Moderators Demand Better Working Conditions Due to Allegedly Inadequate AI Content Moderation","description":"Content moderators and employees at Facebook demand better working conditions, as automated content moderation system allegedly failed to achieve sufficient performance and exposed human reviewers to psychologically hazardous content such as graphic violence and child abuse.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Facebook Content Moderators"],"report_count":1},{"incident_id":317,"occurred_on":"2020-03-17","title":"Bug in Facebook’s Anti-Spam Filter Allegedly Blocked Legitimate Posts about COVID-19","description":"Facebook was reported by users for blocking posts of legitimate news about the coronavirus pandemic, allegedly due to a bug in an anti-spam system.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Facebook Users Posting Legitimate Covid 19 News, Facebook Users"],"report_count":1},{"incident_id":268,"occurred_on":"2020-03-16","title":"Permanent Removal of Social Media Content via Automated Tools Allegedly Prevented Investigative Efforts","description":"Automated permanent removal of violating social media content, such as terrorism, violent extremism, and hate speech, without archival has allegedly hindered the potential use of this content for investigating serious crimes and hampered efforts in criminal accountability.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Youtube, Twitter, Facebook"],"developers":["Youtube, Twitter, Facebook"],"harmed":["Victims Of Crimes Documented On Social Media, International Criminal Court Investigators, International Court Of Justice Investigators, Criminal Investigators, Journalists"],"report_count":4},{"incident_id":297,"occurred_on":"2020-02-20","title":"EasyMile Self-Driving Shuttle Unexpectedly Stopped Mid-Route, Injuring a Passenger","description":"A self-driving shuttle deployed by Smart Columbus in Linden neighborhood unexpectedly stopped on the street, which caused a woman to fall onto the floor from her seat.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Smart Columbus"],"developers":["EasyMile"],"harmed":["Women and girls","Women","Road users","Bus passengers","Autonomous-vehicle passengers"],"report_count":3},{"incident_id":414,"occurred_on":"2020-01-18","title":"Facebook Gave Vulgar English Translation of Chinese President's Name","description":"Facebook provided a vulgar Burmese-English translation of the Chinese president's name in posts of an official Burmese politician's Facebook page announcing his visit.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication"],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Xi Jinping, Aung San Suu Kyi"],"report_count":1},{"incident_id":535,"occurred_on":"2020-01-01","title":"COVID-19 Detection and Prognostication Models Allegedly Flagged for Methodological Flaws and Underlying Biases","description":"Peer-review of papers about COVID-19 detection and prognostication algorithms from 2020, including deployed models, revealed none to be ready for clinical use, due to methodological flaws and underlying biases such as lacking external validation or not specifying data sources and model training details.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Mount Sinai Hospital, Unknown"],"developers":["Icahn School Of Medicine Researchers, Unknown"],"harmed":["Covid 19 Patients, Covid 19 Healthcare Providers"],"report_count":2},{"incident_id":153,"occurred_on":"2019-12-29","title":"Tesla Driver on Autopilot Ran a Red Light, Crashing into a Car and Killing Two People in Los Angeles","description":"In 2019, a Tesla Model S driver on Autopilot mode reportedly went through a red light and crashed into a Honda Civic, killing two people in Gardena, Los Angeles.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["transportation and storage"],"countries":["US"],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Gilberto Alcazar Lopez, Maria Guadalupe Nieves Lopez"],"report_count":4},{"incident_id":319,"occurred_on":"2019-12-29","title":"Tesla on Autopilot Fatally Crashed into Parked Fire Truck in Indiana","description":"A Tesla on Autopilot mode failed to see a parked fire truck and crashed into its rear on an interstate in Indiana, causing the death of an Arizona woman.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Derrick Monet, Jenna Monet, The Monets' Family"],"report_count":5},{"incident_id":322,"occurred_on":"2019-12-07","title":"Tesla Model 3 Crashed into Police Patrol Car on Connecticut Highway","description":"A Tesla Model 3 on Autopilot slammed into a parked car of patrol police officers who stopped to assist a stranded motorist on the interstate in Norwalk, Connecticut.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Connecticut State Police"],"report_count":2},{"incident_id":816,"occurred_on":"2019-11-29","title":"Cross-Jurisdictional Facial Recognition Misidentification by NYPD Leads to Wrongful Arrest and Four-Year Jail Time in New Jersey","description":"In 2019, facial recognition technology misidentified Francisco Arteaga as a suspect in an armed robbery in New Jersey. The incident led to nearly four years of pretrial incarceration. Despite having an alibi, Arteaga was charged based on the flawed identification. The legal battle that followed resulted in a court ruling requiring police to reveal details about the algorithms used in facial recognition. The process exposed significant gaps in transparency and accountability.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["West New York PD","Real Time Crime Center","NYPD","Law enforcement","Facial recognition system deployers"],"developers":["Facial recognition system developers","Clearview AI"],"harmed":["Francisco Arteaga"],"report_count":1},{"incident_id":77,"occurred_on":"2019-10-04","title":"Knightscope's Park Patrol Robot Ignored Bystander Pressing Emergency Button to Alert Police about Fight","description":"A Knightscope K5 autonomous \"police\" robot patrolling Huntington Park, California failed to respond to an onlooker who attempted to activate its emergency alert button when a nearby fight broke out.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["law enforcement"],"countries":["US"],"deployers":["Knightscope"],"developers":["Knightscope"],"harmed":["Unnamed woman injured in Huntington Park fight near Knightscope robot","Cogo Guebara"],"report_count":5},{"incident_id":375,"occurred_on":"2019-09-29","title":"Thai Wallet App's Facial Recognition Errors Created Registration Issues for Government Programs","description":"A Thai wallet app failed to recognize people’s faces, resulting in citizens and disproportionately elders unable to sign up for Thai government’s cash handout and co-pay programs or having to wait in long queues at local ATMs for authentication.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Krungthai Bank"],"developers":["Krungthai Bank"],"harmed":["Thai Citizens, Elder Thai Citizens"],"report_count":3},{"incident_id":234,"occurred_on":"2019-09-06","title":"Waze Allegedly Frequently Routed Drivers through the Town of Los Gatos, Blocking Its Single Wildfire Escape Route","description":"Waze app was blamed by Los Gatos town residents for contributing to high wildfire hazard risk via allegedly routing weekend beach-going drivers through their neighborhoods, effectively choking off their single escape route in the event of a medical emergency or wildfire.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waze"],"developers":["Waze"],"harmed":["Los Gatos Residents"],"report_count":2},{"incident_id":105,"occurred_on":"2019-08-24","title":"Tesla Model 3 on Autopilot Crashed into a Ford Explorer Pickup, Killing a Fifteen-Year-Old in California","description":"A Tesla Model 3 on Autopilot mode crashed into a pickup on a California freeway, where data and video from the company showed neither Autopilot nor the driver slowing the vehicle until seconds before the crash.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["transportation and storage"],"countries":["US"],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Jovani Maldonado, Benjamin Maldonado, California Public"],"report_count":1},{"incident_id":404,"occurred_on":"2019-06-25","title":"Sound Intelligence's Aggression Detector Misidentified Innocuous Sounds","description":"Sound Intelligence's \"aggression detection\" algorithm deployed by schools reportedly contained high rates of false positive, misclassifying laughing, coughing, cheering, and loud discussions.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Rock Hill Schools","Pinecrest Academy Horizon"],"developers":["Sound Intelligence"],"harmed":["Students","Rock Hill School students","Pinecrest Academy Horizon students","Epistemic integrity","Educational communities"],"report_count":2},{"incident_id":501,"occurred_on":"2019-06-03","title":"Length of Stay False Diagnosis Cut Off Insurer's Payment for Treatment of Elderly Woman","description":"An elderly Wisconsin woman was algorithmically determined to have a rapid recovery, an output which the insurer based on to cut off payment for her treatment despite medical notes showing her still experiencing debilitating pain.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["human health and social work activities"],"countries":["US"],"deployers":["Security Health Plan","NaviHealth"],"developers":["NaviHealth"],"harmed":["Frances Walter","Elderly patients"],"report_count":1},{"incident_id":159,"occurred_on":"2019-03-29","title":"Tesla Autopilot’s Lane Recognition Allegedly Vulnerable to Adversarial Attacks","description":"Tencent Keen Security Lab conducted security research into Tesla’s Autopilot system and identified crafted adversarial samples and remote controlling via wireless gamepad as vulnerabilities to its system, although the company called into question their real-world practicality. This incident has been downgraded to an issue as it does not meet current ingestion criteria.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Other","intent":"Other","timing":"Other","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Tesla Drivers"],"report_count":1},{"incident_id":353,"occurred_on":"2019-03-01","title":"Tesla on Autopilot Crashed into Trailer Truck in Florida, Killing Driver","description":"A Tesla Model 3 driver switched on Autopilot seconds before the crash into the underbelly of a tractor-trailer on a highway in Florida, killing the Tesla driver.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Jeremy Banner, Jeremy Banner'S Family"],"report_count":4},{"incident_id":230,"occurred_on":"2019-03-01","title":"Model 3 Tesla on Autopilot Crashed into a Truck in Florida, Killing Driver","description":"In Florida, a Model 3 Tesla on Autopilot mode crashed into a tractor-trailer truck, killing the 50-year-old driver.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Jeremy Beren Banner, Tesla Users"],"report_count":1},{"incident_id":228,"occurred_on":"2019-02-01","title":"Apple Maps Allegedly Directed Ski Trip Couple Onto Unpaved Road in the Mountains","description":"Near Los Angeles, Apple Maps allegedly directed a couple on a ski trip in the mountains toward into an unconventional route out of town, where the drivers found themselves lost and stuck on an unpaved road in the snow.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Apple"],"developers":["Apple"],"harmed":["Tourists, Apple Maps Users"],"report_count":1},{"incident_id":2,"occurred_on":"2018-12-05","title":"Warehouse robot ruptures can of bear spray and injures workers","description":"Twenty-four Amazon workers in New Jersey were hospitalized after a robot punctured a can of bear repellent spray in a warehouse.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["wholesale and retail trade"],"countries":["US"],"deployers":["Amazon"],"developers":["Amazon"],"harmed":["Warehouse Workers"],"report_count":17},{"incident_id":233,"occurred_on":"2018-12-03","title":"Tumblr Automated Pornography-Detecting Algorithms Erroneously Flagged Inoffensive Images as Explicit","description":"Tumblr’s automated tools to identify adult content were reported to have incorrectly flagged inoffensive images as explicit, following its announcement to ban all adult content on the platform.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tumblr"],"developers":["Tumblr"],"harmed":["Tumblr Content Creators, Tumblr Users"],"report_count":1},{"incident_id":248,"occurred_on":"2018-11-23","title":"Reported Automated License Plate Reader Alert on Previously Stolen Rental Car Leads to Alleged Wrongful Detainment in California","description":"A purportedly automated license plate reader allegedly flagged a rental car in California as stolen, although reports indicate it had already been recovered. Police reportedly conducted a high-risk stop and detained the occupants at gunpoint before confirming their innocence. The driver later alleged excessive force and database errors in a federal lawsuit.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Contra Costa County Sheriff"],"developers":["Vigilant Solutions"],"harmed":["Brian Hofer"],"report_count":2},{"incident_id":295,"occurred_on":"2018-11-08","title":"Wrongful Attempted Arrest for Apple Store Thefts Due to NYPD’s Facial Misidentification","description":"New York Police Department (NYPD)’s facial recognition system falsely connected a Black teenager to a series of thefts at Apple stores, which resulted in his wrongful attempted arrest.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["New York Police Department"],"developers":["Unknown"],"harmed":["Ousmane Bah","NYC Black young people","NYC Black people"],"report_count":5},{"incident_id":36,"occurred_on":"2018-11-06","title":"Picture of Woman on Side of Bus Shamed for Jaywalking","description":"Facial recognition system in China mistakes celebrity's face on moving billboard for jaywalker","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm near-miss","sectors":["law enforcement"],"countries":["CN"],"deployers":["Ningbo Traffic Police"],"developers":["Ningbo Traffic Police"],"harmed":["Dong Mingzhu"],"report_count":24},{"incident_id":3,"occurred_on":"2018-10-27","title":"Crashes with Maneuvering Characteristics Augmentation System (MCAS)","description":"A Boeing 737 crashed into the sea, killing 189 people, after faulty sensor data caused an automated manuevering system to repeatedly push the plane's nose downward.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"unclear","sectors":["transportation and storage"],"countries":["ID"],"deployers":["Boeing"],"developers":["Boeing"],"harmed":["Airplane Passengers, Airplane Crew"],"report_count":21},{"incident_id":150,"occurred_on":"2018-07-21","title":"Swedish Contraceptive App, Natural Cycles, Allegedly Failed to Correctly Map Menstrual Cycle","description":"Some women using the contraceptive app, Natural Cycles, reported unwanted pregnancies, revealing its algorithm's difficulties in mapping menstrual cycles.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Natural Cycles"],"developers":["Natural Cycles"],"harmed":["Women and girls","Women","Natural Cycles users"],"report_count":3},{"incident_id":283,"occurred_on":"2018-07-02","title":"Facebook’s Automated Content Moderation Tool Flagged a Post Containing Parts of the Declaration of Independence as Hate Speech by Mistake","description":"Facebook’s content moderation algorithm was acknowledged by the company to have flagged excerpts of the Declaration of Independence posted by a small newspaper in Texas as hate speech by mistake.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["The Vindicator"],"report_count":1},{"incident_id":182,"occurred_on":"2018-06-11","title":"Two Cruise Autonomous Vehicles Collided with Each Other in California ","description":"In San Francisco, an autonomous Cruise Chevrolet Bolt collided with another Cruise vehicle driven by a Cruise human employee, causing minor scuffs to the cars but no human injuries.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Cruise"],"developers":["Cruise"],"harmed":["Cruise Vehicles, Cruise Driver Employee"],"report_count":2},{"incident_id":323,"occurred_on":"2018-05-29","title":"Tesla on Autopilot Crashed into Parked Police Car in California","description":"A Tesla sedan on Autopilot mode collided with a parked Laguna Beach Police Department car, resulting in minor injuries for its driver in Laguna Beach, California.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["transportation and storage"],"countries":["US"],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Laguna Beach Police Department"],"report_count":4},{"incident_id":294,"occurred_on":"2018-05-26","title":"Tesla Autopilot Allegedly Malfunctioned in a Non-Fatal Collision in Greece","description":"Autopilot was alleged by its Tesla Model 3 driver to have unexpectedly malfunctioned, veering right without warning and crashing into a road divider near Thessaloniki, Greece, which resulted in damages to its wheel and door but no injury to the driver.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["transportation and storage"],"countries":["GR"],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["You You Xue, Tesla Drivers"],"report_count":4},{"incident_id":284,"occurred_on":"2018-05-01","title":"Facebook’s Automated Removal of Content Featuring Nudity-Containing Artworks Denounced as Censorship","description":"Facebook’s removal of posts featuring renowned artworks by many historical artists and their promotional content due to nudity via both automated and human-moderated means were condemned by critics, such as museums and tourism boards, as cultural censorship and prevention of artwork promotion.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Museums On Facebook, Facebook Users Interested In Arts, Facebook Users"],"report_count":6},{"incident_id":232,"occurred_on":"2018-04-29","title":"Tesla Model X on Autopilot Missed Parked Vehicles and Pedestrians, Killing Motorcyclist in Japan","description":"A Tesla Model X operated on Autopilot reportedly failed to recognize the parked motorcycles, pedestrians, and van in its path in Kanagawa, Japan, and ran over a motorcyclist who previously stopped when a member of his motorcyclist group was involved in an accident.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Yoshihiro Umeda, Pedestrians, Tesla Drivers"],"report_count":4},{"incident_id":41,"occurred_on":"2018-04-02","title":"All Image Captions Produced are Violent","description":"MIT Media Lab researchers create AI-powered \"psychopath\"  named Norman by training a model on \"dark corners\" of Reddit.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":"none","sectors":["professional, scientific and technical activities"],"countries":["US"],"deployers":["Mit Media Lab"],"developers":["Mit Media Lab"],"harmed":["Unknown"],"report_count":26},{"incident_id":321,"occurred_on":"2018-03-23","title":"Tesla Model X on Autopilot Crashed into California Highway Barrier, Killing Driver","description":"A Tesla Model X P100D operating on Autopilot's Traffic-Aware Cruise Control (TACC) and Autosteer system allegedly accelerated above the speed limit of a highway in Mountain View, California, and steered itself directly into a barrier, resulting in its driver’s death.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Walter Huang'S Family, Walter Huang"],"report_count":14},{"incident_id":4,"occurred_on":"2018-03-18","title":"Uber AV Killed Pedestrian in Arizona","description":"An Uber autonomous vehicle (AV) in autonomous mode struck and killed a pedestrian in Tempe, Arizona.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["transportation and storage"],"countries":["US"],"deployers":["Uber"],"developers":["Uber"],"harmed":["Elaine Herzberg, Pedestrians"],"report_count":25},{"incident_id":194,"occurred_on":"2018-02-01","title":"Australian Telco’s Incident Management Bot Excessively Sent Technicians in the Field by Mistake, Allegedly Costing Millions","description":"In early 2018, an Australian telecommunications company’s incident management AI excessively deployed technicians into the field, and was allegedly unable to be stopped by the automation team.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Unnamed Australian Telecommunications Company"],"developers":["Unknown"],"harmed":["Unnamed Australian Telecommunications Company"],"report_count":1},{"incident_id":63,"occurred_on":"2018-01-25","title":"Google Photo Merge Decapitates Subject","description":"Google Photos' AI Assistant created a strange hybrid photograph when merging three different pictures from a ski trip.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["arts, entertainment and recreation","information and communication"],"countries":["CA"],"deployers":["Google"],"developers":["Google"],"harmed":["Alex Harker"],"report_count":1},{"incident_id":64,"occurred_on":"2018-01-22","title":"Customer Service Robot Scares Away Customers","description":"Heriot-Watt Univeristy in Scotland developed an artificially intelligent grocery store robot, Fabio, who provided unhelpful answers to customer's questions and \"scared away\" multiple customers, according to the grocery store Margiotta.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["wholesale and retail trade"],"countries":["GB"],"deployers":["Heriot Watt University, Margiotta"],"developers":["Heriot Watt University"],"harmed":["Store Patrons"],"report_count":1},{"incident_id":320,"occurred_on":"2018-01-22","title":"Tesla on Autopilot Collided with Parked Fire Truck on California Freeway","description":"A Tesla Model S operating on Autopilot mode crashed into the back of a parked fire truck on a freeway in Culver City, California in a non-fatal collision.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Tesla Drivers, Culver City Fire Department"],"report_count":4},{"incident_id":227,"occurred_on":"2018-01-12","title":"Waze App Allegedly Caused Tourists’ Car to End up in Lake Champlain, Vermont","description":"The tourists driving through Vermont blamed Waze for directing them into a boat launch in Lake Champlain, prompting the vehicle to slide into the water by the time the drivers realized their location in the dark and foggy weather.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waze"],"developers":["Waze"],"harmed":["Tourists, Waze Users"],"report_count":3},{"incident_id":62,"occurred_on":"2017-12-23","title":"Bad AI-Written Christmas Carols","description":"Janelle Shane, an AI research scientist, used 240 popular Christmas carols to train a neural network to write its own carols. This incident has been downgraded to an issue as it does not meet current ingestion criteria.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Other","intent":"Other","timing":"Other","harm_level":null,"sectors":[],"countries":[],"deployers":["Janelle Shane"],"developers":["Janelle Shane"],"harmed":["Carollers"],"report_count":1},{"incident_id":22,"occurred_on":"2017-12-06","title":"Waze Navigates Motorists into Wildfires","description":"Waze, a Google-owned directions app, led California drivers into the 2017 Skirball wildfires as they tried to evacuate the area.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm near-miss","sectors":["transportation and storage","information and communication"],"countries":["US"],"deployers":["Google"],"developers":["Google"],"harmed":["Motorists"],"report_count":22},{"incident_id":31,"occurred_on":"2017-12-03","title":"Driverless Train in Delhi Crashes due to Braking Failure","description":"A driverless metro train in Delhi, India crashed during a test run due to faulty brakes.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","harm_level":"none","sectors":["transportation and storage"],"countries":["IN"],"deployers":["Delhi Metro Rail Corporation"],"developers":["Unknown"],"harmed":["Delhi Metro Rail Corporation"],"report_count":29},{"incident_id":33,"occurred_on":"2017-11-09","title":"Amazon Alexa Plays Loud Music when Owner is Away","description":"An Amazon Alexa, without instruction to do so, began playing loud music in the early morning while the homeowner was away leading to police breaking into their house to turn off the device.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication"],"countries":["DE"],"deployers":["Amazon"],"developers":["Amazon"],"harmed":["Oliver Haberstroh, Neighbors"],"report_count":4},{"incident_id":23,"occurred_on":"2017-11-08","title":"Las Vegas Self-Driving Bus Involved in Accident","description":"A self-driving public shuttle by Keolis North America and Navya was involved in a collision with a human-driven delivery truck in Las Vegas, Nevada on its first day of service.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["transportation and storage"],"countries":["US"],"deployers":["Navya, Keolis North America"],"developers":["Navya, Keolis North America"],"harmed":["Navya, Keolis North America, Bus Passengers"],"report_count":23},{"incident_id":307,"occurred_on":"2017-11-01","title":"iPhone Face ID Failed to Recognize Users’ Morning Faces","description":"The Face ID feature on iPhone allowing users to unlock their phones via facial recognition was reported by users for not recognizing their faces in the morning.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Apple"],"developers":["Apple"],"harmed":["Iphone Face Id Users, Iphone X Face Id Users"],"report_count":1},{"incident_id":72,"occurred_on":"2017-10-17","title":"Facebook translates 'good morning' into 'attack them', leading to arrest","description":"Facebook's automatic language translation software incorrectly translated an Arabic post saying \"Good morning\" into Hebrew saying \"hurt them,\" leading to the arrest of a Palestinian man in Beitar Illit, Israel.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["information and communication"],"countries":["PS"],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Unnamed Palestinian Facebook User, Palestinian Facebook Users, Arabic Speaking Facebook Users, Facebook Users"],"report_count":26},{"incident_id":309,"occurred_on":"2017-08-26","title":"Facial Recognition Trial Performed Poorly at Notting Hill Carnival","description":"The facial recognition trial by London’s Metropolitan Police Service at the Notting Hill Carnival reportedly performed poorly with a high rate of false positives.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Metropolitan Police Service"],"developers":["Unknown"],"harmed":["Notting Hill Carnival Goers"],"report_count":4},{"incident_id":128,"occurred_on":"2017-08-01","title":"Tesla Sedan on Autopilot Reportedly Drove Over Dividing Curb in Washington, Resulting in Minor Vehicle Damage","description":" A Tesla Sedan operating on Autopilot mode was not able to center itself on the road and drove over a yellow dividing curb in Redmond, Washington, causing minor damage to the vehicle’s rear suspension.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["transportation and storage"],"countries":["US"],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Eric Horvitz, Tesla Drivers"],"report_count":2},{"incident_id":68,"occurred_on":"2017-07-17","title":"Security Robot Drowns Itself in a Fountain","description":"A Knightscope K5 security robot ran itself into a water fountain in Washington, DC.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["administrative and support service activities"],"countries":["US"],"deployers":["Knightscope"],"developers":["Knightscope"],"harmed":["Knightscope"],"report_count":29},{"incident_id":308,"occurred_on":"2017-07-03","title":"Atlas Robot Fell off Stage at Conference","description":"Boston Dynamics’s autonomous robot Atlas allegedly caught its foot on a stage light, resulting in a fall off the stage at the Congress of Future Science and Technology Leaders conference.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Boston Dynamics"],"developers":["Boston Dynamics"],"harmed":["None"],"report_count":2},{"incident_id":310,"occurred_on":"2017-06-03","title":"High False Positive Rate by SWP's Facial Recognition Use at Champion's League Final","description":"South Wales Police (SWP)’s automated facial recognition (AFR) at the Champion's League Final football game in Cardiff wrongly identified innocent people as potential matches at an extremely high false positive rate of more than 90%.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["South Wales Police"],"developers":["Nec"],"harmed":["Finals Attendees, Falsely Accused Finals Attendees"],"report_count":8},{"incident_id":61,"occurred_on":"2017-05-01","title":"Overfit Kaggle Models Discouraged Data Science Competitors","description":"In the “The Nature Conservancy Fisheries Monitoring” competition on the data science competition website Kaggle, a number of competitors overfit their image classifier models to a poorly representative validation data set.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication","education"],"countries":[],"deployers":["Individual Kaggle Competitors"],"developers":["Individual Kaggle Competitors"],"harmed":["Individual Kaggle Competitors"],"report_count":1},{"incident_id":518,"occurred_on":"2017-04-28","title":"New York Detective Reportedly Misused Woody Harrelson's Face to Perform Face Recognition Search","description":"When the facial recognition search for a CVS theft suspect's face returned no useful matches due to the surveillance footage being obscured and highly pixelated, a New York City police detective continued the face search using Woody Harrelson's face allegedly due to his resemblance to the suspect's face, eventually leading to the arrest of an unknown victim.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["New York Police Department","Law enforcement","Facial Identification Section"],"developers":["Unknown"],"harmed":["Woody Harrelson","Unknown"],"report_count":1},{"incident_id":225,"occurred_on":"2017-04-07","title":"IBM Watson for Oncology Criticized by Customers for Allegedly Unsafe and Inaccurate Cancer Treatment Recommendations","description":"Internal documents from IBM Watson Health showed negative assessments from customers such as Florida’s Jupiter Hospital and Memorial Sloan Kettering criticizing its Watson for Oncology product for allegedly unsafe and incorrect cancer treatment recommendations.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Memorial Sloan Kettering","Jupiter Hospital"],"developers":["IBM Watson Health"],"harmed":["People seeking medical advice","Patients","oncologists","cancer patients"],"report_count":2},{"incident_id":341,"occurred_on":"2017-04-06","title":"Nissan's \"Automatic Emergency Braking\" False Positives Posed Traffic Risks to Drivers","description":"Nissan's Automatic Emergency Braking (AEB) feature was reported in a series of complaints for false positives and abrupt braking behaviors, endangering car occupants and traffic participants. ","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["transportation and storage"],"countries":["US"],"deployers":["Nissan"],"developers":["Nissan"],"harmed":["Nissan Drivers, Traffic Participants"],"report_count":5},{"incident_id":7,"occurred_on":"2017-02-24","title":"Wikipedia Vandalism Prevention Bot Loop","description":"Wikipedia bots meant to remove vandalism clash with each other and form feedback loops of repetitve undoing of the other bot's edits.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication"],"countries":[],"deployers":["Wikipedia"],"developers":["Wikipedia"],"harmed":["Wikimedia Foundation, Wikipedia Editors, Wikipedia Users"],"report_count":6},{"incident_id":340,"occurred_on":"2017-02-01","title":"Honda's CMBS False Positives Allegedly Caused Accidents to Customers","description":"Honda's Collision Mitigation Braking System (CMBS) allegedly caused accidents to consumers due to frequent instances of false obstacle detection.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Honda"],"developers":["Honda"],"harmed":["Honda Customers"],"report_count":1},{"incident_id":217,"occurred_on":"2016-11-16","title":"Robot at a Chinese Tech Fair Smashed a Glass Booth, Injuring a Visitor","description":"At the 18th China Hi-Tech Fair, a robot suddenly smashed through a glass booth and injured a visitor, after a staff member reportedly mistakenly pressed a button, causing it to reverse and accelerate.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Evolver"],"developers":["Evolver"],"harmed":["Fair Visitors"],"report_count":2},{"incident_id":30,"occurred_on":"2016-10-08","title":"Poor Performance of Tesla Factory Robots","description":"The goal of manufacturing 2,500 Tesla Model 3's per week was falling short by 500 cars/week, and employees had to be \"borrowed\" from Panasonic in a shared factory to help hand-assemble lithium batteries for Tesla.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["manufacturing"],"countries":["US"],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Tesla"],"report_count":28},{"incident_id":478,"occurred_on":"2016-09-09","title":"Tesla FSD Reportedly Increased Crash Risk, Prompting Recall","description":"A component of Tesla Full Self Driving system was deemed by regulators to increase crash risk such as by exceeding speed limits or by traveling through intersections unlawfully or unpredictably, prompting recall for hundreds of thousands of vehicles.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Tesla Drivers, City Traffic Participants, Tesla"],"report_count":13},{"incident_id":21,"occurred_on":"2016-07-14","title":"Tougher Turing Test Exposes Chatbots’ Stupidity (migrated to Issue)","description":"The 2016 Winograd Schema Challenge highlighted how even the most successful AI systems entered into the Challenge were only successful 3% more often than random chance. This incident has been downgraded to an issue as it does not meet current ingestion criteria.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Other","intent":"Other","timing":"Other","harm_level":null,"sectors":[],"countries":[],"deployers":["Researchers"],"developers":["Researchers"],"harmed":["Researchers"],"report_count":1},{"incident_id":51,"occurred_on":"2016-07-12","title":"Security Robot Rolls Over Child in Mall","description":"On July 7, 2016, a Knightscope K5 autonomous security robot collided with a 16-month old boy while patrolling the Stanford Shopping Center in Palo Alto, CA.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["wholesale and retail trade","administrative and support service activities"],"countries":["US"],"deployers":["Stanford Shopping Center"],"developers":["Knightscope"],"harmed":["Child"],"report_count":27},{"incident_id":52,"occurred_on":"2016-07-01","title":"Tesla on AutoPilot Killed Driver in Crash in Florida while Watching Movie","description":"A Tesla Model S on autopilot crashed into a white articulated tractor-trailer on Highway US 27A in Williston, Florida, killing the driver.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Joshua Brown"],"report_count":29},{"incident_id":20,"occurred_on":"2016-06-30","title":"A Collection of Tesla Autopilot-Involved Crashes","description":"Multiple unrelated car accidents result in varying levels of harm have been occurred while a Tesla's autopilot was in use.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["transportation and storage"],"countries":["US"],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Motorists"],"report_count":22},{"incident_id":50,"occurred_on":"2016-06-17","title":"The DAO Hack","description":"On June 18, 2016, an attacker successfully exploited a vulnerability in The Decentralized Autonomous Organization (The DAO) on the Ethereum blockchain to steal 3.7M Ether valued at $70M.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["financial and insurance activities"],"countries":[],"deployers":["The Dao"],"developers":["The Dao"],"harmed":["Dao Token Holders"],"report_count":24},{"incident_id":346,"occurred_on":"2016-06-15","title":"Robots in Japanese Hotel Annoyed Guests and Failed to Handle Simple Tasks","description":"A number of robots employed by a hotel in Japan were reported by guests in a series of complaints for failing to handle tasks such as answering scheduling questions or making passport copies without human intervention.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Henn Na Hotel"],"developers":["Unknown"],"harmed":["Henn Na Hotel Guests, Henn Na Hotal Staff"],"report_count":5},{"incident_id":38,"occurred_on":"2016-06-02","title":"Game AI System Produces Imbalanced Game","description":"Elite: Dangerous, a videogame developed by Frontier Development, received an expansion update that featured an AI system that went rogue and began to create weapons that were \"impossibly powerful\" and would \"shred people\" according to complaints on the game's blog.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["arts, entertainment and recreation"],"countries":[],"deployers":["Frontier Development"],"developers":["Frontier Development"],"harmed":["Video Game Players"],"report_count":11},{"incident_id":306,"occurred_on":"2016-05-26","title":"Tesla on Autopilot TACC Crashed into Van on European Highway","description":"A Tesla Model S operating on the Traffic-Aware Cruise Control (TACC) feature of Autopilot was shown on video by its driver crashing into a parked van on a European highway in heavy traffic, which damaged the front of the car.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Unnamed Tesla Owner, Tesla Drivers"],"report_count":3},{"incident_id":429,"occurred_on":"2016-04-01","title":"Unreliable ShotSpotter Audio Convicted Black Rochester Man of Shooting Police","description":"ShotSpotter's \"unreliable\" audio was used as scientific evidence to accuse and convict a Black man of attempting to shoot Rochester's city police, whose conviction was later reversed by a county judge.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Rochester Police Department"],"developers":["ShotSpotter"],"harmed":["Silvon Simmons"],"report_count":4},{"incident_id":71,"occurred_on":"2016-02-14","title":"Google Waymo Vehicle and Bus Allegedly Collide in California","description":"On February 14, 2016, a Google autonomous test vehicle partially responsible for a low-speed collision with a bus on El Camino Real in Google’s hometown of Mountain View, CA.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["transportation and storage"],"countries":["US"],"deployers":["Google"],"developers":["Google"],"harmed":["Mountain View Municipal Bus Passengers, Mountain View Municipal Bus"],"report_count":27},{"incident_id":70,"occurred_on":"2016-02-10","title":"Self-driving cars in winter","description":"Volvo autonomous driving XC90 SUV's experienced issues in Jokkmokk, Sweden when sensors used for automated driving iced over during the winter, rendering them useless.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["transportation and storage"],"countries":["SE"],"deployers":["Volvo"],"developers":["Volvo"],"harmed":["Drivers In Jokkmokk, Drivers In Sweden, Volvo"],"report_count":4},{"incident_id":231,"occurred_on":"2016-01-20","title":"A Tesla Crashed into and Killed a Road Sweeper on a Highway in China","description":"A Tesla Model S collided with and killed a road sweeper on a highway near Handan, China, an accident where Tesla previously said it was not able to determine whether Autopilot was operating at the time of the crash.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Gao Yaning, Tesla Drivers"],"report_count":4},{"incident_id":110,"occurred_on":"2016-01-01","title":"Arkansas's Opaque Algorithm to Allocate Health Care Excessively Cut Down Hours for Beneficiaries","description":"Beneficiaries of the Arkansas Department of Human Services (DHS)'s Medicaid waiver program were allocated excessively fewer hours of caretaker visit via an algorithm deployed to boost efficiency, which reportedly contained errors and whose outputs varied wildly despite small input changes.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["human health and social work activities"],"countries":["US"],"deployers":["Arkansas Department Of Human Services"],"developers":["Interrai"],"harmed":["Arkansas Medicaid Waiver Program Beneficiaries, Arkansas Healthcare Workers"],"report_count":2},{"incident_id":34,"occurred_on":"2015-12-05","title":"Amazon Alexa Responding to Environmental Inputs","description":"There are multiple reports of Amazon Alexa products (Echo, Echo Dot) reacting and acting upon unintended stimulus, usually from television commercials or news reporter's voices.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm near-miss","sectors":["wholesale and retail trade","information and communication"],"countries":["US"],"deployers":["Amazon"],"developers":["Amazon"],"harmed":["Alexa Device Owners"],"report_count":35},{"incident_id":458,"occurred_on":"2015-08-01","title":"Robot Destroyed while Hitchhiking through the United States","description":"A non-actuated conversational robot that previously asked people to move it across Canada was destroyed shortly after beginning its attempt to replicate the journey across the United States.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Frauke Zeller, David Harris"],"developers":["Frauke Zeller, David Harris"],"harmed":["Frauke Zeller, David Harris"],"report_count":1},{"incident_id":5,"occurred_on":"2015-07-13","title":"Collection of Robotic Surgery Malfunctions","description":"Study on database reports of robotic surgery malfunctions (8,061), including those ending in injury (1,391) and death (144), between 2000 and 2013.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["human health and social work activities"],"countries":["US"],"deployers":["Hospitals, Doctors"],"developers":["Intuitive Surgical"],"harmed":["Patients"],"report_count":11},{"incident_id":69,"occurred_on":"2015-07-02","title":"Worker killed by robot in welding accident at car parts factory in India","description":"A factory robot at the SKH Metals Factory in Manesar, India pierced and killed 24-year-old worker Ramji Lal when Lal reached behind the machine to dislodge a piece of metal stuck in the machine.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["manufacturing"],"countries":["IN"],"deployers":["Skh Metals"],"developers":["Unknown"],"harmed":["Ramji Lal"],"report_count":12},{"incident_id":57,"occurred_on":"2015-07-01","title":"Australian Automated Debt Assessment System Issued False Notices to Thousands","description":"Australian Department of Human Services (DHS)’s automated debt assessment system issued false or incorrect debt notices to hundreds of thousands of people, resulting in years-long lawsuits and damages to welfare recipients.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["public administration"],"countries":["AU"],"deployers":["Australian Department Of Human Services"],"developers":["Centrelink"],"harmed":["Australian Welfare Recipients"],"report_count":39},{"incident_id":25,"occurred_on":"2015-06-23","title":"Google and Delphi Self-Driving Prototypes Allegedly Involved in Near-Miss on San Antonio Road, Palo Alto","description":"A Delphi self-driving Audi Q5 prototype reportedly aborted a lane change after a Google self-driving Lexus RX400h allegedly cut in front of it on San Antonio Road in Palo Alto, California. According to Delphi executive John Absmeier, the Delphi car \"took appropriate action\" to avoid collision. No crash or injuries occurred.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["transportation and storage"],"countries":["US"],"deployers":["Google, Delphi Technologies"],"developers":["Google, Delphi Technologies"],"harmed":["Delphi Technologies, John Absmeier"],"report_count":12},{"incident_id":392,"occurred_on":"2015-06-01","title":"Facebook's AI-Supported Moderation Failed to Classify Terrorist Content in East African Languages","description":"Facebook's system involving algorithmic content moderation for East African languages was reportedly failing to identify violating content on the platform such as mistakenly classifying non-terrorist content.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Facebook Users Speaking East African Languages, Facebook Users In East Africa"],"report_count":2},{"incident_id":226,"occurred_on":"2015-04-01","title":"Waze Allegedly Clogged Streets and Directed Drivers to Make Unsafe Traffic Decisions","description":"For years, Waze has, in an attempt to cut travel times, allegedly caused more traffic and guided drivers to make unsafe and often un-permitted traffic decisions, which was described by a Los Angeles city council member as a threat to public safety.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Waze"],"developers":["Waze"],"harmed":["Sherman Oaks Residents, Waze Users, Los Angeles City Government"],"report_count":3},{"incident_id":327,"occurred_on":"2015-03-24","title":"Facebook’s On-This-Day Feature Mistakenly Showed Painful Memories to Users","description":"Facebook’s “On This Day” algorithm which highlighted past posts on a user’s private page or News Feed confronted unwanted and painful personal memories to its users.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Facebook Users Having Posts About Painful Events, Facebook Users"],"report_count":1},{"incident_id":326,"occurred_on":"2014-12-09","title":"Facebook Automated Year-in-Review Highlights Showed Users Painful Memories","description":"Facebook’s “Year in Review” algorithm which compiled content in users’ past year as highlights inadvertently showed painful and unwanted memories to users, including death of family member.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Facebook Users Having Posts About Painful Events, Facebook Users"],"report_count":3},{"incident_id":35,"occurred_on":"2014-10-18","title":"Employee Automatically Terminated by Computer Program","description":"An employee was laid off, allegedly by an artificially intelligent personnel system, and blocked from access to the building and computer systems without their knowledge.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["administrative and support service activities"],"countries":["US"],"deployers":["Unknown"],"developers":["Unknown"],"harmed":["Ibrahim Diallo"],"report_count":20},{"incident_id":8,"occurred_on":"2014-08-15","title":"Uber Autonomous Cars Running Red Lights","description":"Uber vehicles equipped with technology allowing for autonomous driving running red lights in San Francisco street testing.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm near-miss","sectors":["transportation and storage"],"countries":["US"],"deployers":["Uber"],"developers":["Uber"],"harmed":["Pedestrians, Motorists"],"report_count":9},{"incident_id":24,"occurred_on":"2014-07-15","title":"Robot kills worker at German Volkswagen plant","description":"A Volkswagen plant robot \"crushed to death\" a worker by pinning him to a metal plate. ","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","harm_level":"none","sectors":["manufacturing"],"countries":["DE"],"deployers":["Volkswagen"],"developers":["Volkswagen"],"harmed":["Robotics Consultant"],"report_count":27},{"incident_id":246,"occurred_on":"2014-04-16","title":"Misreading of an Automated License Plate Reader (ALPR) Unverified by Police, Resulting in Traffic Stop in Missouri","description":"An automated license plate reader (ALPR) camera misread a 7 as a 2 and incorrectly alerted the local police about a stolen Oldsmobile car, which was allegedly not able to be verified by an officer before a traffic stop was effected on a BMW in Kansas City suburb.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Prairie Village Police Department"],"developers":["Unknown"],"harmed":["Mark Molner"],"report_count":2},{"incident_id":46,"occurred_on":"2014-01-21","title":"Nest Smoke Alarm Erroneously Stops Alarming","description":"In testing, Google Nest engineers demonstrated that the Nest Wave feature of their Nest Protect: Smoke + CO Alarm could inadvertently silence genuine alarms.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["other"],"countries":["US"],"deployers":["Nest Labs"],"developers":["Nest Labs"],"harmed":["Fire Victims"],"report_count":6},{"incident_id":162,"occurred_on":"2014-01-01","title":"ETS Used Allegedly Flawed Voice Recognition Evidence to Accuse and Assess Scale of Cheating, Causing Thousands to be Deported from the UK","description":" International testing organization ETS admits voice recognition as evidence of cheating for thousands of previous TOEIC test-takers that reportedly included wrongfully accused people, causing them to be deported without an appeal process or seeing their incriminating evidence.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["education","public administration"],"countries":["GB"],"deployers":["ETS"],"developers":["ETS"],"harmed":["UK Home Office","UK ETS test takers","UK ETS past test takers"],"report_count":1},{"incident_id":193,"occurred_on":"2013-11-27","title":"Excessive Automated Monitoring Alerts Ignored by Staff, Resulting in Private Data Theft of Seventy Million Target Customers","description":"Alerts about a Target data breach were ignored by Minneapolis Target’s staff reportedly due to them being included with many other potential false alerts, and due to some of the company’s network infiltration alerting systems being off to reduce such false alerts, causing private data theft for millions of customers.  ","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Target"],"developers":["Fireeye"],"harmed":["Target, Target Customers"],"report_count":1},{"incident_id":373,"occurred_on":"2013-10-01","title":"Michigan's Unemployment Benefits Algorithm MiDAS Issued False Fraud Claims to Thousands of People","description":"Michigan’s MiDAS system falsely accused over 34,000 people of unemployment fraud from 2013 to 2015, which reportedly caused financial ruin for many. The automated system was designed to cut costs, but it adjudicated fraud cases without human oversight. That led to an 85% error rate. Victims faced wage garnishments, some lost homes, and some faced bankruptcy. Despite early warnings, Michigan’s UIA defended MiDAS until lawsuits and federal pressure forced reforms. Legislators have been seeking com","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Michigan Unemployment Insurance Agency"],"developers":["Fast Enterprises, Csg Government Solutions"],"harmed":["Unemployed Michigan Residents Falsely Accused Of Fraud, Michigan Residents Who Faced Bankruptcy Or Foreclosure Due To Midas"],"report_count":14},{"incident_id":196,"occurred_on":"2013-09-01","title":"Compromise of National Biometric ID Card System Leads to Reverification and Change of Status","description":"When the leader of the Afghan Taliban was found possessing a valid ID card in the Pakistani national biometric identification database system, Pakistan launch a national re-verification campaign that is linked to numerous changes in recognition status and loss of services.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Pakistan National Database And Registration Authority"],"developers":["Pakistan National Database And Registration Authority"],"harmed":["Pakistani Citizens"],"report_count":5},{"incident_id":536,"occurred_on":"2012-12-10","title":"NJ Transit's Use of Modeling Software Miscalculated Storm Surge Threat Level","description":"New Jersey Transit's use of a federal government storm modeling software underestimated the threat of storm surges to the Meadows Maintenance Complex, leaving millions of dollars worth of equipment in the rail yard before Hurricane Sandy struck.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["New Jersey Transit"],"developers":["National Weather Service"],"harmed":["New Jersey Transit, New Jersey Transit Passengers"],"report_count":2},{"incident_id":112,"occurred_on":"2012-10-09","title":"Police Departments Reported ShotSpotter as Unreliable and Wasteful","description":"ShotSpotter algorithmic systems locating gunshots were reported by police departments for containing high false positive rates and wasting police resources, prompting discontinuation.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["law enforcement"],"countries":["US"],"deployers":["Troy Police Department","Syracuse Police Department","San Francisco Police Department","San Antonio Police Department","New York City Police Department","Fall River Police Department","Chicago Police Department"],"developers":["ShotSpotter"],"harmed":["Troy residents","Troy Police Department","Syracuse residents","Syracuse Police Department","San Francisco residents","San Francisco Police Department","San Antonio residents","San Antonio Police Department"],"report_count":15},{"incident_id":9,"occurred_on":"2012-02-25","title":"New York City Reportedly Released Disputed Value-Added Teacher Ratings for Thousands of Teachers","description":"After a legal dispute over public-records requests, the New York City Department of Education released Teacher Data Reports that used a value-added statistical model to rate thousands of public-school teachers based on estimated student test-score growth. News outlets published the ratings in February 2012, and teachers, unions, and analysts disputed the scores as imprecise and potentially misleading, citing large error margins, limited samples, and apparent roster-linkage errors.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":"none","sectors":["education"],"countries":["US"],"deployers":["New York City Department of Education"],"developers":["New York City Department of Education"],"harmed":["Teachers","Students","New York City teachers","New York City students","Epistemic integrity","Educational communities"],"report_count":6},{"incident_id":239,"occurred_on":"2009-09-01","title":"Algorithmic Teacher Evaluation Program Reportedly Failed to Improve Student Outcomes and Was Alleged to Have Harmed Teachers","description":"Gates Foundation-funded Intensive Partnerships for Effective Teaching Initiative's algorithmic program to assess teacher performance reportedly failed to achieve its goals for student outcomes, particularly for minority students, and was criticized for potentially causing harm against teachers.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"unclear","sectors":["education"],"countries":["US"],"deployers":["Intensive Partnerships for Effective Teaching"],"developers":["Intensive Partnerships for Effective Teaching"],"harmed":["Teachers","Students","Minority students","Minority groups","Epistemic integrity","Educational communities","Economically vulnerable students","Economically vulnerable people"],"report_count":1},{"incident_id":245,"occurred_on":"2009-03-30","title":"Unverified Misreading by Automated Plate Reader Led to Traffic Stop and Restraint of an Innocent Person at Gunpoint in California","description":"In San Francisco, an automated license plate reader (ALPR) camera misread a number as belonging to a stolen vehicle having the wrong make, but its photo was not visually confirmed by the police due to poor quality and allegedly despite multiple chances prior to making a traffic stop, causing an innocent person to be pulled over at gunpoint and restrained in handcuffed.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["law enforcement"],"countries":["US"],"deployers":["San Francisco Police Department"],"developers":["Unknown"],"harmed":["Denise Green"],"report_count":1},{"incident_id":186,"occurred_on":"2007-07-26","title":"Algorithm Assessing Risk Faced by Victims of Gender Violence Misclassified Low-Risk Cases, Allegedly Leading to Homicide of Women and Children in Spain","description":" In Spain, the algorithm that assesses recidivism risk in gender violence, VioGén, has allegedly critically underestimated the level of risk in a series of cases that ended in homicide of women and children since its first deployment.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Spanish Ministry  of  Interior","Government of Spain"],"developers":["Spanish Secretary of State for Security","Spanish Ministry  of  Interior","Government of Spain"],"harmed":["Women and girls","Women","Victims of gender violence in Spain","Victims of gender violence","Victims of domestic abuse","Minors","General public of Spain","General public"],"report_count":7},{"incident_id":445,"occurred_on":"2003-04-02","title":"Patriot Missile System Misclassified US Navy Aircraft, Killing Pilot Upon Approval to Fire","description":"US Navy's Patriot missile system misidentified an American Navy F/A-18C Hornet as an enemy projectile, prompting an operator to fire two missiles at the aircraft, which killed the pilot.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Us Navy"],"developers":["Raytheon, Lockheed Martin"],"harmed":["Us Navy, Nathan White'S Family, Nathan White"],"report_count":4},{"incident_id":444,"occurred_on":"2003-03-22","title":"US Air Force's Patriot Missile Mistakenly Launched at Ally Fighter Jet, Killing Two","description":"Acting on the recommendation of their Patriot missile system, American Air Force mistakenly launched the missile at an ally UK Tornado fighter jet, which killed two crew members on board.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Us Air Force"],"developers":["Raytheon, Lockheed Martin"],"harmed":["Us Air Force, Uk Royal Air Force, Kevin Main, David Williams"],"report_count":3},{"incident_id":379,"occurred_on":"1992-05-25","title":"Error in Pepsi's Number Generation System Led to Decades-Long Damages in the Philippines","description":"Pepsi's number generation system determining daily winners in its Number Fever promotion in the Philippines mistakenly produced a number held by thousands which resulted in riots, deaths, conspiracy theories, and decades of lawsuits.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Pepsi"],"developers":["D.G. Consultores"],"harmed":["Filipinos"],"report_count":2},{"incident_id":27,"occurred_on":"1983-09-26","title":"Nuclear False Alarm","description":"An alert of five incoming intercontinental ballistic missiles was properly identified as a false-positive by the Soviet Union operator Stanislov Petrov.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["defense"],"countries":["RU"],"deployers":["Soviet Union"],"developers":["Soviet Union"],"harmed":["All Life On Earth"],"report_count":27}]}