{"attribution":{"source":"AI Incident Database (Responsible AI Collaborative)","license":"CC BY-SA 4.0","license_url":"https://creativecommons.org/licenses/by-sa/4.0/","citation":"McGregor, S. (2021). Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. Proceedings of the AAAI Conference on Artificial Intelligence (IAAI-21).","snapshot_date":"2026-09-07"},"exported_at":"2026-09-11"}
{"rows":[{"incident_id":62,"occurred_on":"2017-12-23","title":"Bad AI-Written Christmas Carols","description":"Janelle Shane, an AI research scientist, used 240 popular Christmas carols to train a neural network to write its own carols. This incident has been downgraded to an issue as it does not meet current ingestion criteria.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Other","intent":"Other","timing":"Other","harm_level":null,"sectors":[],"countries":[],"deployers":["Janelle Shane"],"developers":["Janelle Shane"],"harmed":["Carollers"],"report_count":1},{"incident_id":22,"occurred_on":"2017-12-06","title":"Waze Navigates Motorists into Wildfires","description":"Waze, a Google-owned directions app, led California drivers into the 2017 Skirball wildfires as they tried to evacuate the area.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm near-miss","sectors":["transportation and storage","information and communication"],"countries":["US"],"deployers":["Google"],"developers":["Google"],"harmed":["Motorists"],"report_count":22},{"incident_id":31,"occurred_on":"2017-12-03","title":"Driverless Train in Delhi Crashes due to Braking Failure","description":"A driverless metro train in Delhi, India crashed during a test run due to faulty brakes.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","harm_level":"none","sectors":["transportation and storage"],"countries":["IN"],"deployers":["Delhi Metro Rail Corporation"],"developers":["Unknown"],"harmed":["Delhi Metro Rail Corporation"],"report_count":29},{"incident_id":382,"occurred_on":"2017-11-21","title":"Instagram's Exposure of Harmful Content Allegedly Contributed to Teenage Girl's Suicide","description":"Instagram was ruled by a judge to have contributed to the death of a teenage girl in the UK allegedly through its exposure and recommendation of suicide, self-harm, and depressive content.","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Exposure to toxic content","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Meta"],"developers":["Meta"],"harmed":["Women and girls","Molly Rose Russell","Girls","Family of Molly Rose Russell","Adolescent girls"],"report_count":1},{"incident_id":261,"occurred_on":"2017-11-15","title":"Robot Deployed by Animal Shelter to Patrol Sidewalks outside Its Office, Warding off Homeless People in San Francisco","description":"Society for the Prevention of Cruelty to Animals (SPCA) deployed a Knightscope robot to autonomously patrol the area outside its office and ward off homeless people, which was criticized by residents as a tool of intimidation and ordered by the city of San Francisco to stop its use on a public right-of-way.","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Unfair discrimination and misrepresentation","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Society For The Prevention Of Cruelty To Animals"],"developers":["Knightscope"],"harmed":["San Francisco Homeless People"],"report_count":8},{"incident_id":33,"occurred_on":"2017-11-09","title":"Amazon Alexa Plays Loud Music when Owner is Away","description":"An Amazon Alexa, without instruction to do so, began playing loud music in the early morning while the homeowner was away leading to police breaking into their house to turn off the device.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication"],"countries":["DE"],"deployers":["Amazon"],"developers":["Amazon"],"harmed":["Oliver Haberstroh, Neighbors"],"report_count":4},{"incident_id":23,"occurred_on":"2017-11-08","title":"Las Vegas Self-Driving Bus Involved in Accident","description":"A self-driving public shuttle by Keolis North America and Navya was involved in a collision with a human-driven delivery truck in Las Vegas, Nevada on its first day of service.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["transportation and storage"],"countries":["US"],"deployers":["Navya, Keolis North America"],"developers":["Navya, Keolis North America"],"harmed":["Navya, Keolis North America, Bus Passengers"],"report_count":23},{"incident_id":307,"occurred_on":"2017-11-01","title":"iPhone Face ID Failed to Recognize Users’ Morning Faces","description":"The Face ID feature on iPhone allowing users to unlock their phones via facial recognition was reported by users for not recognizing their faces in the morning.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Apple"],"developers":["Apple"],"harmed":["Iphone Face Id Users, Iphone X Face Id Users"],"report_count":1},{"incident_id":14,"occurred_on":"2017-10-26","title":"Biased Sentiment Analysis","description":"Google Cloud's Natural Language API provided racist, homophobic, amd antisemitic sentiment analyses.","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Unfair discrimination and misrepresentation","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication"],"countries":[],"deployers":["Google"],"developers":["Google"],"harmed":["Women and girls","Women","Minority groups"],"report_count":6},{"incident_id":72,"occurred_on":"2017-10-17","title":"Facebook translates 'good morning' into 'attack them', leading to arrest","description":"Facebook's automatic language translation software incorrectly translated an Arabic post saying \"Good morning\" into Hebrew saying \"hurt them,\" leading to the arrest of a Palestinian man in Beitar Illit, Israel.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["information and communication"],"countries":["PS"],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Unnamed Palestinian Facebook User, Palestinian Facebook Users, Arabic Speaking Facebook Users, Facebook Users"],"report_count":26},{"incident_id":58,"occurred_on":"2017-10-12","title":"Russian Chatbot Supports Stalin and Violence","description":"Yandex, a Russian technology company, released an artificially intelligent chat bot named Alice which began to reply to questions with racist, pro-stalin, and pro-violence responses","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Exposure to toxic content","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication"],"countries":["RU"],"deployers":["Yandex"],"developers":["Yandex"],"harmed":["Yandex Users"],"report_count":5},{"incident_id":216,"occurred_on":"2017-10-10","title":"WeChat’s Machine Translation Gave a Racist English Translation for the Chinese Term for “Black Foreigner”","description":"The Chinese platform WeChat provided an inappropriate and racist English translation for the Chinese term for “black foreigner” in its messaging app.","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Exposure to toxic content","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Wechat"],"developers":["Wechat"],"harmed":["Black Wechat Users"],"report_count":5},{"incident_id":370,"occurred_on":"2017-09-27","title":"Google Fined for Changing Shopping Algorithms in EU to Favor Own Service","description":"Google was fined by EU Commission for changing its shopping algorithms in Europe to favor its own comparison service over competitors, resulting in anti-competitive effects.","mit_domain":"Socioeconomic & Environmental Harms","mit_subdomain":"Power centralization and unfair distribution of benefits","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Google"],"developers":["Google"],"harmed":["Google'S Competitor Shopping Services"],"report_count":1},{"incident_id":325,"occurred_on":"2017-09-21","title":"Offensive Instagram User Content Displayed as Facebook Ad","description":"An Instagram user’s image containing violent content was reportedly used as advertisement on Facebook allegedly via automated means.","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Exposure to toxic content","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Olivia Solon, Olivia Solon'S Facebook Connections"],"report_count":2},{"incident_id":329,"occurred_on":"2017-09-18","title":"Amazon Reportedly Recommends Explosive-Producing Ingredients as 'Frequently Bought Together' Items for Chemicals","description":"Amazon was reported to have shown chemical combinations for producing explosives and incendiary devices as \"frequently bought together\" items via automated recommendation.","mit_domain":"Malicious Actors & Misuse","mit_subdomain":"Cyberattacks, weapon development or use, and mass harm","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["wholesale and retail trade"],"countries":[],"deployers":["Amazon"],"developers":["Amazon"],"harmed":["Amazon Users"],"report_count":2},{"incident_id":531,"occurred_on":"2017-09-15","title":"AI-Assisted Body Scanners Reportedly Subjected Transgender Travelers to Invasive Body Searches","description":"Transportation Security Administration (TSA)'s use of image-processing body scanners at airports led transgender and gender-nonconforming travelers to be subjected to allegedly discriminatory and invasive searches, such as being asked to remove undergarments in private rooms by officers not of their gender.","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Unfair discrimination and misrepresentation","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Transportation Security Administration"],"developers":["L3Harris Technologies"],"harmed":["Transgender Travelers, Gender Nonconforming Travelers"],"report_count":1},{"incident_id":26,"occurred_on":"2017-09-13","title":"Hackers Break Apple Face ID","description":"Vietnamese security firm Bkav created an improved mask to bypass Apple's Face ID","mit_domain":"Privacy & Security","mit_subdomain":"AI system security vulnerabilities and attacks","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication"],"countries":["VN"],"deployers":["Apple"],"developers":["Apple"],"harmed":["Apple, Device Owners"],"report_count":24},{"incident_id":32,"occurred_on":"2017-09-13","title":"Identical Twins Can Open Apple FaceID Protected Devices","description":"Apple's iPhone FaceID can be opened by an identical twin of the person who has registered their face to unlock the phone.","mit_domain":"Privacy & Security","mit_subdomain":"AI system security vulnerabilities and attacks","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication"],"countries":["US"],"deployers":["Apple"],"developers":["Apple"],"harmed":["People With Twins"],"report_count":21},{"incident_id":167,"occurred_on":"2017-09-07","title":"Researchers' Homosexual-Men Detection Model Denounced as a Threat to LGBTQ People's Safety and Privacy","description":"Researchers at Stanford Graduate School of Business developed a model that determined, on a binary scale, whether someone was homosexual using only his facial image, which advocacy groups such as GLAAD and the Human Rights Campaign denounced as flawed science and threatening to LGBTQ folks.","mit_domain":"Privacy & Security","mit_subdomain":"Compromise of privacy by obtaining, leaking or correctly inferring sensitive information","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Yilun Wang","Michal Kosinski"],"developers":["Yilun Wang","Michal Kosinski"],"harmed":["Privacy","non-American LGBTQ people","LGBTQ people of color","LGBTQ people","Biometric data subjects"],"report_count":1},{"incident_id":309,"occurred_on":"2017-08-26","title":"Facial Recognition Trial Performed Poorly at Notting Hill Carnival","description":"The facial recognition trial by London’s Metropolitan Police Service at the Notting Hill Carnival reportedly performed poorly with a high rate of false positives.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Metropolitan Police Service"],"developers":["Unknown"],"harmed":["Notting Hill Carnival Goers"],"report_count":4},{"incident_id":1057,"occurred_on":"2017-08-15","title":"'Jewish Baby Strollers' Provided Antisemitic Google Images, Allegedly Resulting from Hate Speech Campaign","description":"Results for Google Images searches for \"Jewish baby strollers\" reportedly showed offensive, antisemitic results, allegedly a result of a coordinated hate-speech campaign involving malicious actors on 4chan.","mit_domain":"Malicious Actors & Misuse","mit_subdomain":"Disinformation, surveillance, and influence at scale","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Google"],"developers":["Google"],"harmed":["Jewish People, Google Images Users"],"report_count":2},{"incident_id":66,"occurred_on":"2017-08-02","title":"Chinese Chatbots Question Communist Party","description":"Chatbots on Chinese messaging service expressed anti-China sentiments, causing the messaging service to remove and reprogram the chatbots.","mit_domain":"Misinformation","mit_subdomain":"False or misleading information","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication"],"countries":["CN"],"deployers":["Tencent Holdings"],"developers":["Microsoft, Turing Robot"],"harmed":["Chinese Communist Party, Tencent Holdings, Microsoft, Turing Robot"],"report_count":16},{"incident_id":128,"occurred_on":"2017-08-01","title":"Tesla Sedan on Autopilot Reportedly Drove Over Dividing Curb in Washington, Resulting in Minor Vehicle Damage","description":" A Tesla Sedan operating on Autopilot mode was not able to center itself on the road and drove over a yellow dividing curb in Redmond, Washington, causing minor damage to the vehicle’s rear suspension.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["transportation and storage"],"countries":["US"],"deployers":["Tesla"],"developers":["Tesla"],"harmed":["Eric Horvitz, Tesla Drivers"],"report_count":2},{"incident_id":1428,"occurred_on":"2017-07-28","title":"UK High Court Found Sky Betting & Gaming Unlawfully Used Automated Profiling and Targeted Marketing to Exploit a Recovering Problem Gambler","description":"In the UK, the High Court found that Sky Betting & Gaming unlawfully used automated profiling and targeted direct marketing to pursue a recovering problem gambler from July 28, 2017 onward without valid consent. Sky reportedly treated him as a high-value customer despite addiction indicators.","mit_domain":"Privacy & Security","mit_subdomain":"Compromise of privacy by obtaining, leaking or correctly inferring sensitive information","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Sky Betting & Gaming","Hestview Ltd","Bonne Terre Ltd"],"developers":["Sky Betting & Gaming","Hestview Ltd","Bonne Terre Ltd"],"harmed":["Sky Betting & Gaming customers with gambling disorders","RTM (recovering problem gambler)","Recovering problem gamblers","Privacy","People with gambling disorders"],"report_count":2},{"incident_id":68,"occurred_on":"2017-07-17","title":"Security Robot Drowns Itself in a Fountain","description":"A Knightscope K5 security robot ran itself into a water fountain in Washington, DC.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["administrative and support service activities"],"countries":["US"],"deployers":["Knightscope"],"developers":["Knightscope"],"harmed":["Knightscope"],"report_count":29},{"incident_id":56,"occurred_on":"2017-07-10","title":"AI-Designed Phone Cases Are Unexpected","description":"A third-party Amazon merchant named “my_handy_design” was suspected of using a bot to generate cell phone case designs based on the bizarre and unattractive designs being offered.","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Exposure to toxic content","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["wholesale and retail trade"],"countries":[],"deployers":["My_Handy_Design"],"developers":["My_Handy_Design"],"harmed":["My_Handy_Design"],"report_count":7},{"incident_id":308,"occurred_on":"2017-07-03","title":"Atlas Robot Fell off Stage at Conference","description":"Boston Dynamics’s autonomous robot Atlas allegedly caught its foot on a stage light, resulting in a fall off the stage at the Congress of Future Science and Technology Leaders conference.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Boston Dynamics"],"developers":["Boston Dynamics"],"harmed":["None"],"report_count":2},{"incident_id":39,"occurred_on":"2017-07-01","title":"Deepfake Obama Introduction of Deepfakes","description":"University of Washington researchers made a deepfake of Obama, followed by Jordan Peele","mit_domain":"Misinformation","mit_subdomain":"False or misleading information","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":"none","sectors":["arts, entertainment and recreation","professional, scientific and technical activities"],"countries":["US"],"deployers":["University Of Washington, Fakeapp"],"developers":["University Of Washington, Fakeapp"],"harmed":["Barack Obama"],"report_count":29},{"incident_id":183,"occurred_on":"2017-07-01","title":"Airbnb's Trustworthiness Algorithm Allegedly Banned Users without Explanation, and Discriminated against Sex Workers","description":" Airbnb allegedly considered publicly available data on users to gauge their trustworthiness via algorithmic assessment of personality and behavioral traits, resulting in unexplained bans and discriminatory bans against sex workers.","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Unfair discrimination and misrepresentation","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Airbnb"],"developers":["Airbnb, Trooly"],"harmed":["Sex Workers, Airbnb Users"],"report_count":6},{"incident_id":267,"occurred_on":"2017-06-15","title":"Clearview AI Algorithm Built on Photos Scraped from Social Media Profiles without Consent","description":"Face-matching algorithm by Clearview AI was built using scraped images from social media sites such as Instagram and Facebook without user consent, violating social media site policies, and allegedly privacy regulations.","mit_domain":"Privacy & Security","mit_subdomain":"Compromise of privacy by obtaining, leaking or correctly inferring sensitive information","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Clearview AI"],"developers":["Clearview AI"],"harmed":["Social media users","Privacy","Instagram users","Facebook users","Biometric data subjects"],"report_count":10},{"incident_id":310,"occurred_on":"2017-06-03","title":"High False Positive Rate by SWP's Facial Recognition Use at Champion's League Final","description":"South Wales Police (SWP)’s automated facial recognition (AFR) at the Champion's League Final football game in Cardiff wrongly identified innocent people as potential matches at an extremely high false positive rate of more than 90%.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["South Wales Police"],"developers":["Nec"],"harmed":["Finals Attendees, Falsely Accused Finals Attendees"],"report_count":8},{"incident_id":428,"occurred_on":"2017-05-19","title":"BBC Reporter's Twin Brother Cracked HSBC's Voice ID Authentication","description":"HSBC’s voice recognition authentication system was fooled after seven repeated attempts  by a BBC reporter's twin brother who mimicked his voice to access his bank account.","mit_domain":"Privacy & Security","mit_subdomain":"AI system security vulnerabilities and attacks","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Hsbc Uk"],"developers":["Nuance Communications"],"harmed":["Hsbc Uk Customers, Dan Simmons"],"report_count":3},{"incident_id":96,"occurred_on":"2017-05-08","title":"Houston ISD's EVAAS Teacher-Evaluation System Reportedly Put Teachers' Jobs at Risk Through Unverifiable Scores","description":"From 2011 to 2015, Houston Independent School District used SAS Institute’s Education Value-Added Assessment System (EVAAS) to rate teacher effectiveness from student test-score growth. Teachers alleged that proprietary, unverifiable EVAAS scores contributed to terminations and contract nonrenewals. On May 4, 2017, a federal judge allowed due-process claims over the system to proceed.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of transparency or interpretability","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":"none","sectors":["education"],"countries":["US"],"deployers":["Houston Independent School District"],"developers":["SAS Institute"],"harmed":["Teachers","Houston Independent School District teachers","Educators","Educational communities"],"report_count":1},{"incident_id":61,"occurred_on":"2017-05-01","title":"Overfit Kaggle Models Discouraged Data Science Competitors","description":"In the “The Nature Conservancy Fisheries Monitoring” competition on the data science competition website Kaggle, a number of competitors overfit their image classifier models to a poorly representative validation data set.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication","education"],"countries":[],"deployers":["Individual Kaggle Competitors"],"developers":["Individual Kaggle Competitors"],"harmed":["Individual Kaggle Competitors"],"report_count":1},{"incident_id":518,"occurred_on":"2017-04-28","title":"New York Detective Reportedly Misused Woody Harrelson's Face to Perform Face Recognition Search","description":"When the facial recognition search for a CVS theft suspect's face returned no useful matches due to the surveillance footage being obscured and highly pixelated, a New York City police detective continued the face search using Woody Harrelson's face allegedly due to his resemblance to the suspect's face, eventually leading to the arrest of an unknown victim.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["New York Police Department","Law enforcement","Facial Identification Section"],"developers":["Unknown"],"harmed":["Woody Harrelson","Unknown"],"report_count":1},{"incident_id":60,"occurred_on":"2017-04-25","title":"FaceApp Racial Filters","description":"FaceApp is criticized for offering racist filters.","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Unfair discrimination and misrepresentation","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["arts, entertainment and recreation","information and communication"],"countries":["RU"],"deployers":["Faceapp"],"developers":["Faceapp"],"harmed":["Minority Groups"],"report_count":23},{"incident_id":408,"occurred_on":"2017-04-15","title":"Facebook Reportedly Outed Sex Workers through Friend Recommendations","description":"Facebook's \"People You May Know\" feature reportedly outed sex workers by recommending clients to their personal accounts or family members to their business accounts with no option to opt out.","mit_domain":"Privacy & Security","mit_subdomain":"Compromise of privacy by obtaining, leaking or correctly inferring sensitive information","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Facebook"],"developers":["Facebook"],"harmed":["Social media users","sex workers using Facebook","sex workers","Privacy","Facebook users","Biometric data subjects"],"report_count":1},{"incident_id":59,"occurred_on":"2017-04-13","title":"Gender Biases in Google Translate","description":"A Cornell University study in 2016 highlighted Google Translate's pattern of assigning gender to occupations in a way showing an implicit gender bias against women.","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Unfair discrimination and misrepresentation","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication"],"countries":[],"deployers":["Google"],"developers":["Google"],"harmed":["Women and girls","Women"],"report_count":10},{"incident_id":502,"occurred_on":"2017-04-10","title":"Pennsylvania County's Family Screening Tool Allegedly Exhibited Discriminatory Effects","description":"Data analysis by the American Civil Liberty Union (ACLU) on Allegheny County's decision-support Family Screening Tool to predict child abuse or neglect risk found the tool resulting in higher screen-in rates for Black families and higher risk scores for households with disabled residents.","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Unfair discrimination and misrepresentation","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Allegheny County"],"developers":["Rhema Vaithianathan","Emily Putnam-Hornstein","Centre for Social Data Analytics"],"harmed":["People with disabilities","Hackneys family","Black families in Allegheny"],"report_count":3},{"incident_id":225,"occurred_on":"2017-04-07","title":"IBM Watson for Oncology Criticized by Customers for Allegedly Unsafe and Inaccurate Cancer Treatment Recommendations","description":"Internal documents from IBM Watson Health showed negative assessments from customers such as Florida’s Jupiter Hospital and Memorial Sloan Kettering criticizing its Watson for Oncology product for allegedly unsafe and incorrect cancer treatment recommendations.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Memorial Sloan Kettering","Jupiter Hospital"],"developers":["IBM Watson Health"],"harmed":["People seeking medical advice","Patients","oncologists","cancer patients"],"report_count":2},{"incident_id":341,"occurred_on":"2017-04-06","title":"Nissan's \"Automatic Emergency Braking\" False Positives Posed Traffic Risks to Drivers","description":"Nissan's Automatic Emergency Braking (AEB) feature was reported in a series of complaints for false positives and abrupt braking behaviors, endangering car occupants and traffic participants. ","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"AI tangible harm event","sectors":["transportation and storage"],"countries":["US"],"deployers":["Nissan"],"developers":["Nissan"],"harmed":["Nissan Drivers, Traffic Participants"],"report_count":5},{"incident_id":394,"occurred_on":"2017-03-15","title":"Social Media's Automated Word-Flagging without Context Shifted Content Creators' Language Use","description":"TikTok's, YouTube's, Instagram's, and Twitch's use of algorithms to flag certain words devoid of context changed content creators' use of everyday language or discussion about certain topics in fear of their content getting flagged or auto-demonetized by mistake.","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Unfair discrimination and misrepresentation","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Youtube, Twitch, Tiktok, Instagram"],"developers":["Youtube, Twitch, Tiktok, Instagram"],"harmed":["Youtube Content Creators, Twitch Content Creators, Tiktok Content Creators, Instagram Content Creators"],"report_count":2},{"incident_id":418,"occurred_on":"2017-03-13","title":"Uber Locked Indian Drivers out of Accounts Allegedly Due to Facial Recognition Fails","description":"Uber drivers in India reported being locked out of their accounts allegedly due to Real-Time ID Check's facial recognition failing to recognize appearance changes or faces in low lighting conditions.","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Unequal performance across groups","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Uber"],"developers":["Uber","Azure Cognitive Services"],"harmed":["Uber drivers in India"],"report_count":3},{"incident_id":496,"occurred_on":"2017-03-01","title":"Male College Freshman Allegedly Made Porn Deepfakes Using Female Friend's Face","description":"A female college student's face was superimposed on another woman's body in deepfake pornographic videos and shared on 4chan allegedly by a male student whose friendship with her fell apart during freshman year.","mit_domain":"Malicious Actors & Misuse","mit_subdomain":"Fraud, scams, and targeted manipulation","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["University students","Synthetic media creators","Students","Men and boys","Men","Deepfake creators","4chan users"],"developers":["Synthetic video generation technology developers","Synthetic media generation technology developers","Deepfake technology developers"],"harmed":["Women and girls","Women","Victims of non-consensual deepfakes","Victims of deepfake abuse","University students","Students","Privacy","Epistemic integrity"],"report_count":2},{"incident_id":13,"occurred_on":"2017-02-27","title":"High-Toxicity Assessed on Text Involving Women and Minority Groups","description":"Google's Perspective API, which assigns a toxicity score to online text, seems to award higher toxicity scores to content involving non-white, male, Christian, heterosexual phrases.","mit_domain":"Discrimination and Toxicity","mit_subdomain":"Unfair discrimination and misrepresentation","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication"],"countries":["US"],"deployers":["Google"],"developers":["Google"],"harmed":["Women and girls","Women","Minority groups"],"report_count":8},{"incident_id":7,"occurred_on":"2017-02-24","title":"Wikipedia Vandalism Prevention Bot Loop","description":"Wikipedia bots meant to remove vandalism clash with each other and form feedback loops of repetitve undoing of the other bot's edits.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"none","sectors":["information and communication"],"countries":[],"deployers":["Wikipedia"],"developers":["Wikipedia"],"harmed":["Wikimedia Foundation, Wikipedia Editors, Wikipedia Users"],"report_count":6},{"incident_id":340,"occurred_on":"2017-02-01","title":"Honda's CMBS False Positives Allegedly Caused Accidents to Customers","description":"Honda's Collision Mitigation Braking System (CMBS) allegedly caused accidents to consumers due to frequent instances of false obstacle detection.","mit_domain":"AI system safety, failures, and limitations","mit_subdomain":"Lack of capability or robustness","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Honda"],"developers":["Honda"],"harmed":["Honda Customers"],"report_count":1},{"incident_id":190,"occurred_on":"2017-01-15","title":"ByteDance Allegedly Trained \"For You\" Algorithm Using Content Scraped without Consent from Other Social Platforms","description":" ByteDance allegedly scraped short-form videos, usernames, profile pictures, and descriptions of accounts on Instagram, Snapchat, and other sources, and uploaded them without consent on Flipagram, TikTok’s predecessor, in order to improve its “For You” algorithm's performance on American users.","mit_domain":"Privacy & Security","mit_subdomain":"Compromise of privacy by obtaining, leaking or correctly inferring sensitive information","entity":"Human","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Bytedance"],"developers":["Bytedance"],"harmed":["Instagram Users, Snapchat Users, American Social Media Users"],"report_count":4},{"incident_id":894,"occurred_on":"2017-01-01","title":"RealPage's YieldStar Allegedly Facilitated Rent Price Coordination Among Canadian Landlords","description":"RealPage’s YieldStar algorithm is alleged to have enabled Canadian landlords to coordinate rental pricing by utilizing sensitive, non-public market data. The software has been accused of contributing to rent increases in non-rent-controlled units through purported collusion and artificial price inflation. Tenants have reported significant rent hikes, which they attribute to landlords' use of the algorithm.","mit_domain":"Malicious Actors & Misuse","mit_subdomain":"Fraud, scams, and targeted manipulation","entity":"AI","intent":"Intentional","timing":"Post-deployment","harm_level":null,"sectors":[],"countries":[],"deployers":["Woodbourne Capital Management International Lp, Tribute Communities, The Rockport Group, Tas Impact, Riocan Living, Rhapsody Property Management Services Llp, Gwl Realty Advisors Residential Inc., First Capital Real Estate Investment Trust, Edgar Development Corp., Dream Unlimited Corp., Core Development Group Ltd., Choice Properties Real Estate Investment Trust, Canadian Property Managers Using Yieldstar"],"developers":["Realpage"],"harmed":["Tenants In Canada, Tenants, Public Trust In The Rental Market, Economically Vulnerable Tenants In Canada, Cynthia Black, Economically Vulnerable People, Economically Vulnerable Tenants"],"report_count":7},{"incident_id":109,"occurred_on":"2017-01-01","title":"PimEyes's Facial Recognition AI Allegedly Lacked Safeguards to Prevent Itself from Being Abused","description":"PimEyes offered its subscription-based AI service to anyone in the public to search for matching facial images across the internet, which critics said lacked public oversight and government rules to prevent itself from misuse such as stalking women.","mit_domain":"Privacy & Security","mit_subdomain":"Compromise of privacy by obtaining, leaking or correctly inferring sensitive information","entity":"AI","intent":"Unintentional","timing":"Post-deployment","harm_level":"unclear","sectors":["information and communication","arts, entertainment and recreation"],"countries":[],"deployers":["PimEyes"],"developers":["PimEyes"],"harmed":["Privacy","internet users","Biometric data subjects"],"report_count":1}]}