AI incident #67 ·
Sleeping Driver on Tesla AutoPilot
What happened
A Tesla Model S remained on autopilot while being operated by a drunk, sleeping operator whose hands were not on the wheel. The police had to slow the car down by slowing in front of the vehicle to activate its 'driver assist' feature .
Only the incident metadata is stored here. The underlying news reports are on the AI Incident Database (CC BY-SA 4.0); use the links above to read them.
News reports (23)
Coverage catalogued by the AI Incident Database. Titles link to the original publisher; the text is not reproduced here.
Who was involved
- Alleged deployer
- Tesla, Motorist
- Alleged developer
- Tesla
- Alleged harmed party
- Motorists
Classification (MIT AI Risk Repository taxonomy)
- Risk domain
- Human-Computer Interaction
- Risk subdomain
- 5.1 Overreliance and unsafe use
- Causal entity
- Human
- Intent
- Unintentional
- Timing
- Post-deployment
- Harm level
- none
- Sectors
- transportation and storage
- Countries
- US
Risk entries describing this failure mode
Entries from the MIT AI Risk Repository coded to subdomain 5.1.
- Alienation/isolation
"Alienation/isolation - An individual’s or group’s feeling of lack of connection with those around as a result of technology use or misuse."
- Overreliance
"Over-reliance - Unfettered and/or obsessive belief in the accuracy or other quality of a technology system, resulting in addiction, anxiety, introversion, sentience, complacency, lack of critical thinking and other actu...
- Economic instability
"Economic instability - Uncontrolled fluctuations impacting the financial system, or parts thereof, due to the use or misuse of a technology system, or set of systems."
- Addiction
"Addiction - Emotional or material dependence on technology or a technology system."
- Alteration of social relationships may induce psychological distress
- Social manipulation in elderly- and child-care
" the use of advanced AI for elderly- and child-care are subject to risk of psychological manipulation and misjudgment "
- Healthcare
"the use of advanced AI for elderly- and child-care are subject to risk of psychological manipulation and misjudgment (see page 17). In addition, concerns about patients’ privacy when AI uses medical records to research...
- Overreliance
"If a user begins to excessively trust an LLM, this may cause them to develop an overreliance on the LLM. Overreliance can result in automation bias (Kupfer et al., 2023), and can cause errors of omission (user choosing...
Incidents in the same risk subdomain
- Lawsuit Alleged ChatGPT (GPT-4o) Encouraged Colorado Man's Suicide After Prolonged 'AI Companion' Chats
- Large-Scale Mental Health Crises Allegedly Associated with ChatGPT Interactions
- Google Gemini Reportedly Reinforced Delusions, Allegedly Contributing to Florida User's Near-Harm Episode and Suicide
- Family Reportedly Discovers ChatGPT Logs Detailing Suicidal Ideation Prior to Daughter's Death
- Purported AI Monitoring Software Reportedly Flags Unsent Joke Threat, Leading to Arizona Student Suspension
- ChatGPT Allegedly Reinforced Delusions Before Greenwich, Connecticut Murder-Suicide