AIPolicyTracker
Menu

AI incident #157 ·

Amazon's Monitoring System Allegedly Pushed Delivery Drivers to Prioritize Speed over Safety, Leading to Crash

Open on the AI Incident Database 1 news report Synced from the AIID API · record last edited 5 Sep 2026

What happened

A lawsuit cited Amazon as liable in a crash involving its delivery driver, alleging that Amazon’s AI-powered driver monitoring system pushed drivers to prioritize speed over safety.

Editor's notes (AI Incident Database)

Alleged Harmed Party's name has been redacted at their request.

Only the incident metadata is stored here. The underlying news reports are on the AI Incident Database (CC BY-SA 4.0); use the links above to read them.

News reports (1)

Coverage catalogued by the AI Incident Database. Titles link to the original publisher; the text is not reproduced here.

  1. Amazon liable for crash because software “micromanages” delivery drivers, victim says
    arstechnica.com · Tim De Chant · AIID #1516

Who was involved

Alleged deployer
Amazon
On AIID: Amazon
Alleged developer
Amazon
On AIID: Amazon
Alleged harmed party
Amazon workers Amazon delivery drivers
On AIID: Amazon workers, Amazon delivery drivers

AI systems implicated

Algorithmic management systems

Classification (MIT AI Risk Repository taxonomy)

Causal entity
AI
Intent
Unintentional
Timing
Post-deployment
Harm level
unclear
Sectors
transportation and storage, wholesale and retail trade
Countries
US

Risk entries describing this failure mode

Entries from the MIT AI Risk Repository coded to subdomain 5.1.

  • Addiction

    "Addiction - Emotional or material dependence on technology or a technology system."

    A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)

  • Overreliance

    "Over-reliance - Unfettered and/or obsessive belief in the accuracy or other quality of a technology system, resulting in addiction, anxiety, introversion, sentience, complacency, lack of critical thinking and other actu...

    A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)

  • Economic instability

    "Economic instability - Uncontrolled fluctuations impacting the financial system, or parts thereof, due to the use or misuse of a technology system, or set of systems."

    A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)

  • Alienation/isolation

    "Alienation/isolation - An individual’s or group’s feeling of lack of connection with those around as a result of technology use or misuse."

    A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)

  • Social manipulation in elderly- and child-care

    " the use of advanced AI for elderly- and child-care are subject to risk of psychological manipulation and misjudgment "

    The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)

  • Healthcare

    "the use of advanced AI for elderly- and child-care are subject to risk of psychological manipulation and misjudgment (see page 17). In addition, concerns about patients’ privacy when AI uses medical records to research...

    The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)

  • Alteration of social relationships may induce psychological distress

    The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)

  • Inconsistent Performance across and within Domains

    "Estimating true capabilities of an LLM is a difficult task (c.f. Section 3.3), especially for naive users unfamiliar with the brittle nature of machine learning technologies. Exaggeration of model capabilities by the de...

    Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

Linked by AIID editors or by its text-similarity model.

Incidents in the same risk subdomain

All incidents in this subdomain

Other incidents involving Amazon