AI incident ·
Amazon Allegedly Forced Deployment of AI-Powered Cameras on Delivery Drivers
In brief
An AI system built by Netradyne and deployed by Amazon allegedly harmed Privacy, Biometric data subjects and 1 other.
- Risk domain
- Privacy & Security
- Occurred
- Coverage
- 4 reports
What happened
Amazon delivery drivers were allegedly forced to consent to algorithmic collection and processing of their location, movement, and biometric data through AI-powered cameras, or be dismissed.
Laws that address this harm
Policy angle: Classified under Privacy & Security (Compromise of privacy by obtaining, leaking or correctly inferring sensitive information) in the MIT AI Risk Repository taxonomy; 5 recorded instruments address this use case.
- Colorado AI Act
- India DPDP Act
- Law No. 132/2025 on artificial intelligence
- EU AI Act
- Texas Responsible AI Governance Act (TRAIGA)
Matched from the record's risk domain and country to the instruments recorded here. A reviewer can correct the match in the repository (data/external/incident_overrides.yaml).
News reports (4)
Titles link to the original publisher; report text is not reproduced here.
Who was involved
- Alleged deployer
- Amazon
- Alleged developer
- Netradyne
- Alleged harmed party
- Privacy Biometric data subjects Amazon delivery drivers
AI systems implicated
Surveillance technologyNetradyne driver-monitoring camerasEnterprise AI systemsDriver monitoring systemsBiometric data processing systemsAlgorithmic management systems
Classification (MIT AI Risk Repository taxonomy)
- Risk domain
- Privacy & Security
- Risk subdomain
- 2.1 Compromise of privacy by obtaining, leaking or correctly inferring sensitive information
- Causal entity
- Human
- Intent
- Intentional
- Timing
- Post-deployment
- Harm level
- —
- Sectors
- —
- Countries
- —
Risk entries describing this failure mode
Entries from the MIT AI Risk Repository coded to subdomain 2.1.
- Risks to privacy
"General- purpose AI models or systems can ‘leak’ information about individuals whose data was used in training. For future models trained on sensitive personal data like health or financial data, this may lead to partic...
- Risks to privacy
"General- purpose AI systems can cause or contribute to violations of user privacy. Violations can occur inadvertently during the training or usage of AI systems, for example through unauthorised processing of personal d...
- Private Training Data
"As recent LLMs continue to incorporate licensed, created, and publicly available data sources in their corpora, the potential to mix private data in the training corpora is significantly increased. The misused private d...
- Association in LLMs
"Association in LLMs refers to the capability to associate various pieces of information related to a person. According to [68], [86], given a pair of PII entities (xi , xj ), which is associated by a model F. Using a pr...
- Privacy Leakage
"Privacy Leakage means the generated content includes sensitive personal information"
- Memorization in LLMs
"Memorization in LLMs refers to the capability to recover the training data with contextual prefixes. According to [88]–[90], given a PII entity x, which is memorized by a model F. Using a prompt p could force the model...
- Privacy Leakage
"The model is trained with personal data in the corpus and unintentionally exposing them during the conversation."
- Privacy and regulation violations
"Some of the broken systems discussed above are also very invasive of people’s privacy, controlling, for instance, the length of someone’s last romantic relationship [51]. More recently, ChatGPT was banned in Italy over...
Related incidents
Linked by editors or by text similarity in the source dataset.
- Motive AI Dashcams Allegedly Collected Truckers' Biometric Data and Built License-Plate Location Records Without Consent
- Kronos Scheduling Algorithm Allegedly Caused Financial Issues for Starbucks Employees
- Amazon's AI Cameras Incorrectly Penalized Delivery Drivers for Mistakes They Did Not Make
- Amazon Flex Drivers Allegedly Fired via Automated Employee Evaluations
Incidents in the same risk subdomain
- Grok Reportedly Disclosed Adult Performer Siri Dahl's Legal Name and Birthdate, Allegedly Contributing to Doxxing and Harassment
- NPR Host David Greene Alleged Google's NotebookLM Replicated His Voice Without Consent, Prompting Lawsuit
- Border Patrol Agent Allegedly Claimed Facial Recognition Identified Minneapolis ICE Observer and Global Entry Was Reportedly Revoked Three Days Later
- Perplexity AI Reportedly Accused in Federal Lawsuit of Purported Copyright Infringement and False Attribution of Chicago Tribune Content
- Secret Desires AI Platform Reportedly Exposed Nearly Two Million Sensitive Images in Cloud Storage Leak
- ChatGPT Reportedly Found to Reproduce Protected German Lyrics in Copyright Case
Other incidents involving Amazon
- Amazon Algorithmic Pricing Allegedly Hiked up Price of Reference Book to Millions
- Amazon Reportedly Sold Products and Recommended Frequently Bought Together Items That Aid Suicide Attempts
- Alexa Recommended Dangerous TikTok Challenge to Ten-Year-Old Girl
- Amazon's AI Cameras Incorrectly Penalized Delivery Drivers for Mistakes They Did Not Make
- Amazon's Monitoring System Allegedly Pushed Delivery Drivers to Prioritize Speed over Safety, Leading to Crash
- Amazon’s Search and Recommendation Algorithms Found by Auditors to Have Boosted Products That Contained Vaccine Misinformation
Source record: incident #395 on the AI Incident Database · all 4 reports