MIT AI Risk Repository · Risk Sub-Category · 62.36.06
Long-term effects of AI model biases on user judgment
Category: Impacts of AI (Bias)
Description
"The initial user exposure to model biases can have a lasting impact beyond the initial interaction with the model. Users who encounter biases in AI models can be affected by and continue to exhibit previously encountered biases in their decision-making, even after they stop using the models [207]."
From Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Causal entity
- AI
- Intent
- Unintentional
- Timing
- Post-deployment
Subdomain definition: Humans delegating key decisions to AI systems, or AI systems making decisions that diminish human control and autonomy, potentially leading to humans feeling disempowered, losing the ability to shape a fulfilling life trajectory or becoming cognitively enfeebled.
Real-world incidents in this subdomain
- DisMech AI Curation Agent Reportedly Completed GitHub Issue Intended as New Contributor's Learning Task
- Waymo Driverless Taxi Allegedly Stalled During Pedestrian Harassment Incident in San Francisco
- California Police Turned on Music to Allegedly Trigger Instagram’s DCMA to Avoid Being Live-Streamed
- Hawaii Police Deployed Robot Dog to Patrol a Homeless Encampment