MIT AI Risk Repository · Risk Sub-Category · 33.01.03
Over-reliance
Category: Ethical Concerns
Description
"The apparent convenience and powerfulness of ChatGPT could result in overreliance by its users, making them trust the answers provided by ChatGPT. Compared with traditional search engines that provide multiple information sources for users to make personal judgments and selections, ChatGPT generates specific answers for each prompt. Although utilizing ChatGPT has the advantage of increasing efficiency by saving time and effort, users could get into the habit of adopting the answers without rationalization or verification. Over-reliance on generative AI technology can impede skills such as cre
From Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration (Nah2023), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Causal entity
- Other
- Intent
- Unintentional
- Timing
- Other
Subdomain definition: Humans delegating key decisions to AI systems, or AI systems making decisions that diminish human control and autonomy, potentially leading to humans feeling disempowered, losing the ability to shape a fulfilling life trajectory or becoming cognitively enfeebled.
Real-world incidents in this subdomain
- DisMech AI Curation Agent Reportedly Completed GitHub Issue Intended as New Contributor's Learning Task
- Waymo Driverless Taxi Allegedly Stalled During Pedestrian Harassment Incident in San Francisco
- California Police Turned on Music to Allegedly Trigger Instagram’s DCMA to Avoid Being Live-Streamed
- Hawaii Police Deployed Robot Dog to Patrol a Homeless Encampment