MIT AI Risk Repository · Risk Sub-Category · 24.06.04

Generating material dependence without adequate commitment to user needs

Category: Appropriate Relationships

Open the MIT Risk Navigator All entries from Gabriel2024

Description

"In addition to emotional dependence, user–AI assistant relationships may give rise to material dependence if the relationships are not just emotionally difficult but also materially costly to exit. For example, a visually impaired user may decide not to register for a healthcare assistance programme to support navigation in cities on the grounds that their AI assistant can perform the relevant navigation functions and will continue to operate into the future. Cases like these may be ethically problematic if the user’s dependence on the AI assistant, to fulfil certain needs in their lives, is

From The Ethics of Advanced AI Assistants (Gabriel2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 5. Human-Computer Interaction
Subdomain: 5.2 Loss of human agency and autonomy
Causal entity: Human
Intent: Unintentional
Timing: Post-deployment

Subdomain definition: Humans delegating key decisions to AI systems, or AI systems making decisions that diminish human control and autonomy, potentially leading to humans feeling disempowered, losing the ability to shape a fulfilling life trajectory or becoming cognitively enfeebled.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

Autonomy
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Autonomy/agency loss
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Personality rights loss
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Gradual loss of control
Dimensional Characterization and Pathway Modeling for Catastrophic AI Risks (Chin2025)
AI enables automation of military decision-making
A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values (Clarke2023)
AI-induced strategic instability
A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values (Clarke2023)
-
Capabilities and Risks from Frontier AI (DSIT2023)
Humans might increasingly hand over control to misaligned AI systems
Capabilities and Risks from Frontier AI (DSIT2023)

Other entries from Gabriel2024

Capability failures Risk Category · 7.3
Lack of capability for task Risk Sub-Category · 7.3
Difficult to develop metrics for evaluating benefits or harms caused by AI assistants Risk Sub-Category · 6.5
Safe exploration problem with widely deployed AI assistants Risk Sub-Category · 7.3
Goal-related failures Risk Category · 7.1
Misaligned consequentialist reasoning Risk Sub-Category · 7.3
Specification gaming Risk Sub-Category · 7.1
Goal misgeneralisation Risk Sub-Category · 7.1
Deceptive alignment Risk Sub-Category · 7.1
Malicious Uses Risk Category · 4.0
Offensive Cyber Operations (General) Risk Sub-Category · 4.2
AI-Powered Spear-Phishing at Scale Risk Sub-Category · 4.1