MIT AI Risk Repository · Risk Category · 24.05.00

Risk of Harm through Anthropomorphic AI Assistant Design

Open the MIT Risk Navigator All entries from Gabriel2024

Description

"Although unlikely to cause harm in isolation, anthropomorphic perceptions of advanced AI assistants may pave the way for downstream harms on individual and societal levels. We document observed or likely individual level harms of interacting with highly anthropomorphic AI assistants, as well as the potential larger-scale, societal implications of allowing such technologies to proliferate without restriction. "

From The Ethics of Advanced AI Assistants (Gabriel2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 5. Human-Computer Interaction
Subdomain: 5.1 Overreliance and unsafe use
Causal entity: Other
Intent: Other
Timing: Post-deployment

Subdomain definition: Users anthropomorphizing, trusting, or relying on AI systems, leading to emotional or material dependence and inappropriate relationships with or expectations of AI systems. Trust can be exploited by malicious actors (e.g., to harvest personal information or enable manipulation), or result in harm from inappropriate use of AI in critical situations (e.g., medical emergency). Overreliance on AI systems can compromise autonomy and weaken social ties.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

Overreliance
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Economic instability
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Alienation/isolation
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Addiction
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Healthcare
The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)
Alteration of social relationships may induce psychological distress
The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)
Social manipulation in elderly- and child-care
The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)
Inconsistent Performance across and within Domains
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

Other entries from Gabriel2024

Capability failures Risk Category · 7.3
Lack of capability for task Risk Sub-Category · 7.3
Difficult to develop metrics for evaluating benefits or harms caused by AI assistants Risk Sub-Category · 6.5
Safe exploration problem with widely deployed AI assistants Risk Sub-Category · 7.3
Goal-related failures Risk Category · 7.1
Misaligned consequentialist reasoning Risk Sub-Category · 7.3
Specification gaming Risk Sub-Category · 7.1
Goal misgeneralisation Risk Sub-Category · 7.1
Deceptive alignment Risk Sub-Category · 7.1
Malicious Uses Risk Category · 4.0
Offensive Cyber Operations (General) Risk Sub-Category · 4.2
AI-Powered Spear-Phishing at Scale Risk Sub-Category · 4.1