MIT AI Risk Repository · Risk Sub-Category · 67.04.03

Capabilities that could be used to reduce human control - Manipulation

Category: Loss of control

Open the MIT Risk Navigator All entries from DSIT2023

Description

"There is evidence that language models tend to respond as though they share the user’s stated views, and larger models do this more than smaller ones.276 The ability to predict people’s views and generate text that they will endorse could be useful for manipulation."

From Capabilities and Risks from Frontier AI (DSIT2023), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 7. AI system safety, failures, & limitations
Subdomain: 7.2 AI possessing dangerous capabilities
Causal entity: Other
Intent: Intentional
Timing: Post-deployment

Subdomain definition: AI systems that develop, access, or are provided with capabilities that increase their potential to cause mass harm through deception, weapons development and acquisition, persuasion and manipulation, political strategy, cyber-offense, AI development, situational awareness, and self-proliferation. These capabilities may cause mass harm due to malicious human actors, misaligned AI systems, or failure in the AI system.

How other frameworks describe this risk

Safety Risks from Affordances Provided to LLM-agents
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Agentic LLMs Pose Novel Risks
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Goal-Directedness Incentivizes Undesirable Behaviors
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Subagents
AGI Safety Literature Review (Everitt2018 )
AI Influence
The Ethics of Advanced AI Assistants (Gabriel2024)
Agency
Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024)
AI System bypassing a sandbox environment
Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024)
Fine-tuning related (Unexpected competence in fine-tuned versions of the upstream model)
Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024)

Other entries from DSIT2023

Societal harms Risk Category
Degradation of the information environment Risk Sub-Category · 3.2
Degradation of the information environment Additional evidence
Degradation of the information environment Additional evidence
Degradation of the information environment Additional evidence
Degradation of the information environment Additional evidence
Labour market disruption Risk Sub-Category · 6.2
Labour market disruption Additional evidence
Bias, Fairness and Representational Harms Risk Category · 1.1
Bias, Fairness and Representational Harms Additional evidence
Bias, Fairness and Representational Harms Additional evidence
Bias, Fairness and Representational Harms Additional evidence