MIT AI Risk Repository · domain 5: Human-Computer Interaction
5.2 Loss of human agency and autonomy
Humans delegating key decisions to AI systems, or AI systems making decisions that diminish human control and autonomy, potentially leading to humans feeling disempowered, losing the ability to shape a fulfilling life trajectory or becoming cognitively enfeebled.
- 47
- 12
- 4
- 4
| Label | Value |
|---|---|
| Other | 24 |
| Human | 12 |
| AI | 11 |
| Label | Value |
|---|---|
| Other | 22 |
| Unintentional | 19 |
| Intentional | 6 |
| Label | Value |
|---|---|
| Post-deployment | 31 |
| Other | 16 |
| Label | Value |
|---|---|
| 2021 | 2 |
| 2024 | 1 |
| 2026 | 1 |
| Label | Value |
|---|---|
| Risk Category | 13 |
| Risk Sub-Category | 33 |
| Additional evidence | 1 |
Risk entries
Browse and export all- Autonomy
"Autonomy - Loss of or restrictions to the ability or rights of an individual, group or entity to make decisions and control their identity and/or output."
- Autonomy/agency loss
"Autonomy/agency loss - Loss of an individual, group or organisation’s ability to make informed decisions or pursue goals."
- Personality rights loss
"Personality rights loss - Loss of or restrictions to the rights of an individual to control the commercial use of their identity, such as name, image, likeness, or other unequivocal identifiers."
- Gradual loss of control
"Gradual or accumulative loss of control risks can be described as risks resulting from the accumulation of less severe disruptions that gradually weakens systemic resilience until a critical event tr...
- AI enables automation of military decision-making
"One concern here is humans not remaining in the loop for some military decisions, creating the possibility of unintentional escalation because of: • Automated tactical decision-making, by ‘in-theatre...
- AI-induced strategic instability
"For example, AI could undermine nuclear strategic stability by making it easier to discover and destroy previously secure nuclear launch facilities [30, 46, 49]. AI may also offer more extreme first-...
- -
"Humans may increasingly hand over control of important decisions to AI systems, due to economic and geopolitical incentives. Some experts are concerned that future advanced AI systems will seek to in...
- Humans might increasingly hand over control to misaligned AI systems
"Organisations around the world are already deploying misaligned AI systems that are causing harm in unexpected ways.250 Recommendation algorithms increase the consumption of extremist content.251 Med...
- Sociocultural and Political Harms
"These harms interfere with the peaceful organisation of social life, including in the cultural and political spheres. AI assistants may cause or contribute to friction in human relationships either d...
- Self-Actualisation Harms
"These harms hinder a person’s ability to pursue a personally fulfilling life. At the individual level, an AI assistant may, through manipulation, cause users to lose control over their future life tr...
- Disorientation
"Given the capacity to fine-tune on individual preferences and to learn from users, personal AI assistants could fully inhabit the users’ opinion space and only say what is pleasing to the user; an il...
- Appropriate Relationships
"We anticipate that relationships between users and advanced AI assistants will have several features that are liable to give rise to risks of harm."
- Limiting users’ opportunities for personal development and growth
some users look to establish relationships with their AI companions that are free from the hurdles that, in human relationships, derive from dealing with others who have their own opinions, preference...
- Generating material dependence without adequate commitment to user needs
"In addition to emotional dependence, user–AI assistant relationships may give rise to material dependence if the relationships are not just emotionally difficult but also materially costly to exit. F...
- Collective action problems
"Collective action problems are ubiquitous in our society (Olson Jr, 1965). They possess an incentive structure in which society is best served if everyone cooperates, but where an individual can achi...
- Overreliance on AI system undermining user autonomy
"AI systems can undermine human autonomy, if they allow for habitually trusting the AI’s suggestions without sufficient exercising of human agency. Over time, a user may develop unjustified trust in o...
- Long-term effects of AI model biases on user judgment
"The initial user exposure to model biases can have a lasting impact beyond the initial interaction with the model. Users who encounter biases in AI models can be affected by and continue to exhibit p...
- Overreliance
"As AI capability increases, humans grant AI more control over critical systems and eventually become irreversibly dependent on systems they don’t fully understand. Failure and unintended outcomes can...
- Enfeeblement
As AI systems encroach on human-level intelligence, more and more aspects of human labor will become faster and cheaper to accomplish with AI. As the world accelerates, organizations may voluntarily c...
- Loss of Autonomy
"Delegating decisions to an AI, especially an AI that is not transparent and not contestable, may leave people feeling helpless, subjected to the decision power of a machine."
- Impact on human agency
"AI might affect the individuals’ ability to make choices and act independently in their best interests."
- Human–AI interaction
"Several participants mentioned how AI systems could influence human agency and decision-making. They emphasized the need of striking a balance between using the benefits of AI and protecting human au...
- -
"Loss of or restrictions to the ability or rights of an individual, group or entity to make decisions and control their identity and/or output due to the use of misuse of a technology system or set of...
- Autonomy / agency loss
"Loss of an individual, group or organisation’s ability to make informed decisions or pursue goals"
- Addiction
"Emotional or material dependence on technology or a technology system"