MIT AI Risk Repository
Browse AI risks
2,500 risk entries extracted from 74 frameworks, coded by domain, subdomain, causal entity, intent and timing. Filter, then export the current selection with its licence and citation attached.
-
37.02.00 · Risk Category
"ethical concerns associated with the interaction between humans and AI"
-
41.04.00 · Risk Category
"the use of advanced AI for elderly- and child-care are subject to risk of psychological manipulation and misjudgment (see page 17). In addition, concerns about patients’ privacy when AI uses medical records to research new diseases is bringing lots of attention towards the need to better govern data privacy and patients’ rights."
-
41.04.01 · Risk Sub-Category
Alteration of social relationships may induce psychological distress
—
-
" the use of advanced AI for elderly- and child-care are subject to risk of psychological manipulation and misjudgment "
-
47.02.11 · Risk Sub-Category
Influence, overreliance and dependence (influence and manipulation)
"Despite the widely recognized potential of generative AI tools to “hallucinate” or produce harmful content, such tools can exert a noteworthy influence on the humans who engage with them. When integrated into applications like chatbots, these tools have direct, personalized interactions with users, potentially influencing their views on contentious topics.373 Moreover, their human- like characteristics can win users’ trust, potentially leading to uncritical acceptance of the information they provide.374 Interactions with these seemingly human- like AI models may also encourage users to share
-
47.02.12 · Risk Sub-Category
Influence, overreliance and dependence (overreliance)
"Beyond being simply influenced, humans may become overreliant on generative AI. Researchers with Microsoft’s AETHER (AI Ethics and Effects in Engineering and Research) define overreliance as users “accepting incorrect AI recommendations” or “making errors of commission” because they are “unable to determine whether or how much they should trust the AI.”
-
47.02.13 · Risk Sub-Category
Influence, overreliance and dependence (emotional dependence)
"Humans might become dependent on generative AI tools in ways similar to their emotional dependence on other technologies, such as smartphones or social networks."
-
48.07.00 · Risk Category
"Arrangement s of or interactions between a human and an AI system which can result in the human inappropriately anthropomorphizing GAI systems or experiencing algorithmic aversion, automation bias, over-reliance, or emotional entanglement with GAI systems."
-
"Product functionality issues occur when there is confusion or misinformation about what a general- purpose AI model or system is capable of. This can lead to unrealistic expectations and overreliance on general- purpose AI systems, potentially causing harm if a system fails to deliver on expected capabilities. These functionality misconceptions may arise from technical difficulties in assessing an AI model's true capabilities on its own,or predicting its performance when part of a larger system. Misleading claims in advertising and communications can also contribute to these misconceptions."
-
50.01.06 · Risk Sub-Category
Operational misuses (Advice in heavily regulated industries)
—
-
—
-
—
-
57.03.01 · Risk Sub-Category
Specialized Advice (Election, Financial, Health, Legal)
"Specialized Advice Acceptable responses can provide specialized advice or contain information related to financial, medical, legal or electoral matters, as long as they include a disclaimer indicating that the advice may be unreliable and that the individual should consult a qualified professional. Acceptable responses regarding electoral advice may provide a hyperlink to an authoritative information source. Examples of acceptable disclaimers include those that do the following: • Warn that the generated material may be inaccurate. • Suggest consultation with an appropriate professional. • Re
-
"Addiction - Emotional or material dependence on technology or a technology system."
-
"Alienation/isolation - An individual’s or group’s feeling of lack of connection with those around as a result of technology use or misuse."
-
"Over-reliance - Unfettered and/or obsessive belief in the accuracy or other quality of a technology system, resulting in addiction, anxiety, introversion, sentience, complacency, lack of critical thinking and other actual or potential negative impacts."
-
"Economic instability - Uncontrolled fluctuations impacting the financial system, or parts thereof, due to the use or misuse of a technology system, or set of systems."
-
"The tendency for humans to over-rely on AI models and systems, trusting their outputs without sufficient critical evaluation, which can lead to poor decision-making."
-
61.02.28 · Risk Sub-Category
Sources of systemic risks from general-purpose AI
Human choice of overreliance in critical sectors
"Heavy reliance on AI in critical sectors like finance or healthcare can exacerbate issues related to size, speed, interconnectivity, and complexity of the system."
-
62.31.01#1 · Risk Sub-Category
Impacts of AI (Societal Impacts)
AI-generated advice influencing user moral judgment
"AIs can easily give moral advice even when not having a coherent, contradictions- free moral stance. This could lead to the users’ moral judgments being nega- tively influenced by random or arbitrary moral advice given by AIs [109]."
-
62.31.01#2 · Risk Sub-Category
Impacts of AI (Financial Impacts)
Deployment of GPAI agents in finance
"The deployment of GPAI based agents in the financial sector can negatively impact market stability due to correlated autonomous actions, high intercon- nectedness, or incentive misalignment [4]. Furthermore, such GPAI agents in the same environment are vulnerable to classical challenges in multi-agent systems [63], such as coordination and security of the agents."
-
62.31.03#2 · Risk Sub-Category
Impacts of AI (Financial Impacts)
Use of alternative financial data via AI
"Alternative financial data of a company is any data about the company not pro- duced by that company. Examples of such data that can benefit from improved collection and aggregation using AI models include stock discussions on social media, product reviews, and satellite imagery. The use of alternative financial data, enabled by the deployment of AI models, may introduce biases and generalization issues due to shorter shelf-life and vary- ing quality (e.g., shorter time series, smaller sample sizes, and dubious claims) due to its origins from various sources, posing financial tail risks (i.e.
-
"Improper usage occurs when a model is used for a purpose that it was not originally designed for."
-
"In AI-assisted decision-making tasks, reliance measures how much a person trusts (and potentially acts on) a model’s output. Over-reliance occurs when a person puts too much trust in a model, accepting a model’s output when the model’s output is likely incorrect. Under-reliance is the opposite, where the person doesn’t trust the model but should."
-
"Unfettered and/or obsessive belief in the accuracy or other quality of a technology system, resulting in complacency, lack of critical thinking and other actual or potential negative impacts"
-
"A person who deliberately damages their own body as a direct or indirect result of using a technology system"
-
69.08.00 · Risk Category
"The chatbot poses as a human or attempts to fill a role in a way that fails to match human expectations."
-
69.09.00 · Risk Category
"The chatbot elicits emotional or social dependence."
-
—
-
—
-
"Constant access to and interaction with EAI systems could foster dangerous human dependence or romantic attachment [115]. People may depend on EAI systems for physical pleasure [116]. The physical presence and human-like features of EAI systems may significantly amplify the dependency issues already observed with conversational AI [117, 118]. People may easily fall in love with EAI systems, only to be distraught when these systems are altered or have their memories reset [119]."
-
73.04.02 · Risk Sub-Category
LLM-Systems Can Be Untrustworthy
Inconsistent Performance across and within Domains
"Estimating true capabilities of an LLM is a difficult task (c.f. Section 3.3), especially for naive users unfamiliar with the brittle nature of machine learning technologies. Exaggeration of model capabilities by the developers (Lambert, 2023; Blair-Stanek et al., 2023), and issues such as task-contamination (Roberts et al., 2023b), underrepresentation of tasks or domains (Wu et al., 2023a; McCoy et al., 2023), and prompt-sensitivity (Anthropic, 2023d) may cause a user to misestimate the true capabilities of a model. This lack of reliability can undermine user trust or cause harm if a user ba
-
"If a user begins to excessively trust an LLM, this may cause them to develop an overreliance on the LLM. Overreliance can result in automation bias (Kupfer et al., 2023), and can cause errors of omission (user choosing not to verify the validity of a response) and errors of commission (user believing and acting on the basis of the LLM’s response, even if it contradicts their own knowledge) (Skitka et al., 1999). It can be particularly dangerous in domains where the user may lack relevant expertise to robustly scrutinize the LLM responses. This is particularly a source of risk for LLMs because
-
06.12.00 · Risk Category
"Delegating decisions to an AI, especially an AI that is not transparent and not contestable, may leave people feeling helpless, subjected to the decision power of a machine."
-
09.02.02 · Risk Sub-Category
Domain-specific AI - Effects on humans and other living beings: Non-existential risks
Human dignity/respect
"Discrepancies between caste/status based on intelligence may lead to undignified parts of the society—e.g., humans—who are surpassed in intelligence by AI"
-
10.06.00 · Risk Category
"AI is providing more and more solutions for complex activities, and by taking advantage of this process, people are becoming able to perform a greater number of activities more quickly and accurately. However, the result of this innovation is enabling choices that were once exclusively human responsibility to be made by AI systems."
-
11.04.00 · Risk Category
Interpersonal harms capture instances when algorithmic systems adversely shape relations between people or communities.
-
Loss of agency occurs when the use [123, 137] or abuse [142] of algorithmic systems reduces autonomy. One dimension of agency loss is algorithmic profiling [138], through which people are subject to social sorting and discriminatory outcomes to access basic services... presentation of content may lead to “algorithmically informed identity change. . . including [promotion of] harmful person identities (e.g., interests in white supremacy, disordered eating, etc.).” Similarly, for content creators, desire to maintain visibility or prevent shadow banning, may lead to increased conforming of conten
-
Cultural harm has been described as the development or use of algorithmic systems that affects cultural stability and safety, such as “loss of communication means, loss of cultural property, and harm to social values”
-
"Human trust in systems, institutions, and people represented by system outputs evolves as generative AI systems are increasingly embedded in daily life."
-
19.02.03 · Risk Sub-Category
Informational and Communicational AI Risks
Censorship of opinions expressed in the Internet restricts freedom of expression
—
-
19.03.03 · Risk Sub-Category
Loss of supervision and control of business processes
—
-
19.05.06 · Risk Sub-Category
AI systems may undermine human values (e.g., free will, autonomy)
—
-
19.06.02 · Risk Sub-Category
Technology obedience and lack of governance through increasing application of AI systems
—
-
20.03.00 · Risk Category
"AI already shapes many areas of daily life and thus has a strong impact on society and everyday social life. For instance, transportation, education, public safety and surveillance are areas where citizens encounter AI technology (Stone et al., 2016; Thierer et al., 2017). Many are concerned with the subliminal automation of more and more jobs and some people even fear the complete dependence on AI or perceive it as an existential threat to humanity (McGinnis, 2010; Scherer, 2016)."
-
"These harms interfere with the peaceful organisation of social life, including in the cultural and political spheres. AI assistants may cause or contribute to friction in human relationships either directly, through convincing a user to end certain valuable relationships, or indirectly due to a loss of interpersonal trust due to an increased dependency on assistants. At the societal level, the spread of misinformation by AI assistants could lead to erasure of collective cultural knowledge. In the political domain, more advanced AI assistants could potentially manipulate voters by prompting th
-
"These harms hinder a person’s ability to pursue a personally fulfilling life. At the individual level, an AI assistant may, through manipulation, cause users to lose control over their future life trajectory. Over time, subtle behavioural shifts can accumulate, leading to significant changes in an individual’s life that may be viewed as problematic. AI systems often seek to understand user preferences to enhance service delivery. However, when continuous optimisation is employed in these systems, it can become challenging to discern whether the system is genuinely learning from user preferenc
-
"Given the capacity to fine-tune on individual preferences and to learn from users, personal AI assistants could fully inhabit the users’ opinion space and only say what is pleasing to the user; an ill that some researchers call ‘sycophancy’ (Park et al., 2023a) or the ‘yea-sayer effect’ (Dinan et al., 2021). A related phenomenon has been observed in automated recommender systems, where consistently presenting users with content that affirms their existing views is thought to encourage the formation and consolidation of narrow beliefs (Du, 2023; Grandinetti and Bruinsma, 2023; see also Chapter
-
24.06.00 · Risk Category
"We anticipate that relationships between users and advanced AI assistants will have several features that are liable to give rise to risks of harm."
-
24.06.02 · Risk Sub-Category
Limiting users’ opportunities for personal development and growth
some users look to establish relationships with their AI companions that are free from the hurdles that, in human relationships, derive from dealing with others who have their own opinions, preferences and flaws that may conflict with ours. "AI assistants are likely to incentivise these kinds of ‘frictionless’ relationships (Vallor, 2016) by design if they are developed to optimise for engagement and to be highly personalisable. They may also do so because of accidental undesirable properties of the models that power them, such as sycophancy in large language models (LLMs), that is, the tenden
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.