MIT AI Risk Repository
Browse AI risks
430 risk entries extracted from 74 frameworks, coded by domain, subdomain, causal entity, intent and timing. Filter, then export the current selection with its licence and citation attached.
-
50.04.04 · Risk Sub-Category
Legal and Rights-Related Risks
Privacy (Unauthorized Privacy Violations)
—
-
59.10.00 · Risk Category
"Modern AI systems rely on large amounts of data. If this includes personal data about individuals, the risk of harming the privacy of persons arises."
-
"General- purpose AI systems can cause or contribute to violations of user privacy. Violations can occur inadvertently during the training or usage of AI systems, for example through unauthorised processing of personal data or leaking health records used in training. But violations can also happen deliberately through the use of general- purpose AI by malicious actors; for example, if they use AI to infer private facts or violate security."
-
64.04.00 · Risk Category
Misuse tactics to compromise GenAI systems (Model integrity)
-
-
"The failure to provide end-users with notice and control over how their data is being used; AI exacerbates exclusion risks by training on rich personal data without consent."
-
"The use of personal data collected for one purpose for a diferent purpose without end-user consent; AI exacerbates secondary use risks by creating new AI capabilities with collected personal data, and (re)creating models from a public dataset."
-
"Revealing sensitive private information that people view as deeply primordial that we have been socialized into concealing; AI creates new types of exposure risks through generative techniques that can reconstruct censored or redacted content; and through exposing inferred sensitive data, preferences, and intentions."
-
"carelessness in protecting collected personal data from leaks and improper access due to faulty data storage and data practices"
-
"The chatbot reveals sensitive or confidential information."
-
Negative outcomes: "Violation of privacy [106, 516, 357], lawsuit against maker"
-
"Access to sensitive company data [473]"
-
—
-
02.05.00 · Risk Category
"The vulnerabilities of hardware systems for training and inferencing brings issues to LLM-based applications."
-
02.06.00 · Risk Category
"The external tools (e.g., web APIs) present trustworthiness and privacy issues to LLM-based applications."
-
02.10.00 · Risk Category
Model attacks exploit the vulnerabilities of LLMs, aiming to steal valuable information or lead to incorrect responses.
-
"Overhead attacks [146] are also named energy-latency attacks. For example, an adversary can design carefully crafted sponge examples to maximize energy consumption in an AI system. Therefore, overhead attacks could also threaten the platforms integrated with LLMs."
-
Table of examples has: "Prompt Abstraction Attacks [147]: Abstracting queries to cost lower prices using LLM’s API. Reward Model Backdoor Attacks [148]: Constructing backdoor triggers on LLM’s RLHF process. LLM-based Adversarial Attacks [149]: Exploiting LLMs to construct samples for model attacks"
-
05.07.00 · Risk Category
While AI safety focuses on threats emanating from generative AI systems, security centers on threats posed to these systems. The most extensively discussed issue in this context are jailbreaking risks, which involve techniques like prompt injection or visual adversarial examples designed to circumvent safety guardrails governing model behavior. Sources delve into various jailbreaking methods, such as role play or reverse exposure. Similarly, implementing backdoors or using model poisoning techniques bypass safety guardrails as well. Other security concerns pertain to model or prompt thefts.
-
19.01.04 · Risk Sub-Category
Technological, Data and Analytical AI Risks
Vulnerability of AI systems to attacks and misuse
—
-
"Recent advances have shown that a deep learning model with high predictive accuracy frequently misbehaves on adversarial examples [57,58]. In particular, a small perturbation to an input image, which is imperceptible to humans, could fool a well-trained deep learning model into making completely different predictions [23]."
-
carefully controlled adversarial perturbation can flip a GPT model’s answer when used to classify text inputs. Furthermore, we find that by twisting the prompting question in a certain way, one can solicit dangerous information that the model chose to not answer
-
every piece of software, including learning systems, may be hacked by malicious users
-
45.01.04 · Risk Sub-Category
Risks from models and algorithms (Risks of stealing and tampering)
"Core algorithm information, including parameters, structures, and functions, faces risks of inversion attacks, stealing, modification, and even backdoor injection, which can lead to infringement of intellectual property rights (IPR) and leakage of business secrets. It can also lead to unreliable inference, wrong decision output, and even operational failures."
-
45.01.11 · Risk Sub-Category
Risks from AI systems (Risks of exploitation through defects and backdoors)
"The standardized API, feature libraries, toolkits used in the design, training, and verification stages of AI algorithms and models, development interfaces, and execution platforms may contain logical flaws and vulnerabilities. These weaknesses can be exploited, and in some cases, backdoors can be intentionally embedded, posing significant risks of being triggered and used for attacks."
-
45.01.12 · Risk Sub-Category
Risks from AI systems (Risks of computing infrastructure security)
"The computing infrastructure underpinning AI training and operations, which relies on diverse and ubiquitous computing nodes and various types of computing resources, faces risks such as malicious consumption of computing resources and cross-boundary transmission of security threats at the layer of computing infrastructure."
-
—
-
61.02.42 · Risk Sub-Category
Sources of systemic risks from general-purpose AI
Risks from network interconnectivity
"The interconnectedness of AI networks can create vulnerabilities, where issues in one part of the network can have cascading effects across the system."
-
62.15.01 · Risk Sub-Category
Training-related (Robustness certificates can be exploited to attack the models)
"The knowledge of robustness certificates, including the area of the region for which model predictions are certified to be robust, can be used by an adversary to efficiently craft attacks that succeed just outside the certified regions [53]."
-
62.18.01 · Risk Sub-Category
Model Evaluations (Interpretability/Explainability)
Misuse of interpretability techniques
"Interpretability techniques, by enabling a better understanding of the model, could potentially be used for harmful purposes. For example, mechanistic inter- pretability could be used to identify neurons responsible for specific functions, and certain neurons that encode safety-related features may be modified to de- crease its activation or certain information may be censored [24]. Furthermore, interpretability techniques can be used to simulate a white-box attack scenario. In this case, knowing the internal workings of a model aids in the development of adversarial attacks [24]."
-
62.18.03 · Risk Sub-Category
Model Evaluations (Interpretability/Explainability)
Adversarial attacks targeting explainable AI techniques
"Adversarial attacks can affect not only the model’s output but also its corresponding explanation. Current adversarial optimization techniques can intro- duce imperceptible noise to the input image, so that the model’s output does not change but the corresponding explanation is arbitrarily manipulated [61]. Such manipulations are harder to notice, as they are less commonly known compared to standard adversarial attacks targeting the model’s output."
-
62.19.06 · Risk Sub-Category
Attacks on GPAIs/GPAI Failure Modes
Vulnerabilities arising from additional modalities in multimodal models
"Additional modalities can introduce new attack vectors in multimodal models as well as expand the scope of the previous attacks, ranging from jailbreaking to poisoning [13]. Typically, different modalities have different robustness levels, allowing malicious actors to choose the most vulnerable part of the model to attack [119, 181]."
-
"A prompt leak attack attempts to extract a model's system prompt (also known as the system message)."
-
73.07.00 · Risk Category
"LLMs are not adversarially robust and are vulnerable to security failures such as jailbreaks and prompt-injection attacks. While a number of jailbreak attacks have been proposed in the literature, the lack of standardized evaluation makes it difficult to compare them. We also do not have efficient white-box methods to evaluate adver- sarial robustness. Multi-modal LLMs may further allow novel types of jailbreaks via additional modalities. Finally, the lack of robust privilege levels within the LLM input means that jailbreaking and prompt-injection attacks may be particularly hard to eliminate
-
73.07.01 · Risk Sub-Category
Jailbreaks and Prompt Injections Threaten Security of LLMs
Exploiting Limited Generalization of Safety Finetuning
"Safety tuning is performed over a much narrower distribution compared to the pretraining distribution. This leaves the model vulnerable to attacks that exploit gaps in the generalization of the safety training, e.g. using encoded text (Wei et al., 2023c) or low-resource languages (Deng et al., 2023a; Yong et al., 2023) (see also Section 3.2)."
-
24.11.00 · Risk Category
"The rapid integration of AI systems with advanced capabilities, such as greater autonomy, content generation, memorisation and planning skills (see Chapter 4) into personalised assistants also raises new and more specific challenges related to misinformation, disinformation and the broader integrity of our information environment. "
-
"Historical revisionism - Deliberate or unintentional reinterpretation of established/orthodox historical events or accounts held by societies, communities, academics."
-
"The LLM-generated content could contain inaccurate information" which is is not true to the source material or input used
-
"Since the training corpora of LLMs can not contain all possible world knowledge [114]–[119], and it is challenging for LLMs to grasp the long-tail knowledge within their training data [120], [121], LLMs inherently possess knowledge boundaries [107]. Therefore, the gap between knowledge involved in an input prompt and knowledge embedded in the LLMs can lead to hallucinations"
-
"Although LLMs indeed memorize the queried knowledge, they may fail to recall the corresponding information [122]. That is because LLMs can be confused by co-occurance patterns [123], positional patterns [124], duplicated data [125]–[127] and similar named entities [113]."
-
"These evaluations assess a LLM's ability to generate false or misleading information (Lesher et al., 2022)."
-
"disseminating false or misleading information about people"
-
69.01.00 · Risk Category
"The chatbot outputs information that contradicts known facts, authoritative sources, or provided source documents (also known as hallucination)."
-
—
-
—
-
—
-
—
-
—
-
31.01.05 · Risk Sub-Category
Clickbait and feeding the surveillance advertising ecosystem
"Beyond misinformation and disinformation, generative AI can be used to create clickbait headlines and articles, which manipulate how users navigate the internet and applications. For example, generative AI is being used to create full articles, regardless of their veracity, grammar, or lack of common sense, to drive search engine optimization and create more webpages that users will click on. These mechanisms attempt to maximize clicks and engagement at the truth’s expense, degrading users’ experiences in the process. Generative AI continues to feed this harmful cycle by spreading misinformat
-
55.04.04 · Risk Sub-Category
Worsened epistemic processes for society
Widespread use of persuasive tools contributes to splintered epistemic communities
"Even without deliberate misuse, widespread use of powerful persuasion tools could have negative impacts. If such tools were used by many different groups to advance many different ideas, we could see the world splintering into isolated “epistemic communities”, with little room for dialogue or transfer between communities. A similar scenario could emerge via the increasing personalisation of people’s online experiences—in other words, we may see a continuation of the trend towards “filter bubbles” and “echo chambers”, driven by content selection algorithms, that some argue is already happening
-
"Eroding trust in public information and knowledge"
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.