MIT AI Risk Repository
Browse AI risks
554 risk entries extracted from 74 frameworks, coded by domain, subdomain, causal entity, intent and timing. Filter, then export the current selection with its licence and citation attached.
-
14.08.00 · Risk Category
"The technological maturity level describes how mature and error-free a certain technology is in a certain application context. If new technologies with a lower level of maturity are used in the development of the AI system, they may contain risks that are still unknown or difficult to assess.Mature technologies, on the other hand, usually have a greater variety of empirical data available, which means that risks can be identified and assessed more easily. However, with mature technologies, there is a risk that risk awareness decreases over time"
-
—
-
"Although competition between companies can be beneficial, creating more useful products for consumers, there are also pitfalls. First, the benefits of economic activity may be unevenly distributed, incentivizing those who benefit most from it to disregard the harms to others. Second, under intense market competition, businesses tend to focus much more on short-term gains than on long-term outcomes. With this mindset, companies often pursue something that can make a lot of profit in the short term, even if it poses a societal risk in the long term."
-
53.04.01 · Risk Sub-Category
Indirect AI contributions to existential risks
Destabilising political impacts from AI systems
"(e.g., polarization, legitimacy of elections), international political economy, or international security196 in terms of the balance of power, technology races and international stability, and the speed and character of war"
-
"AI development may itself become a new flash point for conflicts—causing more conflict to occur— especially conflicts over AI-relevant resources (such as data centres, semiconductor manufacturing facilities and raw materials)."
-
61.02.26 · Risk Sub-Category
Sources of systemic risks from general-purpose AI
Geopolitical competition for superiority
"Strategic competition between nations over AI capabilities could heighten global tensions and destabilize international relations."
-
61.02.27 · Risk Sub-Category
Sources of systemic risks from general-purpose AI
High-speed AI operations
"The fast operational speed of AI models and systems in competitive environments can lead to errors that are difficult to detect and correct in time."
-
68.06.00 · Risk Category
"As AI is increasingly seen as a powerful technology, countries are racing to develop it ahead of their geopolitical rivals, a competition that could lead to geopolitical tensions [138], [139]... The emphasis of this risk is on harms that result from second-order effects, where geopolitical instabilities result from the race to develop AI, rather than on the direct consequences of the deployment or use of AI itself."
-
01.01.00 · Risk Category
Societal-scale harm can arise from AI built by a diffuse collection of creators, where no one is uniquely accountable for the technology's creation or use, as in a classic "tragedy of the commons".
-
19.01.05 · Risk Sub-Category
Technological, Data and Analytical AI Risks
Lack of AI experts with comprehensive AI knowledge
—
-
19.06.00 · Risk Category
"Legal and regulatory risks comprise in particular the unclear definition of responsibilities and accountability in case of AI failures and autonomous decisions with negative impacts (Reed, 2018; Scherer, 2016). Another great risk in this context refers to overlooking the scope of AI governance and missing out on important governance aspects, resulting in negative consequences (Gasser & Almeida, 2017; Thierer et al., 2017)."
-
19.06.01 · Risk Sub-Category
Unclear definition of responsibilities and accountability for AI judgments and their consequences
—
-
19.06.03 · Risk Sub-Category
Great scope and ubiquity of AI make appropriate governance difficult, coverage of governance scope almost impossibl
—
-
accidents can cascade into catastrophes, can be caused by sudden unpredictable developments and it can take years to find severe flaws and risks (not a quote)
-
24.01.02 · Risk Sub-Category
Difficult to develop metrics for evaluating benefits or harms caused by AI assistants
"Another difficulty facing AI assistant systems is that it is challenging to develop metrics for evaluating particular aspects of benefits or harms caused by the assistant – especially in a sufficiently expansive sense, which could involve much of society (see Chapter 19). Having these metrics is useful both for assessing the risk of harm from the system and for using the metric as a training signal."
-
39.10.00 · Risk Category
HLI-based systems such as self-driving drones and vehicles will act autonomously in our world. In these systems, a challenging question is “who is liable when a self-driving system is involved in a crash or failure?”.
-
"Despite the promise of streamlined travel, AI also brings concerns about who is liable in case of accidents and which ethical principles autonomous transportation agents should follow when making decisions with a potentially dangerous impact to humans, for example, in case of an accident."
-
53.04.04 · Risk Sub-Category
Indirect AI contributions to existential risks
Erosion of international law and global governance architectures;
-
-
55.01.02 · Risk Sub-Category
Risks from accelerating scientific progress
Faster scientific progress makes it harder for governance to keep pace with development
"Exacerbating these problems is that faster scientific progress would make it even harder for governance to keep pace with the deployment of new technologies. When these technologies are especially powerful or dangerous, such as those discussed above, insufficient governance can magnify their harms.8 This is known as the pacing problem, and it is an issue that technology governance already faces [47], for a variety of reasons"
-
"The complex and rapidly evolving nature of AI makes them inherently difficult to govern effectively, leading to systemic regulatory and oversight failures."
-
"Harms could result from a combination of regulatory, management, and operational failures."
-
61.02.14 · Risk Sub-Category
Sources of systemic risks from general-purpose AI
Complex attribution and responsibility
"When multiple actors are involved in AI development and deployment, it becomes difficult to assign responsibility for harm, complicating accountability."
-
62.16.02 · Risk Sub-Category
General Evaluations (Limited coverage of capabilities evaluations)
"GPAI model developers might run capabilities evaluations to determine whether it has dangerous or dual-use capabilities, and then decide whether it is safe to deploy. Such capabilities evaluations can fail to demonstrate all the capabilities of a model. For example, evaluations may miss certain capabilities that are difficult to assess, prohibitively costly to verify, or obscured by the model’s tendency to refuse responses due to safety training, even if it possesses some of these capabilities."
-
62.16.06 · Risk Sub-Category
General Evaluations (Biased evaluations of encoded human values)
"Encoded human values in AI models that are easier to evaluate might be preferred for inclusion in evaluations over those that are more difficult to measure [13]. This might come at the expense of more desirable but harder-to-quantify values. This bias can lead to an imbalance, where easier-to-measure values dominate the evaluation process, while other important values are underrepresented."
-
62.16.08 · Risk Sub-Category
Benchmarking (Benchmark leakage or data contamination)
"Benchmark leakage [235, 224, 221, 161] can happen when an AI model is trained or fine-tuned with evaluation-related data. This can lead to an unreliable model evaluation, especially if the data contains question-answer pairs from bench- marks."
-
"This type of contamination [170] occurs when the raw and unlabeled data of a benchmark is used as part of the training set. Such data may not be properly formatted and may contain noise, especially if the contamination happens before the data is pre-processed into the benchmark. If this contamination occurs, it could cast doubt on the few-shot and zero-shot performance of the model on that benchmark."
-
"Models that have been trained on data encoded in multiple languages, such as LLMs trained on web-crawled data, may contain contamination that is obscured by translation [226]. The most basic form of this is when a benchmark is trans- lated to another language and then fed to the model as training data. The fact that the benchmark is translated before becoming training data can obscure the contamination from detection methods, giving false assurance that the model has generalized on the capabilities that the benchmark tests for."
-
"Guideline contamination refers to scenarios where instructions for the collec- tion, annotation, or use of the dataset are exposed to the model [170]. These instructions may contain explicit data-label pairs that can improve the model’s capabilities for the task."
-
"Annotation contamination refers to scenarios where the model is exposed to the benchmark labels during training [170]. This type of contamination can make the model learn the acceptable distribution of outputs. Combining this with raw data contamination of the test split, any evaluation made with the benchmark is invalidated because the entire test split is essentially leaked to the model."
-
"Once a model is deployed, it can be exposed to benchmark data provided by the users [95, 170]. The model may then be further trained by these user inputs containing benchmark data."
-
62.16.14 · Risk Sub-Category
Benchmark Inaccuracy (Benchmarks may not accurately evaluate capabilities)
"Benchmarks of AI systems can both underestimate and overestimate the capa- bilities of those AI systems. Underestimates can happen if an evaluation is not comprehensive enough, if the benchmark is saturated by existing models, or if the capabilities in question depend on a complicated setup, such as realistic computer programming tasks. Overestimates of capabilities can occur if an AI system is trained or fine-tuned on the contents of the benchmark, leading to overfitting."
-
"Auditors may not be able to address all of the specific safety, performance, or validation needs. Reports of passing audits may be more inclusive than can be justified due to a lack of knowledge of specific risks and how they can be tested, or a lack of capacity to perform sufficiently rigorous testing."
-
"Without accurate documentation on how a model's data was collected, curated, and used to train a model, it might be harder to satisfactorily explain the behavior of the model with respect to the data."
-
"Testing is unrepresentative when the test inputs are mismatched with the inputs that are expected during deployment."
-
"Since foundation models can be used for many purposes, a model’s intended use is important for defining the relevant risks of that model. As the use changes, the relevant risks might correspondingly change."
-
"Lack of data transparency is due to insufficient documentation of training or tuning dataset details. "
-
"A metric selected to measure or track a risk is incorrectly selected, incompletely measuring the risk, or measuring the wrong risk for the given context."
-
"AI model risks are socio-technical, so their testing needs input from a broad set of disciplines and diverse testing practices."
-
"Determining responsibility when EAI causes harm requires new accountability and liability frameworks that address the complexities of highly autonomous physical systems. Human users may disagree with decisions taken by expert EAI systems, raising significant questions of delegation and responsibility [108]. Lack of EAI accountability could lead to confusion for users and breakdowns in traditional justice systems [109]."
-
03.06.00 · Risk Category
"At a time of increasing climate urgency, energy consumption and the carbon footprint of AI applications are also matters of ethics and responsibility [68]. As with other energy-intensive technologies like proof-of-work blockchain, the call is to research more environmentally sustainable algorithms to offset the increasing use scale."
-
05.15.00 · Risk Category
Generative models are known for their substantial energy requirements, necessitating significant amounts of electricity, cooling water, and hardware containing rare metals. The extraction and utilization of these resources frequently occur in unsustainable ways. Consequently, papers highlight the urgency of mitigating environmental costs for instance by adopting renewable energy sources and utilizing energy-efficient hardware in the operation and training of generative AI systems.
-
10.09.00 · Risk Category
"The production process of these devices requires raw materials such as nickel, cobalt, and lithium in such high quantities that the Earth may soon no longer be able to sustain them in sufficient quantities."
-
depletion or contamination of natural resources, and damage to built environments... that may occur throughout the lifecycle of digital technologies [170, 237] from “crale (mining) to usage (consumption) to grave (waste)”
-
"The computing power used in training, testing, and deploying generative AI systems, especially large scale systems, uses substantial energy resources and thereby contributes to the global climate crisis by emitting greenhouse gasses."
-
The risk of harm to the natural environment posed by the ML system.
-
16.06.01 · Risk Sub-Category
Risk area 6: Environmental and Socioeconomic harms
Environmental harms from operating LMs
"LMs (and AI more broadly) can have an environmental impact at different levels, including: (1) direct impacts from the energy used to train or operate the LM, (2) secondary impacts due to emissions from LM-based applications, (3) system-level impacts as LM-based applications influence human behaviour (e.g. increasing environmental awareness or consumption), and (4) resource impacts on precious metals and other materials required to build hardware on which the computations are run e.g. data centres, chips, or devices. Some evidence exists on (1), but (2) and (3) will likely be more significant
-
17.06.01 · Risk Sub-Category
Automation, Access and Environmental Harms
Environmental harms from operation LMs
"Large-scale machine learning models, including LMs, have the potential to create significant environmental costs via their energy demands, the associated carbon emissions for training and operating the models, and the demand for fresh water to cool the data centres where computations are run (Mytton, 2021; Patterson et al., 2021)."
-
"Creating negative environmental impacts though model development and deployment"
-
31.06.00 · Risk Category
"the growing field of generative AI, which brings with it direct and severe impacts on our climate: generative AI comes with a high carbon footprint and similarly high resource price tag, which largely flies under the radar of public AI discourse. Training and running generative AI tools requires companies to use extreme amounts of energy and physical resources. Training one natural language processing model with normal tuning and experiments emits, on average, the same amount of carbon that seven people do over an entire year.121'
-
39.02.00 · Risk Category
Some learning algorithms, including deep learning, utilize iterative learning processes [23]. This approach results in high energy consumption.
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.