MIT AI Risk Repository
Browse AI risks
2,500 risk entries extracted from 74 frameworks, coded by domain, subdomain, causal entity, intent and timing. Filter, then export the current selection with its licence and citation attached.
-
"Models that have been trained on data encoded in multiple languages, such as LLMs trained on web-crawled data, may contain contamination that is obscured by translation [226]. The most basic form of this is when a benchmark is trans- lated to another language and then fed to the model as training data. The fact that the benchmark is translated before becoming training data can obscure the contamination from detection methods, giving false assurance that the model has generalized on the capabilities that the benchmark tests for."
-
"Guideline contamination refers to scenarios where instructions for the collec- tion, annotation, or use of the dataset are exposed to the model [170]. These instructions may contain explicit data-label pairs that can improve the model’s capabilities for the task."
-
"Annotation contamination refers to scenarios where the model is exposed to the benchmark labels during training [170]. This type of contamination can make the model learn the acceptable distribution of outputs. Combining this with raw data contamination of the test split, any evaluation made with the benchmark is invalidated because the entire test split is essentially leaked to the model."
-
"Once a model is deployed, it can be exposed to benchmark data provided by the users [95, 170]. The model may then be further trained by these user inputs containing benchmark data."
-
62.16.14 · Risk Sub-Category
Benchmark Inaccuracy (Benchmarks may not accurately evaluate capabilities)
"Benchmarks of AI systems can both underestimate and overestimate the capa- bilities of those AI systems. Underestimates can happen if an evaluation is not comprehensive enough, if the benchmark is saturated by existing models, or if the capabilities in question depend on a complicated setup, such as realistic computer programming tasks. Overestimates of capabilities can occur if an AI system is trained or fine-tuned on the contents of the benchmark, leading to overfitting."
-
"Benchmark saturation refers to benchmarks reaching their evaluation ceiling. The tendency towards benchmark saturation has been demonstrated in various benchmarks [19]. When benchmarks reach or are close to saturation, they stop being effective measures for new models, as more nuanced capability gains might not be detected."
-
62.16.16 · Risk Sub-Category
Benchmark Limitations (Insufficient benchmarks for AI safety evaluation)
"Benchmarks dedicated to measuring the performance of AI systems (e.g., on programming or math tasks) are more well-developed than those for assessing safety and harms in AI systems [234]. This gap can lead to AI systems excelling in specific tasks while exhibiting harmful behaviors that go undetected. More safety-related evaluation datasets can help in identifying previously overlooked undesirable model behaviors."
-
62.16.17 · Risk Sub-Category
Benchmark Limitations (Underestimating capabilities that are not covered by benchmarks)
"A lack of test coverage by benchmarks on specific abilities of a model can obscure the model’s capabilities from both the developer and the user [160]. This can lead to a false sense of safety and trust due to a lack of understanding of the model’s limitations."
-
62.17.00 · Risk Category
-
-
62.17.01 · Risk Sub-Category
Conflicts of interest in auditor selection
"Conflicts of interest can arise if there is no independence in the auditor selection process or if the auditors are closely associated with the developer [123, 157]. In such cases, the conflict of interest can appear even if third-party evaluators are involved. In the case of external auditing, the potential candidates might be selected from a narrow group of auditors, or have conflicting financial incentives for whether to report model shortcomings publicly."
-
"Auditors may not be able to address all of the specific safety, performance, or validation needs. Reports of passing audits may be more inclusive than can be justified due to a lack of knowledge of specific risks and how they can be tested, or a lack of capacity to perform sufficiently rigorous testing."
-
"Auditors may not publicly disclose risks they find, may be required to not pub- licize shortcomings, or may not receive sufficient cooperation from the relevant internal parties."
-
"Without accurate documentation on how a model's data was collected, curated, and used to train a model, it might be harder to satisfactorily explain the behavior of the model with respect to the data."
-
"Data provenance refers to tracing history of data, which includes its ownership, origin, and transformations. Without standardized and established methods for verifying where the data came from, there are no guarantees that the data is the same as the original source and has the correct usage terms."
-
"Determining who is responsible for an AI model is challenging without good documentation and governance processes."
-
"Insufficient documentation of the system that uses the model and the model’s purpose within the system in which it is used."
-
"Testing is unrepresentative when the test inputs are mismatched with the inputs that are expected during deployment."
-
"Since foundation models can be used for many purposes, a model’s intended use is important for defining the relevant risks of that model. As the use changes, the relevant risks might correspondingly change."
-
"Lack of data transparency is due to insufficient documentation of training or tuning dataset details. "
-
"A metric selected to measure or track a risk is incorrectly selected, incompletely measuring the risk, or measuring the wrong risk for the given context."
-
"AI model risks are socio-technical, so their testing needs input from a broad set of disciplines and diverse testing practices."
-
"Determining responsibility when EAI causes harm requires new accountability and liability frameworks that address the complexities of highly autonomous physical systems. Human users may disagree with decisions taken by expert EAI systems, raising significant questions of delegation and responsibility [108]. Lack of EAI accountability could lead to confusion for users and breakdowns in traditional justice systems [109]."
-
"EAI deployment could fundamentally reshape society, particularly if the speed of technological development outpaces society’s ability to adapt [103, 120]. For example, EAI systems could provide physical threats of violence and mass surveillance capabilities to back up AI-enabled authoritarianism [121]."
-
03.06.00 · Risk Category
"At a time of increasing climate urgency, energy consumption and the carbon footprint of AI applications are also matters of ethics and responsibility [68]. As with other energy-intensive technologies like proof-of-work blockchain, the call is to research more environmentally sustainable algorithms to offset the increasing use scale."
-
05.15.00 · Risk Category
Generative models are known for their substantial energy requirements, necessitating significant amounts of electricity, cooling water, and hardware containing rare metals. The extraction and utilization of these resources frequently occur in unsustainable ways. Consequently, papers highlight the urgency of mitigating environmental costs for instance by adopting renewable energy sources and utilizing energy-efficient hardware in the operation and training of generative AI systems.
-
10.09.00 · Risk Category
"The production process of these devices requires raw materials such as nickel, cobalt, and lithium in such high quantities that the Earth may soon no longer be able to sustain them in sufficient quantities."
-
depletion or contamination of natural resources, and damage to built environments... that may occur throughout the lifecycle of digital technologies [170, 237] from “crale (mining) to usage (consumption) to grave (waste)”
-
"The computing power used in training, testing, and deploying generative AI systems, especially large scale systems, uses substantial energy resources and thereby contributes to the global climate crisis by emitting greenhouse gasses."
-
"Impacts at a high-level, from the AI ecosystem to the Earth itself, are necessarily broad but can be broken down into components for evaluation."
-
The risk of harm to the natural environment posed by the ML system.
-
16.06.01 · Risk Sub-Category
Risk area 6: Environmental and Socioeconomic harms
Environmental harms from operating LMs
"LMs (and AI more broadly) can have an environmental impact at different levels, including: (1) direct impacts from the energy used to train or operate the LM, (2) secondary impacts due to emissions from LM-based applications, (3) system-level impacts as LM-based applications influence human behaviour (e.g. increasing environmental awareness or consumption), and (4) resource impacts on precious metals and other materials required to build hardware on which the computations are run e.g. data centres, chips, or devices. Some evidence exists on (1), but (2) and (3) will likely be more significant
-
17.06.01 · Risk Sub-Category
Automation, Access and Environmental Harms
Environmental harms from operation LMs
"Large-scale machine learning models, including LMs, have the potential to create significant environmental costs via their energy demands, the associated carbon emissions for training and operating the models, and the demand for fresh water to cool the data centres where computations are run (Mytton, 2021; Patterson et al., 2021)."
-
"Creating negative environmental impacts though model development and deployment"
-
31.06.00 · Risk Category
"the growing field of generative AI, which brings with it direct and severe impacts on our climate: generative AI comes with a high carbon footprint and similarly high resource price tag, which largely flies under the radar of public AI discourse. Training and running generative AI tools requires companies to use extreme amounts of energy and physical resources. Training one natural language processing model with normal tuning and experiments emits, on average, the same amount of carbon that seven people do over an entire year.121'
-
32.04.00 · Risk Category
Environmental harm, Sustainability
-
39.02.00 · Risk Category
Some learning algorithms, including deep learning, utilize iterative learning processes [23]. This approach results in high energy consumption.
-
41.06.00 · Risk Category
"AI is already helping to combat the impact of climate change with smart technology and sensors reducing emissions. However, it is also a key component in the development of nanobots, which could have dangerous environmental impacts by invisibly modifying substances at nanoscale."
-
41.06.01 · Risk Sub-Category
Accelerated development of nanotechnology produces uncontrolled production of toxic nanoparticles
"AI is a key component for the development of nanobots, which could have dangerous environmental implications by invisibly modifying substances at nanoscale. For example, nanobots could start chemical reactions that would create invisible nanoparticles that are toxic and potentially lethal."
-
44.01.00 · Risk Category
"Many intentional harms, including confinement, husbandry procedures like tail-docking, and slaughter, are legal or socially accepted, while others such as wildlife trafficking and violence against companion animals are generally socially condemned and often illegal. AI can be designed or adopted by humans who harm animals to pursue their goals more effectively. We therefore distinguish AI-facilitated intentional harms that are currently socially accepted and generally legal, from uses and abuses of AI that cause harms that are not socially accepted and are often illegal."
-
44.01.01 · Risk Sub-Category
Intentional: socially condemned/illegal
AI intentionally designed and used to harm animals in ways that contradict social values or are illegal
—
-
44.01.02 · Risk Sub-Category
Intentional: socially condemned/illegal
AI designed to benefit animals, humans, or ecosystems is intentionally abused to harm animals in ways that contradict social values or are illegal
—
-
44.02.00 · Risk Category
"AI designed to impact animals in harmful ways that reflect and amplify existing social values or are legal"
-
44.03.00 · Risk Category
"AI designed to benefit animals, humans, or ecosystems has unintended harmful impact on animals"
-
44.03.01 · Risk Sub-Category
AI is designed in a way that shows ignorant, reckless, or prejudiced lack of consideration for its impact on animals
—
-
44.03.02 · Risk Sub-Category
AI harms animals due to mistake or misadventure in the way the AI operates in practice
—
-
44.04.00 · Risk Category
"AI impacts human or ecological systems in ways that ultimately harm animals"
-
"AI proliferation causes harm to the environment through energy use and e-waste thereby destroying animal habitat"
-
"Replacement by AI of human observation and interaction leads to neglect of certain interests"
-
"Algorithmic recommender systems reinforce and amplify anthropocentric bias or desire of some people for animal cruelty as entertainment — leading to greater harm to animals through reinforcement of meat eating from factory farms, cruel uses of animals for entertainment, etc"
-
44.05.00 · Risk Category
"AI is disused (not developed or deployed) in directions that would benefit animals (and instead developments that harm or do no benefit to animals are invested in)"
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.