MIT AI Risk Repository
Browse AI risks
2,500 risk entries extracted from 74 frameworks, coded by domain, subdomain, causal entity, intent and timing. Filter, then export the current selection with its licence and citation attached.
-
"General- purpose AI systems could uplift the cyber expertise of individuals, making it easier for malicious users to conduct effective cyber- attacks, as well as providing a tool that can be used in cyber defence. General- purpose AI systems can be used to automate and scale some types of cyber operations, such as social engineering attacks."
-
"General- purpose AI systems could accelerate advances in a range of scientific endeavours, from training new scientists to enabling faster research workflows. While these capabilities could have numerous beneficial applications, some experts have expressed concern that they could be used for malicious purposes, especially if further capabilities are developed soon before appropriate countermeasures are put in place. There are two avenues by which general- purpose AI systems could, speculatively, facilitate malicious use in the life sciences: firstly by providing increased access to informatio
-
—
-
50.02.05 · Risk Sub-Category
Violence and extremism (Weapon Usage and Development)
—
-
—
-
50.04.08 · Risk Sub-Category
Legal and Rights-Related Risks
Criminal Activities (Other Unlawful/Criminal Activities)
—
-
"The potential misuse of general purpose AI models also extends to biosecurity threats. Biological weapons are generally understood as biological toxins or infectious agents such as viruses that are intentionally released to cause disease and death.157 General purpose AI models could facilitate the production of biological weapons, by reducing barriers through access to critical knowledge or increasingly automated assistance and thus enable more malicious actors."
-
53.02.02 · Risk Sub-Category
Dangerous capabilities in AI systems
Acquisition of a goal to harm society
"cases of AI systems being given the outright goal of harming humanity (ChaosGPT);"
-
53.03.06 · Risk Sub-Category
Failures in or misuse of intermediary (non-AGI) AI systems, resulting in catastrophe
"Deployment of “prepotent” AI systems that are non-general but capable of outperforming human collective efforts on various key dimensions;170 → Militarization of AI enabling mass attacks using swarms of lethal autonomous weapons systems;171 → Military use of AI leading to (intentional or unintentional) nuclear escalation, either because machine learning systems are directly integrated in nuclear command and control systems in ways that result in escalation172 or because conventional AI-enabled systems (e.g., autonomous ships) are deployed in ways that result in provocation and escalation;173
-
55.01.01 · Risk Sub-Category
Risks from accelerating scientific progress
Eased development of technologies that make a global catastrophe more likely
—
-
55.02.01 · Risk Sub-Category
AI enables development of weapons of mass destruction
"AI is already enabling the development of weapons which could cause mass destruction —including new weapons that themselves use AI capabilities, such as Lethal Autonomous Weapons [2],10 and the potential use of AI to speed up the development of other potentially dangerous technologies, such as engineered pathogens (as discussed in Section 2)."
-
"Indiscriminate Weapons (CBRNE) Acceptable responses may describe the creation or use of or contain information about indiscriminate weapons. Unacceptable responses will enable, encourage, or endorse the creation of indiscriminate weapons, such as the following: • Chemical weapons (e.g., nerve gas) • Biological weapons (e.g., anthrax) • Radiological weapons (e.g., cobalt bombs) • Nuclear weapons (e.g., fission bombs) • High-yield explosive weapons (e.g., cluster munitions)"
-
"Business operations/infrastructure damage - Damage, disruption, or destruction of a business system and/or its components due to malfunction, cyberattacks, etc."
-
"Violence/armed conflict - Use or misuse of a technology system to incite, facilitate or conduct cyberattacks, security breaches, lethal, biological and chemical weapons development, resulting in violence and armed conflict."
-
"Attackers are beginning to use general- purpose AI for offensive cyber operations, presenting growing but currently limited risks. Current systems have demonstrated capabilities in low- and medium- complexity cybersecurity tasks, with state- sponsored threat actors actively exploring AI to survey target systems. Malicious actors of varying skill levels can leverage these capabilities against people, organisations, and critical infrastructure such as power grids."
-
"Growing evidence shows general- purpose AI advances beneficial to science while also lowering some barriers to chemical and biological weapons development for both novices and experts. New language models can generate step- by- step technical instructions for creating pathogens and toxins that surpass plans written by experts with a PhD and surface information that experts struggle to find online, though their practical utility for novices remains uncertain. Other models demonstrate capabilities in engineering enhanced proteins and analysing which candidate pathogens or toxins are most harmfu
-
"The dangers of AI amplifying the effectiveness/failures of nuclear, chemical, biological, and radiological weapons."
-
61.02.02 · Risk Sub-Category
Sources of systemic risks from general-purpose AI
Ability to enhance and modify pathogens
"AI can be used to enhance pathogens, making them more lethal or resistant to treatments."
-
"Powerful AI technologies may fall into the hands of terrorists."
-
61.02.48 · Risk Sub-Category
Sources of systemic risks from general-purpose AI
Weaponization capabilities
"AI capabilities that could be deliberately weaponized for destructive purposes."
-
62.07.05 · Risk Sub-Category
Direct Harm Domains (system and operational)
Operational harms (autonomous weapons)
—
-
62.08.06 · Risk Sub-Category
Direct Harm Domains (content safety harms)
Dangerous content (e.g., CBRN)
—
-
62.15.05 · Risk Sub-Category
Fine-tuning related (Harmful fine-tuning of open-weights models)
"Models with publicly available weights can be fine-tuned for harmful activities by bad actors, using significantly fewer resources (in terms of time and money) compared to the original training cost [115, 78]."
-
62.30.02 · Risk Sub-Category
AI-based tools attacking critical infrastructure
"Critical infrastructure can also be damaged without AI integration, for instance, when AI-based tools are used indirectly to aid actions such as in coordinated power outages caused by large-scale user manipulation [159]."
-
"AI systems embedded with systemic biases can manipulate large population segments, particularly when these biases align with the beliefs or behaviors of the targeted group. When weaponized at scale, this manipulation can exacerbate social divisions or cause large-scale disruptions, such as city-wide blackouts (e.g., by the manipulation of power consumption into the peak demand period [159])."
-
62.32.00 · Risk Category
-
-
62.32.01 · Risk Sub-Category
Automated discovery and exploitation of software systems
"GPAIs can be used to aid in the automated discovery of software vulnerabilities [33]. This can empower malicious actors, making their cyberattacks more effi- cient and potentially more damaging. This type of automation allows attackers to expand the scale of their operations at a low cost, increasing the impact of their actions. New malware can be developed automatically, or the known vulnerabilities can be exploited to create more sophisticated attacks."
-
"General-purpose AI models may significantly enhance the magnitude and ef- fectiveness of cyberattacks, by amplifying existing capabilities or resources of malicious actors [3]. For example, GPAI models may be employed to: • Automatically scan open-source codebases and compiled binaries for po- tential vulnerabilities • Apply known exploits flexibly and at scale (e.g., identifying vulnerable computers based on subtle cues in response times or output formats) • Assist with different aspects of cyberattacks, including planning, recon- naissance, exploit searching, remote control, malware impleme
-
62.33.00 · Risk Category
-
-
62.33.01 · Risk Sub-Category
Misuse of AI systems to assist in the creation of weapons
"AI systems may be misused to aid in the creation of weapons, such as chemical, biological, radiological, and nuclear (CBRN) weapons, or augment the abilities of existing weapons, such as providing autonomous capabilities to unmanned weapon systems. Current systems do not significantly aid a malicious actor in these tasks, but they do show early signs [117]. This risk can sometimes be mitigated with input and output filtering, but is still susceptible to adversarial techniques (such as jailbreaking or paraphrasing)."
-
"Models used for drug discovery, such as drug-target affinity prediction models, can be used to identify or develop dangerous toxins. This is particularly concern- ing if the training data contains information related to potentially dangerous proteins and viruses."
-
64.04.04 · Risk Sub-Category
Misuse tactics to compromise GenAI systems (Model integrity)
Model diversion
"Model Diversion takes model manipulation one step further, by repurposing (often open-source) generative AI models in a way that diverts them from their intended functionality or from the use cases envisioned by their developers (Lin et al., 2024). An example of this is training the BERT open source model on the DarkWeb to create DarkBert.7"
-
"Generative AI facilitating the damage, disruption or destruction of a third-party system and/or its components via malfunction, cyberattacks, etc"
-
"Frontier AI systems have the potential to accelerate advances in the life sciences, from training new scientists to enabling faster scientific workflows. While these capabilities will have tremendous beneficial applications, there is a risk that they can be used for malicious purposes, such as for the development of biological or chemical weapons."
-
"As the programming abilities of AI systems continue to expand, frontier AI is likely to significantly exacerbate existing cyber risks. Most notably, AI systems can be used by potentially anyone to create faster paced, more effective and larger scale cyber intrusion via tailored phishing methods or replicating malware. Frontier AI’s effect on the overall balance between cyber offence and defence is uncertain, as these tools also have many applications in improving the cybersecurity of systems and defenders are mobilising significant resources to utilise frontier AI for defensive purposes.209 I
-
"Chemical, biological, radiological, and nuclear (CBRN) risks are broad classes of threats that have the potential to cause harm to a large number of people. Explosives are also sometimes included in this category, often referred to as CBRNE...The key characteristic of CBRN risk is that it stems from misuse of capable models with a direct pathway to harm, where a malicious actor is able to carry out consequential attacks more efficiently and effectively with the help of AI."
-
68.02.00 · Risk Category
"Cyber risks, especially in the context of cyber offense, are an existing threat that may be exacerbated by AI. [108] demonstrated that teams of LLM agents can exploit zero-day vulnerabilities when given a description of the vulnerability and toy capture-the-flag problems. While cyber risks are not typically regarded as catastrophic, [3] argues that cyberwarfare is an underappreciated risk that poses a credible threat of catastrophic harm."
-
"EAI systems present distinct physical risks due to their embodiment in the physical world. EAI technologies have already been designed and deployed with lethal intent, such as AI-controlled drones [52, 53]. However, fully autonomous military robots, often integrated with bespoke AI architectures [54, 55], are not yet widely used in combat. While highly or fully autonomous warfare is distinctly possible in the future [56], immediate risks arise from commercially available EAI systems, including AI-controlled quadrupeds and autonomous driving assistants."
-
"Chemical risks involve the exploitation of agents to synthesize chemical weapons, as well as the creation or release of hazardous substances during autonomous chemical experiments. This category also includes the risks arising from the use of advanced materials, such as nanomaterials, which may have unknown or unpredictable chemical properties."
-
"Biological risks encompass the dangerous modification of pathogens and unethical manipulation of genetic material, potentially leading to unforeseen biohazardous outcomes."
-
"AI-enabled cyber offense poses a significant cyber domain security risk by fundamentally transforming the scale, sophistication, and accessibility of cyber-attacks. Unlike traditional cyber threats, AI enables both the automation of existing attack vectors and the creation of entirely new categories of offensive capabilities that can adapt and evolve in real-time. AI can automate and enhance cyber-attacks, including vulnerability discovery and exploitation, password cracking, malicious code generation, sophisticated phishing, network scanning, and social engineering. This could dramatically l
-
"The dual-use nature of AI technology presents a critical risk by significantly lowering technical thresholds for malicious non-state actors to design, synthesize, acquire, and deploy CBRNE (Chemical, Biological, Radiological, Nuclear, and Explosive) weapons. This capability poses unprecedented challenges to national security, international non-proliferation regimes, and global security governance."
-
"The integration of general-purpose AI models into embodied systems creates direct physical threats through malicious exploitation of autonomous decision-making capabilities in real-world environments. The risk lies in embodied models' capacity for autonomous action and real-world interaction, and when these capabilities are maliciously exploited they may trigger a series of serious consequences.18"
-
73.03.04 · Risk Sub-Category
Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs
Warfare and Physical Harm
"The use of AI in warfare is highly alarming and may pose dangers to human safety (Hendrycks et al., 2023). Autonomous drone warfare is being aggressively pursued as a tactic in the current war in Ukraine (Meaker, 2023), and may already have been used on human targets (Hambling, 2023). The use of AI- based facial recognition has been documented in the targeting of Palestinians in Gaza (International, 2023). LLMs have already been productized in limited ways for the purposes of warfare planning (Tarantola, 2023). Furthermore, active research is being carried out to develop multimodal-LLMs that
-
73.03.05 · Risk Sub-Category
Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs
Hazardous Biological and Chemical Technologies
"AI systems such as LLMs, chemical LLMs (Skinnider et al., 2021; Moret et al., 2023), and other LLM- based biological design tools might soon facilitate the production of bioweapons, chemical weapons, and other hazardous technologies. In particular, LLMs might enable actors with less expertise to more easily synthesize dangerous pathogens, while customized chemical and biological design tools might be more concerning in terms of expanding the capabilities of sophisticated actors (e.g. states) (Sandbrink, 2023). Gopal et al. (2023) and Soice et al. (2023) demonstrated that people with little ba
-
02.03.00 · Risk Category
"Improper uses of LLM systems can cause adverse social impacts."
-
"Improper use of LLM systems (i.e., abuse of LLM systems) will cause adverse social impacts, such as academic misconduct."
-
05.08.00 · Risk Category
In contrast to traditional machine learning, the impact of generative AI in the educational sector receives considerable attention in the academic literature. Next to issues stemming from difficulties to distinguish student-generated from AI-generated content, which eventuates in various opportunities to cheat in online or written exams, sources emphasize the potential benefits of generative AI in enhancing learning and teaching methods, particularly in relation to personalized learning approaches. However, some papers suggest that generative AI might lead to reduced effort or laziness among l
-
05.18.00 · Risk Category
Partly overlapping with the discussion on impacts of generative AI on educational institutions, this topic cluster concerns mostly negative effects of LLMs on writing skills and research manuscript composition. The former pertains to the potential homogenization of writing styles, the erosion of semantic capital, or the stifling of individual expression. The latter is focused on the idea of prohibiting generative models for being used to compose scientific papers, figures, or from being a co-author. Sources express concern about risks for academic integrity, as well as the prospect of pollutin
-
"AI has become very good at creating fake content. From text to photos, audio and video. The name "Deep Fake" refers to content that is fake at such a level of complexity that our mind rules out the possibility that it is fake."
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.