MIT AI Risk Repository
Browse AI risks
977 risk entries extracted from 74 frameworks, coded by domain, subdomain, causal entity, intent and timing. Filter, then export the current selection with its licence and citation attached.
-
17.04.03 · Risk Sub-Category
Assisting code generation for cyber attacks, weapons, or malicious use
—
-
"Facilitating the conduct of cyber attacks, weapon development, and security breaches"
-
19.02.04 · Risk Sub-Category
Informational and Communicational AI Risks
Endangerment of data protection through AI cyberattacks
—
-
19.04.03 · Risk Sub-Category
Hazardous misuse of AI systems bears danger to the society in public spaces (e.g., hacker attacks on autonomous weapons)
—
-
"AIs with knowledge of bioengineering could facilitate the creation of novel bioweapons and lower barriers to obtaining such agents."
-
23.05.00 · Risk Category
"This category addresses responses that enable, encourage, or endorse the creation of indiscriminate weapons (colloquially known as "weapons of mass destruction")."
-
"Offensive cyber operations are malicious attacks on computer systems and networks aimed at gaining unauthorized access to, manipulating, denying, disrupting, degrading, or destroying the target system. These attacks can target the system’s network, hardware, or software. Advanced AI assistants can be a double-edged sword in cybersecurity, benefiting both the defenders and the attackers. They can be used by cyber defenders to protect systems from malicious intruders by leveraging information trained on massive amounts of cyber-threat intelligence data, including vulnerabilities, attack pattern
-
"A common element in offensive cyber operations involves the identification and exploitation of system vulnerabilities to gain unauthorized access or control. Until recently, these activities required specialist programming knowledge. In the case of ‘zero-day’ vulnerabilities (flaws or weaknesses in software or an operating system that the creator or vendor is not aware of), considerable resources and technical creativity are typically required to manually discover such vulnerabilities, so their use is limited to well-resourced nation states or technically sophisticated advanced persistent thr
-
"Malicious code is a term for code—whether it be part of a script or embedded in a software system—designed to cause damage, security breaches, or other threats to application security. Advanced AI assistants with the ability to produce source code can potentially lower the barrier to entry for threat actors with limited programming abilities or technical skills to produce malicious code. Recently, a series of proof-of-concept attacks have shown how a benign-seeming executable file can be crafted such that, at every runtime, it makes application programming interface (API) calls to an AI assis
-
25.01.00 · Risk Category
"The model can discover vulnerabilities in systems (hardware, software, data). It can write code for exploiting those vulnerabilities. It can make effective decisions once it has gained access to a system or network, and skilfully evade threat detection and response (both human and system) whilst focusing on a specific objective. If deployed as a coding assistant, it can insert subtle bugs into the code for future exploitation."
-
LAWS are a distinctive category of weapon systems that employ sensor arrays and computer algorithms to detect and attack a target without direct human intervention in the system’s operation
-
ability of LLMs to write reasonably good-quality code with extremely low cost and incredible speed, such great assistance can equally facilitate malicious attacks. In particular, malicious hackers can leverage LLMs to assist with performing cyberattacks leveraged by the low cost of LLMs and help with automating the attacks.
-
"Though chatbots cannot (yet) develop their own novel malware from scratch, hackers could soon potentially use the coding abilities of large language models like ChatGPT to create malware that can then be minutely adjusted for maximum reach and effect, essentially allowing more novice hackers to become a serious security risk"
-
35.01.00 · Risk Category
weaponizing AI may be an onramp to more dangerous outcomes. In recent years, deep RL algorithms can outperform humans at aerial combat [18], AlphaFold has discovered new chemical weapons [66], researchers have been developing AI systems for automated cyberattacks [11, 14], military leaders have discussed having AI systems have decisive control over nuclear silos
-
41.05.00 · Risk Category
"AI could enable more serious incidents to occur by lowering the cost of devising cyber-attacks and enabling more targeted incidents. The same programming error or hacker attack could be replicated on numerous machines. Or one machine could repeat the same erroneous activity several times, leading to an unforeseen accumulation of losses."
-
41.05.01 · Risk Sub-Category
Catastrophic risk due to autonomous weapons programmed with dangerous targets
"AI could enable autonomous vehicles, such as drones, to be utilized as weapons. Such threats are often underestimated."
-
"Implications of the weaponization of AI for defence (the embeddedness of AI-based capabilities across the land, air, naval and space domains may affect combined arms operations)."
-
45.02.04 · Risk Sub-Category
Safety risks in AI Applications
Cyberspace risks (Risks of abuse for cyberattacks)
"AI can be used in launching automatic cyberattacks or increasing attack efficiency, including exploring and making use of vulnerabilities, cracking passwords, generating malicious codes, sending phishing emails, network scanning, and social engineering attacks. All these lower the threshold for cyberattacks and increase the difficulty of security protection."
-
45.02.08 · Risk Sub-Category
Safety risks in AI Applications
Real-world risks (Risks of misuse of dual-use items and technologies)
"Due to improper use or abuse, AI can pose serious risks to national security, economic security, and public health security, such as greatly reducing the capability requirements for non-experts to design, synthesize, acquire, and use nuclear, biological, and chemical weapons and missiles; and designing cyber weapons that launch network attacks on a wide range of potential targets through methods like automatic vulnerability discovery and exploitation."
-
"Generative AI can help amplify the frequency and destructiveness of cyberattacks.311 It has the capacity “to increase the accessibility, success rate, scale, speed, stealth, and potency of cyberattacks. It enables the identification of critical vulnerabilities within targeted systems, facilitates the increase of the scale of cyberattacks, and accelerates the process by discovering innovative methods of system infiltration. Cyberattacks can inflict significant damage and may impact critical infrastructure, including electrical grids, financial systems, and weapons management systems."
-
"Many fear that generative AI could make the creation of biological weapons easier by providing access to critical knowledge and automated assistance to a wider range of actors to engage in malicious activities."
-
47.02.06 · Risk Sub-Category
Malicious use and abuse (military applications)
"The advancement of AI for military purposes is rapidly ushering in a new phase of growth in military technology. Lethal Autonomous Weapons Systems (LAWS) possess the capability to detect, engage, and eliminate human targets independently, without human input.341 In 2020, a sophisticated AI agent surpassed experienced F-16 pilots in multiple simulated aerial combat scenarios, notably achieving a 5-0 victory against a human pilot through “aggressive and precise maneuvers” that the human could not surpass.342 Additionally, fully autonomous drones are already operational."
-
48.01.00 · Risk Category
"Eased access to or synthesis of materially nefarious information or design capabilities related to chemical, biological, radiological, or nuclear (CBRN) weapons or other dangerous materials or agents."
-
48.09.00 · Risk Category
"Lowered barriers for offensive cyber capabilities, including via automated discovery and exploitation of vulnerabilities to ease hacking, malware, phishing, offensive cyber operations, or other cyberattacks; increased attack surface for targeted cyberattacks, which may compromise a system’s availability or the confidentiality or integrity of training data, code, or model weights."
-
"General- purpose AI systems could uplift the cyber expertise of individuals, making it easier for malicious users to conduct effective cyber- attacks, as well as providing a tool that can be used in cyber defence. General- purpose AI systems can be used to automate and scale some types of cyber operations, such as social engineering attacks."
-
"General- purpose AI systems could accelerate advances in a range of scientific endeavours, from training new scientists to enabling faster research workflows. While these capabilities could have numerous beneficial applications, some experts have expressed concern that they could be used for malicious purposes, especially if further capabilities are developed soon before appropriate countermeasures are put in place. There are two avenues by which general- purpose AI systems could, speculatively, facilitate malicious use in the life sciences: firstly by providing increased access to informatio
-
—
-
50.02.05 · Risk Sub-Category
Violence and extremism (Weapon Usage and Development)
—
-
—
-
50.04.08 · Risk Sub-Category
Legal and Rights-Related Risks
Criminal Activities (Other Unlawful/Criminal Activities)
—
-
"The potential misuse of general purpose AI models also extends to biosecurity threats. Biological weapons are generally understood as biological toxins or infectious agents such as viruses that are intentionally released to cause disease and death.157 General purpose AI models could facilitate the production of biological weapons, by reducing barriers through access to critical knowledge or increasingly automated assistance and thus enable more malicious actors."
-
53.03.06 · Risk Sub-Category
Failures in or misuse of intermediary (non-AGI) AI systems, resulting in catastrophe
"Deployment of “prepotent” AI systems that are non-general but capable of outperforming human collective efforts on various key dimensions;170 → Militarization of AI enabling mass attacks using swarms of lethal autonomous weapons systems;171 → Military use of AI leading to (intentional or unintentional) nuclear escalation, either because machine learning systems are directly integrated in nuclear command and control systems in ways that result in escalation172 or because conventional AI-enabled systems (e.g., autonomous ships) are deployed in ways that result in provocation and escalation;173
-
55.01.01 · Risk Sub-Category
Risks from accelerating scientific progress
Eased development of technologies that make a global catastrophe more likely
—
-
55.02.01 · Risk Sub-Category
AI enables development of weapons of mass destruction
"AI is already enabling the development of weapons which could cause mass destruction —including new weapons that themselves use AI capabilities, such as Lethal Autonomous Weapons [2],10 and the potential use of AI to speed up the development of other potentially dangerous technologies, such as engineered pathogens (as discussed in Section 2)."
-
"Indiscriminate Weapons (CBRNE) Acceptable responses may describe the creation or use of or contain information about indiscriminate weapons. Unacceptable responses will enable, encourage, or endorse the creation of indiscriminate weapons, such as the following: • Chemical weapons (e.g., nerve gas) • Biological weapons (e.g., anthrax) • Radiological weapons (e.g., cobalt bombs) • Nuclear weapons (e.g., fission bombs) • High-yield explosive weapons (e.g., cluster munitions)"
-
"Business operations/infrastructure damage - Damage, disruption, or destruction of a business system and/or its components due to malfunction, cyberattacks, etc."
-
"Violence/armed conflict - Use or misuse of a technology system to incite, facilitate or conduct cyberattacks, security breaches, lethal, biological and chemical weapons development, resulting in violence and armed conflict."
-
"Attackers are beginning to use general- purpose AI for offensive cyber operations, presenting growing but currently limited risks. Current systems have demonstrated capabilities in low- and medium- complexity cybersecurity tasks, with state- sponsored threat actors actively exploring AI to survey target systems. Malicious actors of varying skill levels can leverage these capabilities against people, organisations, and critical infrastructure such as power grids."
-
"Growing evidence shows general- purpose AI advances beneficial to science while also lowering some barriers to chemical and biological weapons development for both novices and experts. New language models can generate step- by- step technical instructions for creating pathogens and toxins that surpass plans written by experts with a PhD and surface information that experts struggle to find online, though their practical utility for novices remains uncertain. Other models demonstrate capabilities in engineering enhanced proteins and analysing which candidate pathogens or toxins are most harmfu
-
61.02.02 · Risk Sub-Category
Sources of systemic risks from general-purpose AI
Ability to enhance and modify pathogens
"AI can be used to enhance pathogens, making them more lethal or resistant to treatments."
-
61.02.48 · Risk Sub-Category
Sources of systemic risks from general-purpose AI
Weaponization capabilities
"AI capabilities that could be deliberately weaponized for destructive purposes."
-
62.15.05 · Risk Sub-Category
Fine-tuning related (Harmful fine-tuning of open-weights models)
"Models with publicly available weights can be fine-tuned for harmful activities by bad actors, using significantly fewer resources (in terms of time and money) compared to the original training cost [115, 78]."
-
62.30.02 · Risk Sub-Category
AI-based tools attacking critical infrastructure
"Critical infrastructure can also be damaged without AI integration, for instance, when AI-based tools are used indirectly to aid actions such as in coordinated power outages caused by large-scale user manipulation [159]."
-
"AI systems embedded with systemic biases can manipulate large population segments, particularly when these biases align with the beliefs or behaviors of the targeted group. When weaponized at scale, this manipulation can exacerbate social divisions or cause large-scale disruptions, such as city-wide blackouts (e.g., by the manipulation of power consumption into the peak demand period [159])."
-
62.32.00 · Risk Category
-
-
62.32.01 · Risk Sub-Category
Automated discovery and exploitation of software systems
"GPAIs can be used to aid in the automated discovery of software vulnerabilities [33]. This can empower malicious actors, making their cyberattacks more effi- cient and potentially more damaging. This type of automation allows attackers to expand the scale of their operations at a low cost, increasing the impact of their actions. New malware can be developed automatically, or the known vulnerabilities can be exploited to create more sophisticated attacks."
-
"General-purpose AI models may significantly enhance the magnitude and ef- fectiveness of cyberattacks, by amplifying existing capabilities or resources of malicious actors [3]. For example, GPAI models may be employed to: • Automatically scan open-source codebases and compiled binaries for po- tential vulnerabilities • Apply known exploits flexibly and at scale (e.g., identifying vulnerable computers based on subtle cues in response times or output formats) • Assist with different aspects of cyberattacks, including planning, recon- naissance, exploit searching, remote control, malware impleme
-
62.33.00 · Risk Category
-
-
62.33.01 · Risk Sub-Category
Misuse of AI systems to assist in the creation of weapons
"AI systems may be misused to aid in the creation of weapons, such as chemical, biological, radiological, and nuclear (CBRN) weapons, or augment the abilities of existing weapons, such as providing autonomous capabilities to unmanned weapon systems. Current systems do not significantly aid a malicious actor in these tasks, but they do show early signs [117]. This risk can sometimes be mitigated with input and output filtering, but is still susceptible to adversarial techniques (such as jailbreaking or paraphrasing)."
-
"Models used for drug discovery, such as drug-target affinity prediction models, can be used to identify or develop dangerous toxins. This is particularly concern- ing if the training data contains information related to potentially dangerous proteins and viruses."
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.