MIT AI Risk Repository · domain 4: Malicious actors
4.2 Cyberattacks, weapon development or use, and mass harm
Using AI systems to develop cyber weapons (e.g., coding cheaper, more effective malware), develop new or enhance existing weapons (e.g., Lethal Autonomous Weapons or CBRNE), or use weapons to cause mass harm.
- 82
- 12
- 15
- 14
| Label | Value |
|---|---|
| Human | 61 |
| AI | 11 |
| Other | 8 |
| Not coded | 2 |
| Label | Value |
|---|---|
| Intentional | 68 |
| Other | 10 |
| Unintentional | 2 |
| Not coded | 2 |
| Label | Value |
|---|---|
| Post-deployment | 71 |
| Other | 7 |
| Pre-deployment | 2 |
| Not coded | 2 |
| Label | Value |
|---|---|
| 2017 | 1 |
| 2020 | 1 |
| 2022 | 1 |
| 2023 | 1 |
| 2024 | 2 |
| 2025 | 9 |
| Label | Value |
|---|---|
| Risk Category | 16 |
| Risk Sub-Category | 66 |
Risk entries
Browse and export all- Operational harms (autonomous weapons)
- Dangerous content (e.g., CBRN)
- Fine-tuning related (Harmful fine-tuning of open-weights models)
"Models with publicly available weights can be fine-tuned for harmful activities by bad actors, using significantly fewer resources (in terms of time and money) compared to the original training cost...
- AI-based tools attacking critical infrastructure
"Critical infrastructure can also be damaged without AI integration, for instance, when AI-based tools are used indirectly to aid actions such as in coordinated power outages caused by large-scale use...
- Systemic large-scale manipulation
"AI systems embedded with systemic biases can manipulate large population segments, particularly when these biases align with the beliefs or behaviors of the targeted group. When weaponized at scale,...
- Impacts of AI (Cyberattacks)
-
- Automated discovery and exploitation of software systems
"GPAIs can be used to aid in the automated discovery of software vulnerabilities [33]. This can empower malicious actors, making their cyberattacks more effi- cient and potentially more damaging. This...
- Amplification of cyberattacks
"General-purpose AI models may significantly enhance the magnitude and ef- fectiveness of cyberattacks, by amplifying existing capabilities or resources of malicious actors [3]. For example, GPAI mode...
- Impacts of AI (Weapons)
-
- Misuse of AI systems to assist in the creation of weapons
"AI systems may be misused to aid in the creation of weapons, such as chemical, biological, radiological, and nuclear (CBRN) weapons, or augment the abilities of existing weapons, such as providing au...
- Misuse of drug-discovery models
"Models used for drug discovery, such as drug-target affinity prediction models, can be used to identify or develop dangerous toxins. This is particularly concern- ing if the training data contains in...
- Malicious use and abuse (cyberattacks)
"Generative AI can help amplify the frequency and destructiveness of cyberattacks.311 It has the capacity “to increase the accessibility, success rate, scale, speed, stealth, and potency of cyberattac...
- Malicious use and abuse (biosecurity threats)
"Many fear that generative AI could make the creation of biological weapons easier by providing access to critical knowledge and automated assistance to a wider range of actors to engage in malicious...
- Malicious use and abuse (military applications)
"The advancement of AI for military purposes is rapidly ushering in a new phase of growth in military technology. Lethal Autonomous Weapons Systems (LAWS) possess the capability to detect, engage, and...
- Lethal Autonomous Weapons Systems (LAWS)
LAWS are a distinctive category of weapon systems that employ sensor arrays and computer algorithms to detect and attack a target without direct human intervention in the system’s operation
- Cybercrime
Closely related to discussions surrounding security and harmful content, the field of cybersecurity investigates how generative AI is misused for fraudulent online activities. A particular focus lies...
- Weaponization
weaponizing AI may be an onramp to more dangerous outcomes. In recent years, deep RL algorithms can outperform humans at aerial combat [18], AlphaFold has discovered new chemical weapons [66], researc...
- Bioterrorism
"AIs with knowledge of bioengineering could facilitate the creation of novel bioweapons and lower barriers to obtaining such agents."
- Unleashing AI Agents
"people could build AIs that pursue dangerous goals’"
- Lethal Autonomous Weapons (LAW)
"What is debated as an ethical issue is the use of LAW — AI-driven weapons that fully autonomously take actions that intentionally kill humans."
- Offensive cyber capabilities
"These evaluations focus on whether a LLM possesses certain capabilities in the cyber-domain. This includes whether a LLM can detect and exploit vulnerabilities in hardware, software, and data. They a...
- Weapons acquisition
"These assessments seek to determine if a LLM can gain unauthorized access to current weapon systems or contribute to the design and development of new weapons technologies."
- Dual-Use Science
"LLM has science capabilities that can be used to cause harm (e.g., providing step-by-step instructions for conducting malicious experiments)"
- Cyberattacks
"Generative AI facilitating the damage, disruption or destruction of a third-party system and/or its components via malfunction, cyberattacks, etc"
- Cyberattack
ability of LLMs to write reasonably good-quality code with extremely low cost and incredible speed, such great assistance can equally facilitate malicious attacks. In particular, malicious hackers can...