MIT AI Risk Repository
Browse AI risks
2,500 risk entries extracted from 74 frameworks, coded by domain, subdomain, causal entity, intent and timing. Filter, then export the current selection with its licence and citation attached.
-
62.31.03#1 · Risk Sub-Category
Impacts of AI (Societal Impacts)
Automatically generating disinformation at scale
"Disinformation (in various modalities: text, audio, images, video, etc.) can be generated with minimal human oversight and effort. Disinformation tools are relatively cheap and their technology is widely available. Such deployments can be particularly widespread in sensitive political contexts."
-
62.31.05 · Risk Sub-Category
Impacts of AI (Societal Impacts)
Generative AI use in political influence campaigns
"GPAI tools can be used in automation and scaling of influence campaigns [178]. Public opinion may be manipulated by targeted misleading or manipulative information. This can lead to rising political polarization and diminishing trust in public institutions."
-
62.31.10 · Risk Sub-Category
Impacts of AI (Societal Impacts)
Misuse for surveillance and population control
"AI tools can be misused by human or institutional actors for monitoring, control- ling, or suppressing individuals [178]. Massive data collection and automated analysis are often conducted, and AI tools can further exacerbate such practices."
-
62.31.12 · Risk Sub-Category
Impacts of AI (Societal Impacts)
Diminishing societal trust due to disinformation or manipulation
"The use of GPAIs may contribute to the proliferation of either deliberate dis- information or unintended misinformation can severely erode trust in public figures and democratic institutions. This diminishing trust can extend to other forms of media, making the public less informed."
-
"Automatic generation of disinformation can be personalized to target specific groups or individuals. Such attacks can be more effective in achieving their goals, and their costs can be significantly reduced when using GPAIs."
-
64.01.03 · Risk Sub-Category
Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness)
Sockpuppeting
"Create synthetic online personas or accounts"
-
64.02.01 · Risk Sub-Category
Misuse tactics that exploit GenAI capabilities (Realistic depictions of non-humans)
Falisification
"Fabricate or falsely represent evidence, incl. reports, IDs, documents"
-
"Generative AI models might be used to intentionally create misleading or false information to deceive or influence a targeted audience."
-
"Erosion of trust in public institutions and weakened checks and balances due to mis/disinformation, influence operations, or real or perceived misuse of generative AI"
-
"Use of a technology system to covertly alter user beliefs and behaviour using nudging, dark patterns and/or other opaque techniques"
-
"In addition to unintentional degradation of the information environment (discussed in the section on Societal Harms above), frontier AI can be misused to deliberately spread false information to create disruption, persuade people on political issues, or cause other forms of harm or damage."
-
"AI systems can be gravely misused to distort public perception and compromise social stability through the generation of synthetic content (e.g., deepfakes, sophisticated fake news) and the strategic manipulation of digital platforms with large user bases to disseminate or precisely target misleading information or ideologies."
-
73.03.03 · Risk Sub-Category
Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs
Surveillance and Censorship
"Content moderation has emerged as one of the key use-cases of LLMs (Weng et al., 2023), indicating the potential of LLMs for surveillance and censorship as well (Edwards, 2023). Surveillance and censorship are one of the primary tools employed by governments with dictatorial tendencies to suppress opposing political and social voices. These censorship measures, however, are often quite crude and can be escaped with little ingenuity...However, LLMs could enable significantly more sophisticated surveillance and censorship operations at scale (Feldstein, 2019). Multimodal-LLMs or LLMs combined w
-
01.05.00 · Risk Category
One or more criminal entities could create AI to intentionally inflict harms, such as for terrorism or combating law enforcement.
-
01.06.00 · Risk Category
AI deployed by states in war, civil war, or law enforcement can easily yield societal-scale harm
-
"Hackers can obtain malicious code in a low-cost and efficient manner to automate cyber attacks with powerful LLM systems."
-
05.10.00 · Risk Category
Closely related to discussions surrounding security and harmful content, the field of cybersecurity investigates how generative AI is misused for fraudulent online activities. A particular focus lies on social engineering attacks, for instance by utilizing generative AI to impersonate humans, creating fake identities, cloning voices, or crafting phishing messages. Another prevalent concern is the use of LLMs for generating malicious code or hacking.
-
06.10.00 · Risk Category
"What is debated as an ethical issue is the use of LAW — AI-driven weapons that fully autonomously take actions that intentionally kill humans."
-
"AI machines could be hacked and misused, e.g. manipulating an airport luggage screening system to smuggle weapons"
-
12.01.00 · Risk Category
"The potential for AI systems to be used maliciously or irresponsibly, including for creating deepfakes, automated cyber attacks, or invasive surveillance systems. Specifically denotes intentional use of AI for harm."
-
16.04.02 · Risk Sub-Category
Assisting code generation for cyber security threats
Anticipated risk: "Creators of the assistive coding tool Co-Pilot based on GPT-3 suggest that such tools may lower the cost of developing polymorphic malware which is able to change its features in order to evade detection [37]."
-
17.04.03 · Risk Sub-Category
Assisting code generation for cyber attacks, weapons, or malicious use
—
-
"Facilitating the conduct of cyber attacks, weapon development, and security breaches"
-
19.02.04 · Risk Sub-Category
Informational and Communicational AI Risks
Endangerment of data protection through AI cyberattacks
—
-
19.04.03 · Risk Sub-Category
Hazardous misuse of AI systems bears danger to the society in public spaces (e.g., hacker attacks on autonomous weapons)
—
-
"AIs with knowledge of bioengineering could facilitate the creation of novel bioweapons and lower barriers to obtaining such agents."
-
"people could build AIs that pursue dangerous goals’"
-
23.05.00 · Risk Category
"This category addresses responses that enable, encourage, or endorse the creation of indiscriminate weapons (colloquially known as "weapons of mass destruction")."
-
"Offensive cyber operations are malicious attacks on computer systems and networks aimed at gaining unauthorized access to, manipulating, denying, disrupting, degrading, or destroying the target system. These attacks can target the system’s network, hardware, or software. Advanced AI assistants can be a double-edged sword in cybersecurity, benefiting both the defenders and the attackers. They can be used by cyber defenders to protect systems from malicious intruders by leveraging information trained on massive amounts of cyber-threat intelligence data, including vulnerabilities, attack pattern
-
"A common element in offensive cyber operations involves the identification and exploitation of system vulnerabilities to gain unauthorized access or control. Until recently, these activities required specialist programming knowledge. In the case of ‘zero-day’ vulnerabilities (flaws or weaknesses in software or an operating system that the creator or vendor is not aware of), considerable resources and technical creativity are typically required to manually discover such vulnerabilities, so their use is limited to well-resourced nation states or technically sophisticated advanced persistent thr
-
"Malicious code is a term for code—whether it be part of a script or embedded in a software system—designed to cause damage, security breaches, or other threats to application security. Advanced AI assistants with the ability to produce source code can potentially lower the barrier to entry for threat actors with limited programming abilities or technical skills to produce malicious code. Recently, a series of proof-of-concept attacks have shown how a benign-seeming executable file can be crafted such that, at every runtime, it makes application programming interface (API) calls to an AI assis
-
25.01.00 · Risk Category
"The model can discover vulnerabilities in systems (hardware, software, data). It can write code for exploiting those vulnerabilities. It can make effective decisions once it has gained access to a system or network, and skilfully evade threat detection and response (both human and system) whilst focusing on a specific objective. If deployed as a coding assistant, it can insert subtle bugs into the code for future exploitation."
-
LAWS are a distinctive category of weapon systems that employ sensor arrays and computer algorithms to detect and attack a target without direct human intervention in the system’s operation
-
ability of LLMs to write reasonably good-quality code with extremely low cost and incredible speed, such great assistance can equally facilitate malicious attacks. In particular, malicious hackers can leverage LLMs to assist with performing cyberattacks leveraged by the low cost of LLMs and help with automating the attacks.
-
"Though chatbots cannot (yet) develop their own novel malware from scratch, hackers could soon potentially use the coding abilities of large language models like ChatGPT to create malware that can then be minutely adjusted for maximum reach and effect, essentially allowing more novice hackers to become a serious security risk"
-
35.01.00 · Risk Category
weaponizing AI may be an onramp to more dangerous outcomes. In recent years, deep RL algorithms can outperform humans at aerial combat [18], AlphaFold has discovered new chemical weapons [66], researchers have been developing AI systems for automated cyberattacks [11, 14], military leaders have discussed having AI systems have decisive control over nuclear silos
-
"This group comprises 11% of the articles and centers on risks stemming from AI systems designed with malicious intent or that can end up in a threat to human life. It can be divided into two key themes: threats to law and democracy, and transhumanism."
-
41.05.00 · Risk Category
"AI could enable more serious incidents to occur by lowering the cost of devising cyber-attacks and enabling more targeted incidents. The same programming error or hacker attack could be replicated on numerous machines. Or one machine could repeat the same erroneous activity several times, leading to an unforeseen accumulation of losses."
-
41.05.01 · Risk Sub-Category
Catastrophic risk due to autonomous weapons programmed with dangerous targets
"AI could enable autonomous vehicles, such as drones, to be utilized as weapons. Such threats are often underestimated."
-
"Implications of the weaponization of AI for defence (the embeddedness of AI-based capabilities across the land, air, naval and space domains may affect combined arms operations)."
-
"These evaluations focus on whether a LLM possesses certain capabilities in the cyber-domain. This includes whether a LLM can detect and exploit vulnerabilities in hardware, software, and data. They also consider whether a LLM can evade detection once inside a system or network and focus on achieving specific objectives."
-
"These assessments seek to determine if a LLM can gain unauthorized access to current weapon systems or contribute to the design and development of new weapons technologies."
-
"LLM has science capabilities that can be used to cause harm (e.g., providing step-by-step instructions for conducting malicious experiments)"
-
45.02.04 · Risk Sub-Category
Safety risks in AI Applications
Cyberspace risks (Risks of abuse for cyberattacks)
"AI can be used in launching automatic cyberattacks or increasing attack efficiency, including exploring and making use of vulnerabilities, cracking passwords, generating malicious codes, sending phishing emails, network scanning, and social engineering attacks. All these lower the threshold for cyberattacks and increase the difficulty of security protection."
-
45.02.08 · Risk Sub-Category
Safety risks in AI Applications
Real-world risks (Risks of misuse of dual-use items and technologies)
"Due to improper use or abuse, AI can pose serious risks to national security, economic security, and public health security, such as greatly reducing the capability requirements for non-experts to design, synthesize, acquire, and use nuclear, biological, and chemical weapons and missiles; and designing cyber weapons that launch network attacks on a wide range of potential targets through methods like automatic vulnerability discovery and exploitation."
-
"Generative AI can help amplify the frequency and destructiveness of cyberattacks.311 It has the capacity “to increase the accessibility, success rate, scale, speed, stealth, and potency of cyberattacks. It enables the identification of critical vulnerabilities within targeted systems, facilitates the increase of the scale of cyberattacks, and accelerates the process by discovering innovative methods of system infiltration. Cyberattacks can inflict significant damage and may impact critical infrastructure, including electrical grids, financial systems, and weapons management systems."
-
"Many fear that generative AI could make the creation of biological weapons easier by providing access to critical knowledge and automated assistance to a wider range of actors to engage in malicious activities."
-
47.02.06 · Risk Sub-Category
Malicious use and abuse (military applications)
"The advancement of AI for military purposes is rapidly ushering in a new phase of growth in military technology. Lethal Autonomous Weapons Systems (LAWS) possess the capability to detect, engage, and eliminate human targets independently, without human input.341 In 2020, a sophisticated AI agent surpassed experienced F-16 pilots in multiple simulated aerial combat scenarios, notably achieving a 5-0 victory against a human pilot through “aggressive and precise maneuvers” that the human could not surpass.342 Additionally, fully autonomous drones are already operational."
-
48.01.00 · Risk Category
"Eased access to or synthesis of materially nefarious information or design capabilities related to chemical, biological, radiological, or nuclear (CBRN) weapons or other dangerous materials or agents."
-
48.09.00 · Risk Category
"Lowered barriers for offensive cyber capabilities, including via automated discovery and exploitation of vulnerabilities to ease hacking, malware, phishing, offensive cyber operations, or other cyberattacks; increased attack surface for targeted cyberattacks, which may compromise a system’s availability or the confidentiality or integrity of training data, code, or model weights."
Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.