MIT AI Risk Repository · Risk Sub-Category · 24.03.06

Adversarial AI: Circumvention of Technical Security Measures

Category: Malicious Uses

Open the MIT Risk Navigator All entries from Gabriel2024

Description

"The technical measures to mitigate misuse risks of advanced AI assistants themselves represent a new target for attack. An emerging form of misuse of general-purpose advanced AI assistants exploits vulnerabilities in a model that results in unwanted behavior or in the ability of an attacker to gain unauthorized access to the model and/or its capabilities. While these attacks currently require some level of prompt engineering knowledge and are often patched by developers, bad actors may develop their own adversarial AI agents that are explicitly trained to discover new vulnerabilities that all

From The Ethics of Advanced AI Assistants (Gabriel2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 2. Privacy & Security
Subdomain: 2.2 AI system security vulnerabilities and attacks
Causal entity: Other
Intent: Intentional
Timing: Post-deployment

Subdomain definition: Vulnerabilities in AI systems, software development toolchains, and hardware that can be exploited, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

Privacy loss
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Exploiting Limited Generalization of Safety Finetuning
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Jailbreaks and Prompt Injections Threaten Security of LLMs
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
“Model Psychology” Attacks
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Attacking LLMs via Additional Modalities a
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Vulnerability to Poisoning and Backdoors
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Model Attacks
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)
Hardware Vulnerabilities
Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

Other entries from Gabriel2024

Capability failures Risk Category · 7.3
Lack of capability for task Risk Sub-Category · 7.3
Difficult to develop metrics for evaluating benefits or harms caused by AI assistants Risk Sub-Category · 6.5
Safe exploration problem with widely deployed AI assistants Risk Sub-Category · 7.3
Goal-related failures Risk Category · 7.1
Misaligned consequentialist reasoning Risk Sub-Category · 7.3
Specification gaming Risk Sub-Category · 7.1
Goal misgeneralisation Risk Sub-Category · 7.1
Deceptive alignment Risk Sub-Category · 7.1
Malicious Uses Risk Category · 4.0
Offensive Cyber Operations (General) Risk Sub-Category · 4.2
AI-Powered Spear-Phishing at Scale Risk Sub-Category · 4.1