Model Attacks

Classification

Domain

2. Privacy & Security

Subdomain

2.2 AI system security vulnerabilities and attacks

Causal entity

Human

Intent

Intentional

Timing

Other

Subdomain definition: Vulnerabilities in AI systems, software development toolchains, and hardware that can be exploited, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.

Real-world incidents in this subdomain

• 30 May 2026COEMPT Quality Assurance Engineers Allegedly Violated Indian CBSE Student Data Privacy Rights by Processing It with Google Gemini
• 13 May 2026Hidden Prompt Injection in Brazilian Labor-Court Petition Reportedly Tried to Manipulate Galileu
• 18 Mar 2026Meta Internal AI Agent Reportedly Gave Advice That Allegedly Exposed Sensitive Data to Unauthorized Employees
• 28 Feb 2026CodeWall's Autonomous Agent Reportedly Obtained Unauthorized Access to McKinsey's Lilli AI Platform Database
• 23 Feb 2026Anthropic Said DeepSeek, Moonshot, and MiniMax Used Fraudulent Accounts and Proxies to Illicitly Distill Claude Capabilities at Scale
• 8 Feb 2026DJI Romo Cloud Authorization Bug Reportedly Exposed Camera, Microphone, and Home-Mapping Data From Nearly 7,000 Robot Vacuums

How other frameworks describe this risk

• Privacy loss

  A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)

• Exploiting Limited Generalization of Safety Finetuning

  Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

• Jailbreaks and Prompt Injections Threaten Security of LLMs

  Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

• “Model Psychology” Attacks

  Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

• Attacking LLMs via Additional Modalities a

  Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

• Vulnerability to Poisoning and Backdoors

  Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

• Security

  AGI Safety Literature Review (Everitt2018 )

• Adversarial AI (General)

  The Ethics of Advanced AI Assistants (Gabriel2024)

Other entries from Cui2024

• Harmful Content Risk Category · 1.2
• Bias Risk Sub-Category · 1.1
• Toxicity Risk Sub-Category · 1.2
• Privacy Leakage Risk Sub-Category · 2.1
• Untruthful Content Risk Category · 3.1
• Factuality Errors Risk Sub-Category · 3.1
• Faithfulness Errors Risk Sub-Category · 3.1
• Unhelpful Uses Risk Category · 4.3
• Academic Misconduct Risk Sub-Category · 4.3
• Copyright Violation Risk Sub-Category · 6.3
• Cyber Attacks Risk Sub-Category · 4.2
• Software Vulnerabilities Risk Sub-Category · 2.2