Exploiting Limited Generalization of Safety Finetuning

Classification

Domain

2. Privacy & Security

Subdomain

2.2 AI system security vulnerabilities and attacks

Causal entity

Other

Intent

Unintentional

Timing

Other

Subdomain definition: Vulnerabilities in AI systems, software development toolchains, and hardware that can be exploited, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.

Real-world incidents in this subdomain

• 30 May 2026COEMPT Quality Assurance Engineers Allegedly Violated Indian CBSE Student Data Privacy Rights by Processing It with Google Gemini
• 13 May 2026Hidden Prompt Injection in Brazilian Labor-Court Petition Reportedly Tried to Manipulate Galileu
• 18 Mar 2026Meta Internal AI Agent Reportedly Gave Advice That Allegedly Exposed Sensitive Data to Unauthorized Employees
• 28 Feb 2026CodeWall's Autonomous Agent Reportedly Obtained Unauthorized Access to McKinsey's Lilli AI Platform Database
• 23 Feb 2026Anthropic Said DeepSeek, Moonshot, and MiniMax Used Fraudulent Accounts and Proxies to Illicitly Distill Claude Capabilities at Scale
• 8 Feb 2026DJI Romo Cloud Authorization Bug Reportedly Exposed Camera, Microphone, and Home-Mapping Data From Nearly 7,000 Robot Vacuums

How other frameworks describe this risk

• Privacy loss

  A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)

• Hardware Vulnerabilities

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

• Network Devices

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

• Issues on External Tools

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

• Software Vulnerabilities

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

• Software Security Issues

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

• Programming Language

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

• Model Attacks

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

Other entries from Anwar2024

• Agentic LLMs Pose Novel Risks Risk Category · 7.2
• Natural Language Underspecifies Goals Risk Sub-Category · 7.1
• Goal-Directedness Incentivizes Undesirable Behaviors Risk Sub-Category · 7.2
• Safety Risks from Affordances Provided to LLM-agents Risk Sub-Category · 7.2
• Multi-Agent Safety Is Not Assured by Single-Agent Safety Risk Category · 7.6
• Foundationality May Cause Correlated Failures Risk Sub-Category · 7.6
• Groups of LLM-Agents May Show Emergent Functionality Risk Sub-Category · 7.6
• Collusion between LLM-Agents Risk Sub-Category · 7.6
• Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs Risk Category · 4.0
• Misinformation and Manipulation Risk Sub-Category · 4.3
• Cybersecurity Risk Sub-Category · 4.3
• Cybersecurity Additional evidence