Misuse tactics to compromise GenAI systems (Model integrity)

Classification

Domain

2. Privacy & Security

Subdomain

2.1 Compromise of privacy by obtaining, leaking or correctly inferring sensitive information

Causal entity

Human

Intent

Intentional

Timing

Other

Subdomain definition: AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or loss of confidential intellectual property.

Real-world incidents in this subdomain

• 27 Feb 2026Meta AI Smart Glasses Reportedly Routed Intimate Imagery to Reviewers at Kenyan Contractor Sama Before Meta Ended Contract
• 19 Feb 2026Grok Reportedly Disclosed Adult Performer Siri Dahl's Legal Name and Birthdate, Allegedly Contributing to Doxxing and Harassment
• 23 Jan 2026NPR Host David Greene Alleged Google's NotebookLM Replicated His Voice Without Consent, Prompting Lawsuit
• 10 Jan 2026Border Patrol Agent Allegedly Claimed Facial Recognition Identified Minneapolis ICE Observer and Global Entry Was Reportedly Revoked Three Days Later
• 4 Dec 2025Perplexity AI Reportedly Accused in Federal Lawsuit of Purported Copyright Infringement and False Attribution of Chicago Tribune Content
• 19 Nov 2025Secret Desires AI Platform Reportedly Exposed Nearly Two Million Sensitive Images in Cloud Storage Leak

How other frameworks describe this risk

• Risks to privacy

  International Scientific Report on the Safety of Advanced AI (Bengio2024)

• Risks to privacy

  International AI Safety Report 2025 (Bengio2025)

• Privacy Leakage

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

• Private Training Data

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

• Memorization in LLMs

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

• Association in LLMs

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

• Privacy Leakage

  Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

• Privacy and regulation violations

  Navigating the Landscape of AI Ethics and Responsibility (Cunha2023)

Other entries from Marchal2024

• Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness) Risk Category · 4.3
• Impersonation Risk Sub-Category · 4.3
• Impersonation Additional evidence
• Appropriated Likeness Risk Sub-Category · 4.3
• Appropriated Likeness Additional evidence
• Sockpuppeting Risk Sub-Category · 4.1
• Sockpuppeting Additional evidence
• Non-consensual intimate imagery (NCII) Risk Sub-Category · 4.3
• Non-consensual intimate imagery (NCII) Additional evidence
• Child sexual abuse material (CSAM) Risk Sub-Category · 4.3
• Child sexual abuse material (CSAM) Additional evidence
• Misuse tactics that exploit GenAI capabilities (Realistic depictions of non-humans) Risk Category · 4.3