AIPolicyTracker
Menu

MIT AI Risk Repository · Risk Sub-Category · 24.08.01

Private information leakage

Category: Privacy

Open the MIT Risk Navigator All entries from Gabriel2024

Description

"First, because LLMs display immense modelling power, there is a risk that the model weights encode private information present in the training corpus. In particular, it is possible for LLMs to ‘memorise’ personally identifiable information (PII) such as names, addresses and telephone numbers, and subsequently leak such information through generated text outputs (Carlini et al., 2021). Private information leakage could occur accidentally or as the result of an attack in which a person employs adversarial prompting to extract private information from the model. In the context of pre-training da

From The Ethics of Advanced AI Assistants (Gabriel2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Causal entity
Other
Intent
Other
Timing
Other

Subdomain definition: AI systems that memorize and leak sensitive personal data or infer private information about individuals without their consent. Unexpected or unauthorized sharing of data and information can compromise user expectation of privacy, assist identity theft, or loss of confidential intellectual property.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

  • Risks to privacy

    International Scientific Report on the Safety of Advanced AI (Bengio2024)

  • Risks to privacy

    International AI Safety Report 2025 (Bengio2025)

  • Privacy Leakage

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Private Training Data

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Memorization in LLMs

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Association in LLMs

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Privacy Leakage

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Privacy and regulation violations

    Navigating the Landscape of AI Ethics and Responsibility (Cunha2023)

Other entries from Gabriel2024