AIPolicyTracker
Menu

AI incident ·

Google's Face Grouping Allegedly Collected and Analyzed Users' Facial Structure without Consent, Violated BIPA

2 news reports Synced from source · record last edited 31 Aug 2026

In brief

An AI system built and deployed by Google allegedly harmed Privacy, Illinois residents and 3 others.

Risk domain
Privacy & Security Compromise of privacy by obtaining, leaking or correctly inferring sensitive information
Occurred
Coverage
2 reportsJun 2022 - Sep 2022

What happened

A class-action lawsuit alleged Google failing to provide notice, obtain informed written consent, or publish data retention policies about the collection, storage, and analysis of its face-grouping feature in Google Photos, which violated Illinois Biometric Information Privacy Act (BIPA).

Laws that address this harm

Policy angle: Classified under Privacy & Security (Compromise of privacy by obtaining, leaking or correctly inferring sensitive information) in the MIT AI Risk Repository taxonomy; 3 recorded instruments address this use case in the United States.

Matched from the record's risk domain and country to the instruments recorded here. A reviewer can correct the match in the repository (data/external/incident_overrides.yaml).

News reports (2)

Titles link to the original publisher; report text is not reproduced here.

  1. Google to pay $100 million to Illinois residents for Photos’ face grouping feature
    theverge.com · Emma Roth
  2. Saturday is the last day for Illinois residents to claim a cut of $100 million Google Photos privacy settlement
    chicagotribune.com · Talia Soglin

Who was involved

Alleged deployer
Google
Alleged developer
Google
Alleged harmed party
Privacy Illinois residents Google Photos users residing in Illinois Google Photos users Biometric data subjects

AI systems implicated

Google Photos Face GroupingFacial recognition systemsFaceprint generation systemsBiometric data processing systems

Classification (MIT AI Risk Repository taxonomy)

Causal entity
AI
Intent
Unintentional
Timing
Post-deployment
Harm level
AI tangible harm event
Sectors
information and communication
Countries
US

Risk entries describing this failure mode

Entries from the MIT AI Risk Repository coded to subdomain 2.1.

  • Risks to privacy

    "General- purpose AI models or systems can ‘leak’ information about individuals whose data was used in training. For future models trained on sensitive personal data like health or financial data, this may lead to partic...

    International Scientific Report on the Safety of Advanced AI (Bengio2024)

  • Risks to privacy

    "General- purpose AI systems can cause or contribute to violations of user privacy. Violations can occur inadvertently during the training or usage of AI systems, for example through unauthorised processing of personal d...

    International AI Safety Report 2025 (Bengio2025)

  • Private Training Data

    "As recent LLMs continue to incorporate licensed, created, and publicly available data sources in their corpora, the potential to mix private data in the training corpora is significantly increased. The misused private d...

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Association in LLMs

    "Association in LLMs refers to the capability to associate various pieces of information related to a person. According to [68], [86], given a pair of PII entities (xi , xj ), which is associated by a model F. Using a pr...

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Privacy Leakage

    "Privacy Leakage means the generated content includes sensitive personal information"

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Memorization in LLMs

    "Memorization in LLMs refers to the capability to recover the training data with contextual prefixes. According to [88]–[90], given a PII entity x, which is memorized by a model F. Using a prompt p could force the model...

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Privacy Leakage

    "The model is trained with personal data in the corpus and unintentionally exposing them during the conversation."

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Privacy and regulation violations

    "Some of the broken systems discussed above are also very invasive of people’s privacy, controlling, for instance, the length of someone’s last romantic relationship [51]. More recently, ChatGPT was banned in Italy over...

    Navigating the Landscape of AI Ethics and Responsibility (Cunha2023)

Incidents in the same risk subdomain

All incidents in this subdomain

Other incidents involving Google

Source record: incident #254 on the AI Incident Database · all 2 reports