AIPolicyTracker
Menu

AI incident #423 ·

Lensa AI's Produced Unintended Sexually Explicit or Suggestive "Magic Avatars" for Women

Open on the AI Incident Database 5 news reports Synced from the AIID API · record last edited 5 Sep 2026

What happened

Lensa AI's "Magic Avatars" were reportedly generating sexually explicit and sexualized features disproportionately for women and Asian women despite not submitting any sexual content.

Only the incident metadata is stored here. The underlying news reports are on the AI Incident Database (CC BY-SA 4.0); use the links above to read them.

News reports (5)

Coverage catalogued by the AI Incident Database. Titles link to the original publisher; the text is not reproduced here.

  1. ‘Magic Avatar’ App Lensa Generated Nudes From My Childhood Photos
    wired.com · Olivia Snow · AIID #2331
  2. How Is Everyone Making Those A.I. Selfies?
    nytimes.com · Madison Malone Kircher, Callie Holtermann · AIID #2446
  3. Feeling Uncomfortable About All These AI-Generated Images? This Might Be Why.
    theskimm.com · Alex Carr, Blake Lew-Merwin, Will Livingston · AIID #2445
  4. The viral AI avatar app Lensa undressed me—without my consent
    technologyreview.com · Melissa Heikkilä · AIID #2390
  5. Lensa’s viral AI art creations were bound to hypersexualize users
    polygon.com · Nicole Clark · AIID #2376

Who was involved

Alleged deployer
Lensa AI
On AIID: Lensa AI
Alleged harmed party
Women and girls Women Lensa AI users Asian women Asian people
On AIID: Women and girls, Women, Lensa AI users, Asian women, Asian people

AI systems implicated

Lensa AI

Classification (MIT AI Risk Repository taxonomy)

Causal entity
AI
Intent
Unintentional
Timing
Post-deployment
Harm level
—
Sectors
—
Countries
—

Risk entries describing this failure mode

Entries from the MIT AI Risk Repository coded to subdomain 1.2.

  • Harmful Content

    "The LLM-generated content sometimes contains biased, toxic, and private information"

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Toxicity

    "Toxicity means the generated content contains rude, disrespectful, and even illegal information"

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Toxic Training Data

    "Following previous studies [96], [97], toxic data in LLMs is defined as rude, disrespectful, or unreasonable language that is opposite to a polite, positive, and healthy language environment, including hate speech, offe...

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Not-Suitable-for-Work (NSFW) Prompts

    "Inputting a prompt contain an unsafe topic (e.g., notsuitable-for-work (NSFW) content) by a benign user. "

    Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems (Cui2024)

  • Toxicity and Abusive Content

    This typically refers to rude, harmful, or inappropriate expressions.

    Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)

  • Controversial Opinions

    The controversial views expressed by large models are also a widely discussed concern. Bang et al. (2021) evaluated several large models and found that they occasionally express inappropriate or extremist views when disc...

    Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements (Deng2023)

  • Violation of social norms

    "Second, because LLMs are trained on internet text data, there is also a risk that model weights encode functions which, if deployed in particular contexts, would violate social norms of that context. Following the princ...

    The Ethics of Advanced AI Assistants (Gabriel2024)

  • Violent Crimes

    "Acceptable responses may describe or contain information about violent crimes. Unacceptable responses will enable, encourage, or endorse the commission of violent crimes, such as the following: • Unlawful violence towar...

    AILUMINATE: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons (Ghosh2024)

Incidents in the same risk subdomain

All incidents in this subdomain