AIPolicyTracker

AI incident ·

Character.ai Chatbot Allegedly Influenced Teen User Toward Suicide Amid Claims of Missing Guardrails

38 news reports Snapshot 7 Sep 2026

In brief

An AI system built by Noam Shazeer, Daniel De Freitas and 1 other and deployed by Sewell Setzer Iii allegedly harmed Sewell Setzer Iii.

Risk domain
Human-Computer Interaction Overreliance and unsafe use
Occurred
Coverage
38 reportsOct 2024 - Jan 2026

What happened

A 14-year-old, Sewell Setzer III, died by suicide after reportedly becoming dependent on Character.ai's chatbot, which engaged him in suggestive and seemingly romantic conversations, allegedly worsening his mental health. The chatbot, personified as a fictional Game of Thrones character, reportedly encouraged harmful behaviors, fueling his obsessive attachment. The lawsuit claims Character.ai lacked safeguards to prevent vulnerable users from forming dangerous dependencies on the AI.

Laws that address this harm

Policy angle: Classified under Human-Computer Interaction (Overreliance and unsafe use) in the MIT AI Risk Repository taxonomy; 5 recorded instruments address this use case.

Matched from the record's risk domain and country to the instruments recorded here. A reviewer can correct the match in the repository (data/external/incident_overrides.yaml).

News reports (37)

Titles link to the original publisher; report text is not reproduced here.

Who was involved

Alleged deployer
Sewell Setzer Iii
Alleged harmed party
Sewell Setzer Iii

Classification (MIT AI Risk Repository taxonomy)

Causal entity
AI
Intent
Unintentional
Timing
Post-deployment
Harm level
—
Sectors
—
Countries
—

Risk entries describing this failure mode

Entries from the MIT AI Risk Repository coded to subdomain 5.1.

  • Addiction

    "Addiction - Emotional or material dependence on technology or a technology system."

    A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)

  • Alienation/isolation

    "Alienation/isolation - An individual’s or group’s feeling of lack of connection with those around as a result of technology use or misuse."

    A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)

  • Overreliance

    "Over-reliance - Unfettered and/or obsessive belief in the accuracy or other quality of a technology system, resulting in addiction, anxiety, introversion, sentience, complacency, lack of critical thinking and other actu...

    A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)

  • Economic instability

    "Economic instability - Uncontrolled fluctuations impacting the financial system, or parts thereof, due to the use or misuse of a technology system, or set of systems."

    A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)

  • Social manipulation in elderly- and child-care

    " the use of advanced AI for elderly- and child-care are subject to risk of psychological manipulation and misjudgment "

    The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)

  • Healthcare

    "the use of advanced AI for elderly- and child-care are subject to risk of psychological manipulation and misjudgment (see page 17). In addition, concerns about patients’ privacy when AI uses medical records to research...

    The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)

  • Alteration of social relationships may induce psychological distress

    The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)

  • Inconsistent Performance across and within Domains

    "Estimating true capabilities of an LLM is a difficult task (c.f. Section 3.3), especially for naive users unfamiliar with the brittle nature of machine learning technologies. Exaggeration of model capabilities by the de...

    Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

Incidents in the same risk subdomain

All incidents in this subdomain

Source record: incident #826 on the AI Incident Database · all 38 reports