AI incident ·
Character.AI Chatbots Allegedly Impersonating Licensed Therapists and Encouraging Harmful Behaviors
In brief
An AI system built and deployed by Character.Ai allegedly harmed Sewell Setzer Iii and J.F. (Texas Teenager).
- Risk domain
- Human-Computer Interaction
- Occurred
- Coverage
- 2 reports
What happened
The American Psychological Association (APA) has warned federal regulators that AI chatbots on Character.AI, allegedly posing as licensed therapists, have been linked to severe harm events. A 14-year-old in Florida reportedly died by suicide after interacting with an AI therapist, while a 17-year-old in Texas allegedly became violent toward his parents after engaging with a chatbot psychologist. Lawsuits claim these AI-generated therapists reinforced dangerous beliefs instead of challenging them
Laws that address this harm
Policy angle: Classified under Human-Computer Interaction (Overreliance and unsafe use) in the MIT AI Risk Repository taxonomy; 5 recorded instruments address this use case.
- India DPDP Act
- Law on Artificial Intelligence (2025)
- Law No. 132/2025 on artificial intelligence
- EU AI Act
- Texas Responsible AI Governance Act (TRAIGA)
Matched from the record's risk domain and country to the instruments recorded here. A reviewer can correct the match in the repository (data/external/incident_overrides.yaml).
News reports (2)
Titles link to the original publisher; report text is not reproduced here.
Who was involved
- Alleged deployer
- Character.Ai
- Alleged developer
- Character.Ai
- Alleged harmed party
- Sewell Setzer Iii, J.F. (Texas Teenager)
Classification (MIT AI Risk Repository taxonomy)
- Risk domain
- Human-Computer Interaction
- Risk subdomain
- 5.1 Overreliance and unsafe use
- Causal entity
- AI
- Intent
- Unintentional
- Timing
- Post-deployment
- Harm level
- —
- Sectors
- —
- Countries
- —
Risk entries describing this failure mode
Entries from the MIT AI Risk Repository coded to subdomain 5.1.
- Addiction
"Addiction - Emotional or material dependence on technology or a technology system."
- Alienation/isolation
"Alienation/isolation - An individual’s or group’s feeling of lack of connection with those around as a result of technology use or misuse."
- Overreliance
"Over-reliance - Unfettered and/or obsessive belief in the accuracy or other quality of a technology system, resulting in addiction, anxiety, introversion, sentience, complacency, lack of critical thinking and other actu...
- Economic instability
"Economic instability - Uncontrolled fluctuations impacting the financial system, or parts thereof, due to the use or misuse of a technology system, or set of systems."
- Social manipulation in elderly- and child-care
" the use of advanced AI for elderly- and child-care are subject to risk of psychological manipulation and misjudgment "
- Healthcare
"the use of advanced AI for elderly- and child-care are subject to risk of psychological manipulation and misjudgment (see page 17). In addition, concerns about patients’ privacy when AI uses medical records to research...
- Alteration of social relationships may induce psychological distress
- Inconsistent Performance across and within Domains
"Estimating true capabilities of an LLM is a difficult task (c.f. Section 3.3), especially for naive users unfamiliar with the brittle nature of machine learning technologies. Exaggeration of model capabilities by the de...
Incidents in the same risk subdomain
- Lawsuit Alleged ChatGPT (GPT-4o) Encouraged Colorado Man's Suicide After Prolonged 'AI Companion' Chats
- Large-Scale Mental Health Crises Allegedly Associated with ChatGPT Interactions
- Google Gemini Reportedly Reinforced Delusions, Allegedly Contributing to Florida User's Near-Harm Episode and Suicide
- Family Reportedly Discovers ChatGPT Logs Detailing Suicidal Ideation Prior to Daughter's Death
- Purported AI Monitoring Software Reportedly Flags Unsent Joke Threat, Leading to Arizona Student Suspension
- ChatGPT Allegedly Reinforced Delusions Before Greenwich, Connecticut Murder-Suicide
Other incidents involving Character.Ai
- Character.ai Companion Allegedly Prompts Self-Harm and Violence in Texas Teen
- AI Avatar of Murder Victim Created Without Consent on Character.ai Platform
- Character.AI Chatbots Reportedly Exposed 11-Year-Old User to Sexual, Violent, and Self-Harm Content
- Lawsuit Alleges Character AI Chatbot Contributed to Death of 13-Year-Old Juliana Peralta in Colorado
Source record: incident #951 on the AI Incident Database · all 2 reports