AI incident ·
Gaggle Alert Reportedly Leads to Arrest of 15-Year-Old in Volusia County, Florida, for School Threat the Student Claimed Was Not Serious
On this page 9 sections
In brief
An AI system built by Gaggle and deployed by Volusia County Sheriff's Office, Volusia County Schools and 1 other allegedly harmed Unnamed 15-year-old student at Deltona High School, Students and 3 others.
- Risk domain
- Malicious Actors & Misuse
- Occurred
- Coverage
- 1 report
What happened
A 15-year-old student in Volusia County, Florida, was reportedly detained after the Gaggle student monitoring system flagged a written shooting threat on a school-issued laptop. The student reportedly claims the threat was not serious. The alert reportedly led to the student's arrest on felony charges. The system appeared to operate as designed but reportedly resulted in criminal enforcement against a minor.
Laws on record for this risk area
Policy angle: Classified under Malicious Actors & Misuse (Disinformation, surveillance, and influence at scale) in the MIT AI Risk Repository taxonomy; 5 instruments on record cover this risk area.
- India DPDP Act
- Law on Artificial Intelligence (2025)
- Law No. 132/2025 on artificial intelligence
- EU AI Act
- Texas Responsible AI Governance Act (TRAIGA)
Matched from the record's risk category and country to the instruments in effect recorded here: a law listed covers the area, which is not a finding that it applies to this incident. A reviewer can correct the match in the repository (data/external/incident_overrides.yaml).
News reports (1)
Titles link to the original publisher; report text is not reproduced here.
Who was involved
- Alleged deployer
- Volusia County Sheriff's Office Volusia County Schools Gaggle
- Alleged developer
- Gaggle
- Alleged harmed party
- Unnamed 15-year-old student at Deltona High School Students Privacy Minors Educational communities
AI systems implicated
Classification (MIT AI Risk Repository taxonomy)
- Risk domain
- Malicious Actors & Misuse
- Risk subdomain
- 4.1 Disinformation, surveillance, and influence at scale
- Causal entity
- AI
- Intent
- Intentional
- Timing
- Post-deployment
- Harm level
- —
- Sectors
- —
- Countries
- —
Risk entries describing this failure mode
Entries from the MIT AI Risk Repository coded to subdomain 4.1.
- Electoral interference
"Electoral interference - Generation of false or misleading information that can interrupt or mislead voters and/or undermine trust in electoral processes."
- Political and Economic
"Political and Economic - Manipulation of political beliefs, damage to political institutions and the effective delivery of government services."
- Coercion/manipulation
"Coercion/manipulation - Use of a technology system to covertly alter user beliefs and behaviour using nudging, dark patterns and/or other opaque techniques, resulting in potential erosion of privacy, addiction, anxiety/...
- Political manipulation
"Political manipulation - Use or misuse of personal data to target individuals’ interests, personalities and vulnerabilities with tailored political messages via micro-advertising or deepfakes/synthetic media."
- Political
"In the UK, a form of initial computational propaganda has already happened during the Brexit referendum1 . In future, there are concerns that oppressive governments could use AI to shape citizens’ opinions"
- Biased influence through citizen screening and tailored propaganda
"AI-powered chatbots tailor their communication approach to influence individual users' decisions. In the UK, a form of initial computational propaganda has already happened during the Brexit referendum. In future, there...
- Surveillance and Censorship
"Content moderation has emerged as one of the key use-cases of LLMs (Weng et al., 2023), indicating the potential of LLMs for surveillance and censorship as well (Edwards, 2023). Surveillance and censorship are one of th...
- Disinformation and manipulation of public opinion
"AI, particularly general- purpose AI, can be maliciously used for disinformation (351), which for the purpose of this report refers to false information that was generated or spread with the deliberate intent to mislead...
Related incidents
Linked by editors or by text similarity in the source dataset.
- Gaggle AI Monitoring at Lawrence, Kansas High School Reportedly Misflags Student Content and Blocks Emails
- Predictive Policing Program by Florida Sheriff’s Office Allegedly Violated Residents’ Rights and Targeted Children of Vulnerable Groups
- Detroit Police Allegedly Wrongfully Arrested Black Man Due to Purportedly Faulty Facial Recognition Technology
- Security Robot Rolls Over Child in Mall
Incidents in the same risk subdomain
- Suspected AI-Generated Deepfake Video Reportedly Targeted Former Chhattisgarh Chief Minister Bhupesh Baghel on Instagram
- Purportedly AI-Manipulated Satellite Image Reportedly Claimed Iranian Strike Destroyed U.S. Radar in Qatar
- Purported AI-Generated War Footage Reportedly Circulated Widely Online During the Opening Phase of the War in Iran
- Network of Allegedly Fake Facebook Profiles with Purportedly AI-Generated Images Amplified Posts by Bulgaria's 'There Is Such a People' (ITN) Party
- Purportedly AI-Generated Image Reportedly Circulated Ahead of Thai Election Depicting PM Anutin Charnvirakul Dining with Benjamin Mauerberger
- Purportedly AI-Altered Images Reportedly Distort Evidence After Minneapolis Shooting of ICU Nurse Alex Pretti
Source record: incident #1215 on the AI Incident Database · all 1 report