AI incident ·
Discord's Automated Moderation System Reportedly Wrongfully Banned More Than 8,000 Users Over Benign Images
On this page 6 sections
In brief
An AI system built and deployed by Discord and Social Media Platforms allegedly harmed Discord Users and Social Media Users.
- Risk domain
- Not classified
- Occurred
- Coverage
- 2 reports
What happened
Discord reportedly wrongfully banned more than 8,000 accounts after its automated image-moderation workflow flagged harmless uploads, including grids, spreadsheets, chessboards, and game images, as prohibited material. Discord said a bug converted temporary upload restrictions into account bans and prevented cleared accounts from being restored automatically. The company later reinstated the affected accounts.
Laws on record for this risk area
No instrument on record yet covers this risk area where it happened. See the open queue.
Matched from the record's risk category and country to the instruments in effect recorded here: a law listed covers the area, which is not a finding that it applies to this incident. A reviewer can correct the match in the repository (data/external/incident_overrides.yaml).
News reports (2)
Titles link to the original publisher; report text is not reproduced here.
Who was involved
- Alleged deployer
- Discord, Social Media Platforms
- Alleged developer
- Discord, Social Media Platforms
- Alleged harmed party
- Discord Users, Social Media Users
Classification (MIT AI Risk Repository taxonomy)
- Risk domain
- —
- Risk subdomain
- —
- Causal entity
- —
- Intent
- —
- Timing
- —
- Harm level
- —
- Sectors
- —
- Countries
- —
Source record: incident #1585 on the AI Incident Database · all 2 reports