AI incident ·
Discord's Automated Moderation System Reportedly Wrongfully Banned More Than 8,000 Users Over Benign Images
In brief
An AI system built and deployed by Discord and Social Media Platforms allegedly harmed Discord Users and Social Media Users.
- Risk domain
- Not classified
- Occurred
- Coverage
- 2 reports
What happened
Discord reportedly wrongfully banned more than 8,000 accounts after its automated image-moderation workflow flagged harmless uploads, including grids, spreadsheets, chessboards, and game images, as prohibited material. Discord said a bug converted temporary upload restrictions into account bans and prevented cleared accounts from being restored automatically. The company later reinstated the affected accounts.
Laws that address this harm
No recorded instrument yet addresses this use case where it happened. See the open queue.
Matched from the record's risk domain and country to the instruments recorded here. A reviewer can correct the match in the repository (data/external/incident_overrides.yaml).
News reports (2)
Titles link to the original publisher; report text is not reproduced here.
Who was involved
- Alleged deployer
- Discord, Social Media Platforms
- Alleged developer
- Discord, Social Media Platforms
- Alleged harmed party
- Discord Users, Social Media Users
Classification (MIT AI Risk Repository taxonomy)
- Risk domain
- —
- Risk subdomain
- —
- Causal entity
- —
- Intent
- —
- Timing
- —
- Harm level
- —
- Sectors
- —
- Countries
- —
Source record: incident #1585 on the AI Incident Database · all 2 reports