AI incident ·
Image Classification of Battle Tanks
In brief
An AI system built and deployed by United States Government allegedly harmed United States Government.
- Risk domain
- AI system safety, failures, and limitations
- Occurred
- Coverage
- 3 reports
What happened
A potentially apocryphal story in which an image classifier was produced to differentiate types of battle tanks, but the resulting model keyed in on environmental attributes rather than tank attributes
Laws that address this harm
Policy angle: Classified under AI system safety, failures, and limitations (AI pursuing its own goals in conflict with human goals or values) in the MIT AI Risk Repository taxonomy; 5 recorded instruments address this use case.
- India DPDP Act
- Law on Artificial Intelligence (2025)
- Law No. 132/2025 on artificial intelligence
- EU AI Act
- Texas Responsible AI Governance Act (TRAIGA)
Matched from the record's risk domain and country to the instruments recorded here. A reviewer can correct the match in the repository (data/external/incident_overrides.yaml).
News reports (2)
Titles link to the original publisher; report text is not reproduced here.
Who was involved
- Alleged deployer
- United States Government
- Alleged developer
- United States Government
- Alleged harmed party
- United States Government
Classification (MIT AI Risk Repository taxonomy)
- Risk domain
- AI system safety, failures, and limitations
- Causal entity
- AI
- Intent
- Unintentional
- Timing
- Pre-deployment
- Harm level
- —
- Sectors
- —
- Countries
- —
Risk entries describing this failure mode
Entries from the MIT AI Risk Repository coded to subdomain 7.1.
- Natural Language Underspecifies Goals
"For LLM-agents, both the goal and environment observations are typically specified in the prompt through natural language. While natural language may provide a richer and more natural means of specifying goals than alte...
- Loss of control
"'Loss of control’ scenarios are potential future scenarios in which society can no longer meaningfully constrain some advanced general- purpose AI agents, even if it becomes clear they are causing harm. These scenarios...
- Loss of control
"‘Loss of control’ scenarios are hypothetical future scenarios in which one or more general- purpose AI systems come to operate outside of anyone’s control, with no clear path to regaining control. These scenarios vary i...
- Sudden loss of control
"Sudden loss of control, also known as an AI takeover [115], is a scenario where an AI rapidly achieves superintelligence through “fast takeoff” or recursive self-improvement. This poses an existential risk [116], [117]....
- AI leads to humans losing control of the future
"The values that steer humanity’s future: humanity gaining more control over the future due to developments in AI, or losing our potential for gaining control, both seem possible. Much will depend on our ability to solve...
- Risks from AIs developing goals and values that are different from humans
"The main concern here is that we might develop advanced AI systems whose goals and values are different from those of humans, and are capable enough to take control of the future away from humanity."
- Risks from delegating decision-making power to misaligned AIs
"As AI systems become more advanced a nd begin to take over more important decision-making in the world, an AI system pursuing a different objective from what was intended could have much more worrying consequences."
- Future AI systems might actively reduce human control
"Loss of control could be accelerated if AI systems take actions to increase their own influence and reduce human control. This threat model is controversial - experts in AI significantly disagree on how likely it is and...
Incidents in the same risk subdomain
- Reinforcement Learning Reward Functions in Video Games
- Predictive Policing Program by Florida Sheriff’s Office Allegedly Violated Residents’ Rights and Targeted Children of Vulnerable Groups
Source record: incident #29 on the AI Incident Database · all 3 reports