Unrepresentative risk testing

Classification

Domain

6. Socioeconomic & Environmental

Subdomain

6.5 Governance failure

Causal entity

Human

Intent

Unintentional

Timing

Pre-deployment

Subdomain definition: Inadequate regulatory frameworks and oversight mechanisms failing to keep pace with AI development, leading to ineffective governance and the inability to manage AI risks appropriately.

Real-world incidents in this subdomain

• 4 Mar 2026Nippon Life Alleged ChatGPT Practiced Law Without a License in Illinois Disability Case
• 10 Feb 2026OpenAI Allegedly Did Not Alert RCMP After ChatGPT Flagged Violent Chats Before British Columbia School Shooting
• 1 Jun 2022Remotely Operated Taser-Armed Drones Proposed by Taser Manufacturer as Defense for School Shootings in the US

How other frameworks describe this risk

• Mobility

  The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)

• Liability issues in case of accidents

  The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)

• Faster scientific progress makes it harder for governance to keep pace with development

  A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values (Clarke2023)

• Type 1: Diffusion of responsibility

  TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI (Critch2023)

• Products Liability Law

  Generating Harms - Generative AI's impact and paths forwards (EPIC2023)

• Difficult to develop metrics for evaluating benefits or harms caused by AI assistants

  The Ethics of Advanced AI Assistants (Gabriel2024)

• Institutional responsibilities

  The Ethics of Advanced AI Assistants (Gabriel2024)

• Benchmarking (Cross-lingual data contamination)

  Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024)

Other entries from IBM2025

• - Risk Category
• Lack of training data transparency Risk Sub-Category · 6.5
• Lack of training data transparency Additional evidence
• Uncertain data provenance Risk Sub-Category · 6.5
• Uncertain data provenance Additional evidence
• - Risk Category
• Data usage restrictions Risk Sub-Category · 7.3
• Data usage restrictions Additional evidence
• Data acquisition restrictions Risk Sub-Category · 7.3
• Data acquisition restrictions Additional evidence
• Data transfer restrictions Risk Sub-Category · 7.3
• Data transfer restrictions Additional evidence