Decision making transparency

Classification

Domain

7. AI system safety, failures, & limitations

Subdomain

7.4 Lack of transparency or interpretability

Causal entity

AI

Intent

Other

Timing

Post-deployment

Subdomain definition: Challenges in understanding or explaining the decision-making processes of AI systems, which can lead to mistrust, difficulty in enforcing compliance standards or holding relevant actors accountable for harms, and the inability to identify and correct errors.

Real-world incidents in this subdomain

• 10 Feb 2022Uber Launched Opaque Algorithm That Changes Drivers' Payments in the US
• 11 Jan 2021Israeli Tax Authority Reportedly Used an Opaque Automated System to Issue a Fine, Declining to Explain or Disclose the Underlying Calculation
• 7 Jul 2018Uber Allegedly Wrongfully Accused Drivers of Fraud via Automated Systems
• 8 May 2017Houston ISD's EVAAS Teacher-Evaluation System Reportedly Put Teachers' Jobs at Risk Through Unverifiable Scores
• 1 Feb 2016Dutch City Court Defended Home Value Generated by Black-Box Algorithm

How other frameworks describe this risk

• Intelligibility

  AGI Safety Literature Review (Everitt2018 )

• Attributing the responsibility for AI's failures

  What Ethics Can Say on Artificial Intelligence: Insights from a Systematic Literature Review (Giarmoleo2024)

• Model Evaluations (Interpretability/Explainability)

  Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024)

• Model outputs inconsistent with chain-of-thought reasoning

  Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024)

• Lack of understanding of in-context learning in language models

  Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024)

• General Evaluations (Difficulty of identification and measurement of capabilities)

  Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024)

• Lack of transparency and interpretability

  Future Risks of Frontier AI (GOS2023)

• Opacity (the black box problem)

  Regulating under Uncertainty: Governance Options for Generative AI (G'sell2024)

Other entries from Meek2016

• Domain-specific AI - Effects on humans and other living beings: Existential Risks Risk Category
• Unethical decision making Risk Sub-Category · 7.3
• Domain-specific AI - Effects on humans and other living beings: Non-existential risks Risk Category
• Privacy Risk Sub-Category · 2.1
• Human dignity/respect Risk Sub-Category · 5.2
• Safety Risk Sub-Category · 7.3
• Law abiding Risk Sub-Category · 7.3
• Inequality of wealth Risk Sub-Category · 6.2
• Societal manipulation Risk Sub-Category · 7.1
• AGI - Effects on humans and other living beings: Existential risks Risk Category
• Direct competition with humans Risk Sub-Category · 6.2
• Unpredictable outcomes Risk Sub-Category · 7.1