Dutch City Court Defended Home Value Generated by Black-Box Algorithm

Laws that address this harm

Policy angle: Classified under AI system safety, failures, and limitations (Lack of transparency or interpretability) in the MIT AI Risk Repository taxonomy; 5 recorded instruments address this use case.

• India DPDP Act India · Partially applicable · binding
• Law on Artificial Intelligence (2025) Kazakhstan · Adopted · binding
• Law No. 132/2025 on artificial intelligence Italy · In force · binding
• EU AI Act European Union · Partially applicable · binding
• Texas Responsible AI Governance Act (TRAIGA) Texas (United States) · In force · binding

Matched from the record's risk domain and country to the instruments recorded here. A reviewer can correct the match in the repository (data/external/incident_overrides.yaml).

News reports (1)

Titles link to the original publisher; report text is not reproduced here.

1. 25 Nov 2020
   Dutch city uses algorithm to assess home value, but has no idea how it works

   algorithmwatch.org · Nicolas Kayser-Bril

Who was involved

Alleged deployer

Castricum Municipality

Alleged developer

Castricum Municipality

Alleged harmed party

Unnamed Property Owner

Classification (MIT AI Risk Repository taxonomy)

Risk domain

AI system safety, failures, and limitations

Risk subdomain

7.4 Lack of transparency or interpretability

Causal entity

Human

Intent

Unintentional

Timing

Post-deployment

Harm level

—

Sectors

—

Countries

—

Risk entries describing this failure mode

Entries from the MIT AI Risk Repository coded to subdomain 7.4.

• Intelligibility

  "How can we build agent’s whose decisions we can understand? Con- nects explainable decisions (Berkeley) and informed oversight (MIRI)."

  AGI Safety Literature Review (Everitt2018 )

• Opacity (the black box problem)

  "Opacity surrounding the technical, internal decision-making processes of generative AI models is popularly known as the “black box problem.”277 Generative AI models, most ubiquitously built on deep neural networks with...

  Regulating under Uncertainty: Governance Options for Generative AI (G'sell2024)

• Lack of transparency and interpretability

  "Today's Frontier AI is difficult to interpret and lacks transparency. Contextual understanding of the training data is not explicitly embedded within these models. They can fail to capture perspectives of underrepresent...

  Future Risks of Frontier AI (GOS2023)

• Attributing the responsibility for AI's failures

  "This section, constituting almost 8% of the articles, addresses the implications arising from AI acting and learning without direct human supervision, encompassing two main issues: a responsibility gap and AI's moral st...

  What Ethics Can Say on Artificial Intelligence: Insights from a Systematic Literature Review (Giarmoleo2024)

• General Evaluations (Difficulty of identification and measurement of capabilities)

  "The capabilities of general-purpose AI systems can be difficult to measure, compared to the capabilities of more limited and fixed-purpose AI systems. This is in part due to a broader distribution of potential risks, a...

  Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024)

• Model Evaluations (Interpretability/Explainability)

  Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024)

• Model outputs inconsistent with chain-of-thought reasoning

  "Chain-of-thought reasoning is sometimes employed to get a better understanding of the model’s output, where it encourages transparent reasoning in text form. However, in some cases, this reasoning is not consistent with...

  Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024)

• Lack of understanding of in-context learning in language models

  "In-context learning allows the model to learn a new task or improve its perfor- mance by providing examples in the prompt, without changing its weights [101]. Even though this technique is highly effective, its working...

  Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024)

Incidents in the same risk subdomain

• 10 Feb 2022Uber Launched Opaque Algorithm That Changes Drivers' Payments in the US
• 11 Jan 2021Israeli Tax Authority Reportedly Used an Opaque Automated System to Issue a Fine, Declining to Explain or Disclose the Underlying Calculation
• 7 Jul 2018Uber Allegedly Wrongfully Accused Drivers of Fraud via Automated Systems
• 8 May 2017Houston ISD's EVAAS Teacher-Evaluation System Reportedly Put Teachers' Jobs at Risk Through Unverifiable Scores