AI incident ·
Houston ISD's EVAAS Teacher-Evaluation System Reportedly Put Teachers' Jobs at Risk Through Unverifiable Scores
In brief
An AI system built by SAS Institute and deployed by Houston Independent School District allegedly harmed Teachers, Houston Independent School District teachers and 2 others.
- Risk domain
- AI system safety, failures, and limitations
- Occurred
- Coverage
- 1 report
What happened
From 2011 to 2015, Houston Independent School District used SAS Institute’s Education Value-Added Assessment System (EVAAS) to rate teacher effectiveness from student test-score growth. Teachers alleged that proprietary, unverifiable EVAAS scores contributed to terminations and contract nonrenewals. On May 4, 2017, a federal judge allowed due-process claims over the system to proceed.
Laws that address this harm
Policy angle: Classified under AI system safety, failures, and limitations (Lack of transparency or interpretability) in the MIT AI Risk Repository taxonomy; 5 recorded instruments address this use case in the United States.
- Texas Responsible AI Governance Act (TRAIGA)
- California SB 53
- Tennessee ELVIS Act
- EO 14179
- New York RAISE Act (frontier model safety)
Matched from the record's risk domain and country to the instruments recorded here. A reviewer can correct the match in the repository (data/external/incident_overrides.yaml).
News reports (1)
Titles link to the original publisher; report text is not reproduced here.
Who was involved
- Alleged deployer
- Houston Independent School District
- Alleged developer
- SAS Institute
- Alleged harmed party
- Teachers Houston Independent School District teachers Educators Educational communities
AI systems implicated
Value-added teacher evaluation modelsEducation Value-Added Assessment System (EVAAS)Algorithmic management systemsAI-enabled decision support systems
Classification (MIT AI Risk Repository taxonomy)
- Risk domain
- AI system safety, failures, and limitations
- Risk subdomain
- 7.4 Lack of transparency or interpretability
- Causal entity
- AI
- Intent
- Intentional
- Timing
- Post-deployment
- Harm level
- none
- Sectors
- education
- Countries
- US
Risk entries describing this failure mode
Entries from the MIT AI Risk Repository coded to subdomain 7.4.
- Intelligibility
"How can we build agent’s whose decisions we can understand? Con- nects explainable decisions (Berkeley) and informed oversight (MIRI)."
- Opacity (the black box problem)
"Opacity surrounding the technical, internal decision-making processes of generative AI models is popularly known as the “black box problem.”277 Generative AI models, most ubiquitously built on deep neural networks with...
- Lack of transparency and interpretability
"Today's Frontier AI is difficult to interpret and lacks transparency. Contextual understanding of the training data is not explicitly embedded within these models. They can fail to capture perspectives of underrepresent...
- Attributing the responsibility for AI's failures
"This section, constituting almost 8% of the articles, addresses the implications arising from AI acting and learning without direct human supervision, encompassing two main issues: a responsibility gap and AI's moral st...
- General Evaluations (Difficulty of identification and measurement of capabilities)
"The capabilities of general-purpose AI systems can be difficult to measure, compared to the capabilities of more limited and fixed-purpose AI systems. This is in part due to a broader distribution of potential risks, a...
- Model Evaluations (Interpretability/Explainability)
- Model outputs inconsistent with chain-of-thought reasoning
"Chain-of-thought reasoning is sometimes employed to get a better understanding of the model’s output, where it encourages transparent reasoning in text form. However, in some cases, this reasoning is not consistent with...
- Lack of understanding of in-context learning in language models
"In-context learning allows the model to learn a new task or improve its perfor- mance by providing examples in the prompt, without changing its weights [101]. Even though this technique is highly effective, its working...
Related incidents
Linked by editors or by text similarity in the source dataset.
Incidents in the same risk subdomain
- Uber Launched Opaque Algorithm That Changes Drivers' Payments in the US
- Israeli Tax Authority Reportedly Used an Opaque Automated System to Issue a Fine, Declining to Explain or Disclose the Underlying Calculation
- Uber Allegedly Wrongfully Accused Drivers of Fraud via Automated Systems
- Dutch City Court Defended Home Value Generated by Black-Box Algorithm
Source record: incident #96 on the AI Incident Database · all 1 report