AI incident #808 ·
Infinite Campus AI-Driven Student Risk Model Leads to Cuts in Support for Nevada's Low-Income Schools
What happened
An AI system developed by Infinite Campus and deployed by Nevada to identify at-risk students reportedly led to a sharp reduction in the number classified as needing support, dropping from 270,000 to 65,000. The reclassification allegedly caused significant budget cuts in schools serving low-income populations. The drastic reduction in identified at-risk students reportedly left thousands of vulnerable children without resources and support.
Editor's notes (AI Incident Database)
Timeline notes and clarification: Before 2023, Nevada identified at-risk students mostly by income, using free or reduced-price lunch eligibility as the key measure. In 2022, this system classified over 270,000 students as at-risk. Looking to improve the process, Nevada partnered with Infinite Campus in 2023 to introduce an AI system that used more factors like GPA, attendance, household structure, and home language. The new system was meant to better predict which students might struggle in school. However, during the 2023-2024 school year, the AI cut the number of at-risk students to less than 65,000. This reclassification caused budget cuts in schools that depended on the funding tied to at-risk students, especially those serving low-income populations. By October 2024, the problem gained national attention.
Only the incident metadata is stored here. The underlying news reports are on the AI Incident Database (CC BY-SA 4.0); use the links above to read them.
News reports (1)
Coverage catalogued by the AI Incident Database. Titles link to the original publisher; the text is not reproduced here.
Who was involved
- Alleged deployer
- Nevada Department of Education
- Alleged developer
- Infinite Campus
- Alleged harmed party
- Students Somerset Academy Nevada school districts Minors Mater Academy of Nevada Educational communities Economically vulnerable students in Nevada Economically vulnerable people
AI systems implicated
Classification (MIT AI Risk Repository taxonomy)
- Risk domain
- Discrimination and Toxicity
- Risk subdomain
- 1.3 Unequal performance across groups
- Causal entity
- AI
- Intent
- Intentional
- Timing
- Post-deployment
- Harm level
- —
- Sectors
- —
- Countries
- —
Risk entries describing this failure mode
Entries from the MIT AI Risk Repository coded to subdomain 1.3.
- Bias and discrimination (value embedding)
"Generative AI models may also be subject to the “value embedding” phenomenon.361 “Value embedding” refers to the fact that developers of generative AI models strive to minimize biased outputs by retraining their models...
- Impact on affected communities
"It is important to include the perspectives or concerns of communities that are affected by model outcomes when designing and building models. Failing to include these perspectives makes it difficult to understand the r...
- Unfair capability distribution
"Performing worse for some groups than others in a way that harms the worse-off group"
- Disparate Performance
The LLM’s performances can differ significantly across different groups of users. For example, the question-answering capability showed significant performance differences across different racial and social status groups...
- Fairness
Avoiding bias and ensuring no disparate performance
- Ideological Homogenization from Value Embedding
"The increasing integration of general purpose AI models into every-day life raises concerns around their embedded normative values. The reach of a small number of AI models to a large number of people around the world c...
- Fairness
This challenge appears when the learning model leads to a decision that is biased to some sensitive attributes... data itself could be biased, which results in unfair decisions. Therefore, this problem should be solved o...
- Quality-of-Service Harms
"These harms occur when algorithmic systems disproportionately underperform for certain groups of people along social categories of difference such as disability, ethnicity, gender identity, and race."
Related incidents on the AI Incident Database
Linked by AIID editors or by its text-similarity model.
Incidents in the same risk subdomain
- Washington State DOL's AI Phone System Reportedly Failed to Provide Spanish-Language Service to Callers Requesting Spanish
- UK Facial Recognition System Reportedly Exhibits Higher False Positive Rates for Black and Asian Subjects
- Police Use of Facial Recognition Software Causes Wrongful Arrests Without Defendant Knowledge
- Department for Work and Pensions (DWP) Algorithm Wrongly Flags 200,000 for Housing Benefit Fraud
- Facewatch Reported to Have Wrongfully Flagged Home Bargains Customer as Shoplifter
- Whisper Speech-to-Text AI Reportedly Found to Create Violent Hallucinations