System (normal) failures

Description

"In the context of Normal Accident Theory [150], normal accidents are those that “could no longer be ascribed to isolated equipment malfunction, operator error, or acts of God.” We refer to these as “system failures” (to be distinguished from “systemic risks”), while the opposite would be “isolated failures.” For isolated failures, harms are consistent with the underlying failure modes. For example, an AI capable of producing false or misleading content would constitute risks re- lated to misinformation and disinformation. Whereas for system failures, harms are not consistent with the underlyi

From Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems (Gipiškis2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).