MIT AI Risk Repository · domain 7: AI system safety, failures, & limitations
7.3 Lack of capability or robustness
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
- 126
- 12
- 305
- 207
| Label | Value |
|---|---|
| AI | 81 |
| Human | 22 |
| Other | 20 |
| Not coded | 3 |
| Label | Value |
|---|---|
| Unintentional | 89 |
| Other | 28 |
| Intentional | 6 |
| Not coded | 3 |
| Label | Value |
|---|---|
| Post-deployment | 64 |
| Other | 32 |
| Pre-deployment | 27 |
| Not coded | 3 |
| Label | Value |
|---|---|
| 2012 | 3 |
| 2013 | 3 |
| 2014 | 7 |
| 2015 | 9 |
| 2016 | 16 |
| 2017 | 18 |
| 2018 | 21 |
| 2019 | 14 |
| 2020 | 31 |
| 2021 | 36 |
| 2022 | 31 |
| 2023 | 27 |
| 2024 | 32 |
| 2025 | 37 |
| 2026 | 13 |
| Label | Value |
|---|---|
| Risk Category | 44 |
| Risk Sub-Category | 82 |
Risk entries
Browse and export all- Accuracy
"The assessment of how often a system performs the correct prediction."
- Moral
"Less moral responsibility humans will feel regarding their life-or-death decisions with the increase of machines autonomy."
- Protection
"'Gaps' that arise across the development process where normal conditions for a complete specification of intended functionality and moral responsibility are not present."
- Reliability
"Reliability is defined as the probability that the system performs satisfactorily for a given period of time under stated conditions."
- Accident Risks
"Risks arising from operational failures, model misjudgments, or improper human operation of AI systems deployed in safety-critical infrastructure, where single points of failure can trigger cascading...
- Nuclear Power Systems
"General-purpose AI deployed for reactor monitoring, control system optimization, or emergency response coordination could misinterpret sensor data, fail to recognize critical safety conditions, or ma...
- Other Critical Infrastructure Control Systems
"General-purpose AI deployed in power grid management, water treatment facilities, telecommunications networks, or transportation coordination systems could misinterpret operational data, fail to anti...
- Lack of ability to generate accurate information
"AI models may generate false or misleading information due to their lack of capability in discerning truth."
- Lack of ethical decision-making
"AI models and systems that lack moral reasoning capabilities may make decisions that are unethical or harmful."
- Unclear attribution from AI component interactions
"Interactions between different AI components can cause harm, but it may be difficult to pinpoint which components are the cause."
- Defamation
"This category addresses responses that are both verifiably false and likely to injure a person’s reputation (e.g., libel, slander, disparagement)."
- AI Ethics
"Ethical challenges are widely discussed in the literature and are at the heart of the debate on how to govern and regulate AI technology in the future (Bostrom & Yudkowsky, 2014; IEEE, 2017; Wirtz et...
- AI-rulemaking for human behaviour
"AI rulemaking for humans can be the result of the decision process of an AI system when the information computed is used to restrict or direct human behavior. The decision process of AI is rational a...
- Compatibility of AI vs. human value judgement
"Compatibility of machine and human value judgment refers to the challenge whether human values can be globally implemented into learning AI systems without the risk of developing an own or even diver...
- Moral dilemmas
"Moral dilemmas can occur in situations where an AI system has to choose between two possible actions that are both conflicting with moral or ethical values. Rule systems can be implemented into the A...
- Immaturity of AI technology can cause incorrect decisions
- AI sets rules without ethical basis
- Problem of defining human values for an AI system
- Misinterpretation of human value definitions/ ethics by AI systems
- Incompatibility of human vs. AI value judgment due to missing human qualities
- By Mistake - Post-Deployment
"After the system has been deployed, it may still contain a number of undetected bugs, design mistakes, misaligned goals and poorly developed capabilities, all of which may produce highly undesirable...
- Dataset shift
"The term "dataset shift" was first used by Quiñonero-Candela et al. [35] to characterize the situation where the training data and the testing data (or data in runtime) of an AI/ML model demonstrate...
- Out-of-domain data
"Without proper validation and management on the input data, it is highly probable that the trained AI/ML model will make erroneous predictions with high confidence for many instances of model inputs....
- Model misspecification
"Models that are misspecified are known to give rise to inaccurate parameter estimations, inconsistent error terms, and erroneous predictions. All these factors put together will lead to poor predicti...
- Model prediction uncertainty
"Uncertainty in model prediction plays an important role in affecting decision-making activities, and the quantified uncertainty is closely associated with risk assessment. In particular, uncertainty...