MIT AI Risk Repository · domain 7: AI system safety, failures, & limitations
7.3 Lack of capability or robustness
AI systems that fail to perform reliably or effectively under varying conditions, exposing them to errors and failures that can have significant consequences, especially in critical applications or areas that require moral reasoning.
- 126
- 12
- 305
- 207
| Label | Value |
|---|---|
| AI | 81 |
| Human | 22 |
| Other | 20 |
| Not coded | 3 |
| Label | Value |
|---|---|
| Unintentional | 89 |
| Other | 28 |
| Intentional | 6 |
| Not coded | 3 |
| Label | Value |
|---|---|
| Post-deployment | 64 |
| Other | 32 |
| Pre-deployment | 27 |
| Not coded | 3 |
| Label | Value |
|---|---|
| 2012 | 3 |
| 2013 | 3 |
| 2014 | 7 |
| 2015 | 9 |
| 2016 | 16 |
| 2017 | 18 |
| 2018 | 21 |
| 2019 | 14 |
| 2020 | 31 |
| 2021 | 36 |
| 2022 | 31 |
| 2023 | 27 |
| 2024 | 32 |
| 2025 | 37 |
| 2026 | 13 |
| Label | Value |
|---|---|
| Risk Category | 44 |
| Risk Sub-Category | 82 |
Risk entries
Browse and export all- Unreliability in corner cases
"AI systems tend to show unreliable behavior when confronted with rare or ambiguous input data, also called corner cases. Therefore, the controlled behavior is required whenever the AI system is faces...
- Lack of robustness
"Robustness characterizes the resilience of an AI system’s output against minor changes in the input domain. A great variation in an AI system’s response to small input changes indicates unreliable ou...
- Operational data issues
"Until the deployment of the AI application into its operational environment, the AI system has been tested with a test set that aims to approximate the distribution of operational data. However, an u...
- Data drift
"Data drift is a phenomenon in that distribution of operational input data departs from those used during training. This can cause a degradation in performance."
- Technical
"Technical AI hazards are the root causes of technical deficiencies in the AI system. An example of such an AI hazard is overfitting, which describes a model’s excessive adaptation to the training dat...
- Procedural
"The third class encompasses procedural AI hazards. These pertain to issues arising from processes and actions made by individuals involved in the develop- ment process. Such hazards are not readily q...
- Performance & Robustness
"The AI system's ability to fulfill its intended purpose and its resilience to perturbations, and unusual or adverse inputs. Failures of performance are fundamental to the AI system's correct function...
- Performative utterances
"The chatbot makes a deal, commitment, or other consequential action with its output that the deployer did not intend."
- Bad advice/failure to generate helpful content
"The chatbot gives guidance that ranges from simply unhelpful to harmful if acted on."
- Unhelpful responses
- Bad links and references
- Nonsensical content
- Complexity of the Intended Task and Usage Environment
"As a general rule, more complex environments can quickly lead to situations that had not been considered in the design phase of the AI system. Therefore, complex environments can introduce risks with...
- System Hardware
""Faults in the hardware can violate the correct execution of any algorithm by violating its control flow. Hardware faults can also cause memory-based errors and interfere with data inputs, such as se...
- Ethics and Morality
"The content generated by the model endorses and promotes immoral and unethical behavior. When addressing issues of ethics and morality, the model must adhere to pertinent ethical principles and moral...
- Misapplication
This is the risk posed by an ideal system if used for a purpose/in a manner unintended by its creators. In many situations, negative consequences arise when the system is not used in the way or for th...
- Algorithm
"This is the risk of the ML algorithm, model architecture, optimization technique, or other aspects of the training process being unsuitable for the intended application.Since these are key decisions...
- Robustness
"This is the risk of the system failing or being unable to recover upon encountering invalid, noisy, or out-of-distribution (OOD) inputs."
- Design
"This is the risk of system failure due to system design choices or errors."
- Safety
This is the risk of direct or indirect physical or psychological injury resulting from interaction with the ML system.
- Radiological Risks
"Radiological risks involve both immediate operational hazards, such as exposure incidents or containment failures during the automated handling of radioactive materials, and broader security concerns...
- Physical (Mechanical ) Risks
"Physical (mechanical) risks are associated with robotics and automated systems, which could lead to equipment malfunctions or physical harm in laboratory settings."
- Risks from models and algorithms (Risks of robustness)
"As deep neural networks are normally non-linear and large in size, AI systems are susceptible to complex and changing operational environments or malicious interference and inductions, possibly leadi...
- Risks from data (Risks of unregulated training data annotation)
"Issues with training data annotation, such as incomplete annotation guidelines, incapable annotators, and errors in annotation, can affect the accuracy, reliability, and effectiveness of models and a...
- Real-world risks (inducing traditional economic and social security risks)
"Hallucinations and erroneous decisions of models and algorithms, along with issues such as system performance degradation, interruption, and loss of control caused by improper use or external attacks...