MIT AI Risk Repository · domain 7: AI system safety, failures, & limitations
7.4 Lack of transparency or interpretability
Challenges in understanding or explaining the decision-making processes of AI systems, which can lead to mistrust, difficulty in enforcing compliance standards or holding relevant actors accountable for harms, and the inability to identify and correct errors.
- 42
- 12
- 5
- 2
| Label | Value |
|---|---|
| AI | 21 |
| Other | 12 |
| Human | 8 |
| Not coded | 1 |
| Label | Value |
|---|---|
| Unintentional | 23 |
| Other | 18 |
| Not coded | 1 |
| Label | Value |
|---|---|
| Post-deployment | 21 |
| Other | 14 |
| Pre-deployment | 6 |
| Not coded | 1 |
| Label | Value |
|---|---|
| 2016 | 1 |
| 2017 | 1 |
| 2018 | 1 |
| 2021 | 1 |
| 2022 | 1 |
| Label | Value |
|---|---|
| Risk Category | 23 |
| Risk Sub-Category | 19 |
Risk entries
Browse and export all- Lack of transparency, explainability, and trust
"Understanding how AI reaches conclusions or why AI systems perform specific actions motivates an entire branch of interpretability research [111], but physical embodiment raises the stakes for unders...
- Accountability
An essential feature of decision-making in humans, AI, and also HLI-based agents is accountability. Implementing this feature in machines is a difficult task because many challenges should be consider...
- Transparency
an external entity of an AI-based ecosystem may want to know which parts of data affect the final decision in a learning model
- Reproducibility
How a learning model can be reproduced when it is obtained based on various sets of data and a large space of parameters. This problem becomes more challenging in data-driven learning procedures witho...
- Verifiability
In many applications of AI-based systems such as medical healthcare and military services, the lack of verification of code may not be tolerable... due to some characteristics such as the non-linear a...
- Insufficient AI development documentation
"Throughout the development of an AI system, it is vital to document every decision and action taken. This is not only essential to optimize the development process itself but also required for the au...
- Inappropriate degree of transparency to end users
"The transparency to end users of the AI system increases the user’s trust in the AI application. If not adequately integrated into the design, this might prevent the proper operation and cause potent...
- Lack of explainability
"The explainability of AI systems based on so-called black-box models is often limited. This opaqueness of AI systems can prevent developers from detecting shortcomings in the data or the model itself...
- Concept drift
"Concept drift refers to a change in the rela- tionship between input variables and model output. If not treated appropriately, concept drift can reduce the reliability of AI systems."
- Explainability & Transparency
"The feasibility of understanding and interpreting an AI system's decisions and actions, and the openness of the developer about the data used, algorithms employed, and decisions made. Lack of these e...
- Degree of Transparency and Explainability
"Transparency is the characteristic of a system that describes the degree to which appropriate information about the system is communicated to relevant stakeholders, whereas explainability describes t...
- Risks from models and algorithms (Risks of explainability)
"AI algorithms, represented by deep learning, have complex internal workings. Their black-box or grey-box inference process results in unpredictable and untraceable outputs, making it challenging to q...
- Accountability
"The ability to determine whether a decision was made in accordance with procedural and substantive standards and to hold someone responsible if those standards are not met."
- Opacity
"Stems from the mismatch between mathematical optimization in high-dimensionality characteristic of machine learning and the demands of human-scale reasoning and styles of semantic interpretation."
- Explainability
"Any action or procedure performed by a model with the intention of clarifying or detailing its internal functions."
- Complexity-induced knowledge gap
"The complexity of AI models and systems makes it challenging to demonstrate harm or establish a clear causal link between AI actions and their consequences."
- Opaque AI networks
"The complexity and opacity of AI models and systems make it difficult to predict and manage their behavior."