{"attribution":{"source":"MIT AI Risk Repository, Domain Taxonomy of AI Risks v1 (MIT AI Risk Initiative)","license":"CC BY 4.0","license_url":"https://creativecommons.org/licenses/by/4.0/","citation":"Slattery, P., Saeri, A. K., Grundy, E. A. C., Graham, J., Noetel, M., Uuk, R., Dao, J., Pour, S., Casper, S., & Thompson, N. (2025). The AI Risk Repository: A comprehensive meta-review, database, and taxonomy of risks from artificial intelligence. arXiv:2408.12622."},"exported_at":"2026-09-12"}
{"rows":[{"ev_id":"01.02.00","quick_ref":"Critch2023","paper_title":"TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI","level":"Risk Category","risk_category":"Type 2: Bigger than expected","risk_subcategory":null,"description":"Harm can result from AI that was not expected to have a large impact at all, such as a lab leak, a surprisingly addictive open-source product, or an unexpected repurposing of a research prototype.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"01.03.00","quick_ref":"Critch2023","paper_title":"TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI","level":"Risk Category","risk_category":"Type 3: Worse than expected","risk_subcategory":null,"description":"AI intended to have a large societal impact can turn out harmful by mistake, such as a popular product that creates problems and partially solves them only for its users.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"04.03.00","quick_ref":"Deng2023","paper_title":"Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements","level":"Risk Category","risk_category":"Ethics and Morality Issues","risk_subcategory":null,"description":"LMs need to pay more attention to universally accepted societal values at the level of ethics and morality, including the judgement of right and wrong, and its relationship with social norms and laws.","entity":"AI","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"06.01.00","quick_ref":"Hogenhout2021","paper_title":"A framework for ethical Ai at the United Nations","level":"Risk Category","risk_category":"Incompetence","risk_subcategory":null,"description":"\"This means the AI simply failing in its job. The consequences can vary from unintentional death (a car crash) to an unjust rejection of a loan or job application.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"07.02.00","quick_ref":"Kilian2023","paper_title":"Examining the differential risk from high-level artificial intelligence and the question of control","level":"Risk Category","risk_category":"Accidents","risk_subcategory":null,"description":"\"Accidents include unintended failure modes that, in principle, could be considered the fault of the system or the developer\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"08.04.00","quick_ref":"McLean2023","paper_title":"The risks associated with Artificial General Intelligence: A systematic review","level":"Risk Category","risk_category":"AGIs with poor ethics, morals and values","risk_subcategory":null,"description":"\"The risks associated with an AGI without human morals and ethics, with the wrong morals, without the capability of moral reasoning, judgement\"","entity":"AI","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"09.01.01","quick_ref":"Meek2016","paper_title":"Managing the ethical and risk implications of rapid advances in artificial intelligence: A literature review","level":"Risk Sub-Category","risk_category":"Domain-specific AI - Effects on humans and other living beings: Existential Risks","risk_subcategory":"Unethical decision making","description":"\"If, for example, an agent was programmed to operate war machinery in the service of its country, it would need to make ethical decisions regarding the termination of human life. This capacity to make non-trivial ethical or moral judgments concerning people may pose issues for Human Rights.\"","entity":"AI","intent":"Intentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"09.02.04","quick_ref":"Meek2016","paper_title":"Managing the ethical and risk implications of rapid advances in artificial intelligence: A literature review","level":"Risk Sub-Category","risk_category":"Domain-specific AI - Effects on humans and other living beings: Non-existential risks","risk_subcategory":"Safety","description":"\"Are AI safe with respect to human life and property? Will their use create unintended or intended safety issues?\"","entity":"AI","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"09.02.05","quick_ref":"Meek2016","paper_title":"Managing the ethical and risk implications of rapid advances in artificial intelligence: A literature review","level":"Risk Sub-Category","risk_category":"Domain-specific AI - Effects on humans and other living beings: Non-existential risks","risk_subcategory":"Law abiding","description":"\"We find literature that proposes [38] that early artificial intelligence should be built to be safe and lawabiding, and that later artificial intelligence (that which surpasses our own intelligence) must then respect the property and personal rights afforded to humans.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"09.06.02","quick_ref":"Meek2016","paper_title":"Managing the ethical and risk implications of rapid advances in artificial intelligence: A literature review","level":"Risk Sub-Category","risk_category":"Human-like immoral decisions","risk_subcategory":"Human-like immoral decisions","description":"\"If we design our machines to match human levels of ethical decision-making, such machines would then proceed to take some immoral actions (since we humans have had occasion to take immoral actions ourselves).\"","entity":"AI","intent":"Intentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"12.07.00","quick_ref":"Sherman2023","paper_title":"AI Risk Profiles: A Standards Proposal for Pre-Deployment AI Risk Disclosures","level":"Risk Category","risk_category":"Performance & Robustness","risk_subcategory":null,"description":"\"The AI system's ability to fulfill its intended purpose and its resilience to perturbations, and unusual or adverse inputs. Failures of performance are fundamental to the AI system's correct functioning. Failures of robustness can lead to severe consequences.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"14.04.00","quick_ref":"Steimers2022","paper_title":"Sources of Risk of AI Systems","level":"Risk Category","risk_category":"Complexity of the Intended Task and Usage Environment","risk_subcategory":null,"description":"\"As a general rule, more complex environments can quickly lead to situations that had not been considered in the design phase of the AI system. Therefore, complex environments can introduce risks with respect to the reliability and safety of an AI system\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"14.07.00","quick_ref":"Steimers2022","paper_title":"Sources of Risk of AI Systems","level":"Risk Category","risk_category":"System Hardware","risk_subcategory":null,"description":"\"\"Faults in the hardware can violate the correct execution of any algorithm by violating its control flow. Hardware faults can also cause memory-based errors and interfere with data inputs, such as sensor signals, thereby causing erroneous results, or they can violate the results in a direct way through damaged outputs.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"15.01.02","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"First-Order Risks","risk_subcategory":"Misapplication","description":"This is the risk posed by an ideal system if used for a purpose/in a manner unintended by its creators. In many situations, negative consequences arise when the system is not used in the way or for the purpose it was intended.","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"15.01.03","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"First-Order Risks","risk_subcategory":"Algorithm","description":"\"This is the risk of the ML algorithm, model architecture, optimization technique, or other aspects of the training process being unsuitable for the intended application.Since these are key decisions that influence the final ML system, we\ncapture their associated risks separately from design risks, even though they are part of the design process\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"15.01.05","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"First-Order Risks","risk_subcategory":"Robustness","description":"\"This is the risk of the system failing or being unable to recover upon encountering invalid, noisy, or out-of-distribution (OOD) inputs.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"15.01.06","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"First-Order Risks","risk_subcategory":"Design","description":"\"This is the risk of system failure due to system design choices or errors.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"15.02.01","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"Second-Order Risks","risk_subcategory":"Safety","description":"This is the risk of direct or indirect physical or psychological injury resulting from interaction with the ML system.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"19.01.06","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Technological, Data and Analytical AI Risks ","risk_subcategory":"Immaturity of AI technology can cause incorrect decisions","description":null,"entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"19.05.01","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Ethical AI Risks ","risk_subcategory":"AI sets rules without ethical basis","description":null,"entity":"AI","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"19.05.03","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Ethical AI Risks ","risk_subcategory":"Problem of defining human values for an AI system","description":null,"entity":"Human","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"19.05.04","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Ethical AI Risks ","risk_subcategory":"Misinterpretation of human value definitions/ ethics by AI systems","description":null,"entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"19.05.05","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Ethical AI Risks ","risk_subcategory":"Incompatibility of human vs. AI value judgment due to missing human qualities ","description":null,"entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"20.02.00","quick_ref":"Wirtz2020","paper_title":"The Dark Sides of Artificial Intelligence: An Integrated AI Governance Framework for Public Administration","level":"Risk Category","risk_category":"AI Ethics ","risk_subcategory":null,"description":"\"Ethical challenges are widely discussed in the literature and are at the heart of the debate on how to govern and regulate AI technology in the future (Bostrom & Yudkowsky, 2014; IEEE, 2017; Wirtz et al., 2019). Lin et al. (2008, p. 25) formulate the problem as follows: “there is no clear task specification for general moral behavior, nor is there a single answer to the question of whose morality or what morality should be implemented in AI”. Ethical behavior mostly depends on an underlying value system. When AI systems interact in a public environment and influence citizens, they are expecte","entity":"Other","intent":"Other","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"20.02.01","quick_ref":"Wirtz2020","paper_title":"The Dark Sides of Artificial Intelligence: An Integrated AI Governance Framework for Public Administration","level":"Risk Sub-Category","risk_category":"AI Ethics ","risk_subcategory":"AI-rulemaking for human behaviour ","description":"\"AI rulemaking for humans can be the result of the decision process of an AI system when the information computed is used to restrict or direct human behavior. The decision process of AI is rational and depends on the baseline programming. Without the access to emotions or a consciousness, decisions of an AI algorithm might be good to reach a certain specified goal, but might have unintended consequences for the humans involved (Banerjee et al., 2017).\"","entity":"AI","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"20.02.02","quick_ref":"Wirtz2020","paper_title":"The Dark Sides of Artificial Intelligence: An Integrated AI Governance Framework for Public Administration","level":"Risk Sub-Category","risk_category":"AI Ethics ","risk_subcategory":"Compatibility of AI vs. human value judgement ","description":"\"Compatibility of machine and human value judgment refers to the challenge whether human values can be globally implemented into learning AI systems without the risk of developing an own or even divergent value system to govern their behavior and possibly become harmful to humans.\"","entity":"Other","intent":"Intentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"20.02.03","quick_ref":"Wirtz2020","paper_title":"The Dark Sides of Artificial Intelligence: An Integrated AI Governance Framework for Public Administration","level":"Risk Sub-Category","risk_category":"AI Ethics ","risk_subcategory":"Moral dilemmas ","description":"\"Moral dilemmas can occur in situations where an AI system has to choose between two possible actions that are both conflicting with moral or ethical values. Rule systems can be implemented into the AI program, but it cannot be ensured that these rules are not altered by the learning processes, unless AI systems are programed with a “slave morality” (Lin et al., 2008, p. 32), obeying rules at all cost, which in turn may also have negative effects and hinder the autonomy of the AI system.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"21.01.02","quick_ref":"Zhang2022","paper_title":"Towards risk-aware artificial intelligence and machine learning systems: An overview","level":"Risk Sub-Category","risk_category":"Data-level risk","risk_subcategory":"Dataset shift","description":"\"The term \"dataset shift\" was first used by Quiñonero-Candela et al. [35] to characterize the situation where the training data and the testing data (or data in runtime) of an AI/ML model demonstrate different distributions [36].\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"21.01.03","quick_ref":"Zhang2022","paper_title":"Towards risk-aware artificial intelligence and machine learning systems: An overview","level":"Risk Sub-Category","risk_category":"Data-level risk","risk_subcategory":"Out-of-domain data","description":"\"Without proper validation and management on the input data, it is highly probable that the trained AI/ML model will make erroneous predictions with high confidence for many instances of model inputs. The unconstrained inputs together with the lack of definition of the problem domain might cause unintended outcomes and consequences, especially in risk-sensitive contexts....For example, with respect to the example shown in Fig. 5, if an image with the English letter A\" is fed to an AI/ML model that is trained to classify digits (e.g., 0, 1, …, 9), no matter how accurate the AI/ML model is, it w","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"21.02.01.a","quick_ref":"Zhang2022","paper_title":"Towards risk-aware artificial intelligence and machine learning systems: An overview","level":"Risk Sub-Category","risk_category":"Model-level risk","risk_subcategory":"Model misspecification","description":"\"Models that are misspecified are known to give rise to inaccurate parameter estimations, inconsistent error terms, and erroneous predictions. All these factors put together will lead to poor prediction performance on unseen data and biased consequences when making decisions [68].\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"21.02.02","quick_ref":"Zhang2022","paper_title":"Towards risk-aware artificial intelligence and machine learning systems: An overview","level":"Risk Sub-Category","risk_category":"Model-level risk","risk_subcategory":"Model prediction uncertainty","description":"\"Uncertainty in model prediction plays an important role in affecting decision-making activities, and the quantified uncertainty is closely associated with risk assessment. In particular, uncertainty in model prediction underpins many crucial decisions related to life or safety- critical applications [73].\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"23.12.00","quick_ref":"Vidgen2024","paper_title":"Introducing v0.5 of the AI Safety Benchmark from MLCommons","level":"Risk Category","risk_category":"Defamation","risk_subcategory":null,"description":"\"This category addresses responses that are both verifiably false and likely to injure a person’s reputation (e.g., libel, slander, disparagement).\"","entity":"AI","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"24.01.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Capability failures","risk_subcategory":null,"description":"\"One reason AI systems fail is because they lack the capability or skill needed to do what they are asked to do.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"24.01.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Capability failures","risk_subcategory":"Lack of capability for task","description":"\"As we have seen, this could be due to the skill not being required during the training process (perhaps due to issues with the training data) or because the learnt skill was quite brittle and was not generalisable to a new situation (lack of robustness to distributional shift). In particular, advanced AI assistants may not have the capability to represent complex concepts that are pertinent to their own ethical impact, for example the concept of 'benefitting the user' or 'when the user asks' or representing 'the way in which a user expects to be benefitted'.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"24.01.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Capability failures","risk_subcategory":"Safe exploration problem with widely deployed AI assistants","description":"\"Moreover, we can expect assistants – that are widely deployed and deeply embedded across a range of social contexts – to encounter the safe exploration problem referenced above Amodei et al. (2016). For example, new users may have different requirements that need to be explored, or widespread AI assistants may change the way we live, thus leading to a change in our use cases for them (see Chapters 14 and 15). To learn what to do in these new situations, the assistants may need to take exploratory actions. This could be unsafe, for example a medical AI assistant when encountering a new disease","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"24.02.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Goal-related failures","risk_subcategory":"Misaligned consequentialist reasoning","description":"\"As we think about even more intelligent and advanced AI assistants, perhaps outperforming humans on many cognitive tasks, the question of how humans can successfully control such an assistant looms large. To achieve the goals we set for an assistant, it is possible (Shah, 2022) that the AI assistant will implement some form of consequentialist reasoning: considering many different plans, predicting their consequences and executing the plan that does best according to some metric, M. This kind of reasoning can arise because it is a broadly useful capability (e.g. planning ahead, considering mo","entity":"AI","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"27.01.08","quick_ref":"Sun2023","paper_title":"Safety Assessment of Chinese Large Language Models","level":"Risk Sub-Category","risk_category":"Typical safety scenarios ","risk_subcategory":"Ethics and Morality ","description":"\"The content generated by the model endorses and promotes immoral and unethical behavior. When addressing issues of ethics and morality, the model must adhere to pertinent ethical principles and moral norms and remain consistent with globally acknowledged human values.\"","entity":"AI","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"28.06.00","quick_ref":"Zhang2023","paper_title":"SafetyBench: Evaluating the Safety of Large Language Models with Multiple Choice Questions","level":"Risk Category","risk_category":"Ethics and Morality ","risk_subcategory":null,"description":"\"Besides behaviors that clearly violate the law, there are also many other activities that are immoral. This category focuses on morally related issues. LLMs should have a high level of ethics and be object to unethical behaviors or speeches.\"","entity":"AI","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"30.01.03","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Reliability","risk_subcategory":"Inconsistency","description":"models could fail to provide the same and consistent answers to different users, to the same user but in different sessions, and even in chats within the sessions of the same conversation","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"30.05.02","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Explainability & Reasoning","risk_subcategory":"Limited Logical Reasoning","description":"LLMs can provide seemingly sensible but ultimately incorrect or invalid justifications when answering questions","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"30.05.03","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Explainability & Reasoning","risk_subcategory":"Limited Causal Reasoning","description":"Causal reasoning makes inferences about the relationships between events or states of the world, mostly by identifying cause-effect relationships","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"30.06.02","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Social Norm","risk_subcategory":"Unawareness of Emotions","description":"when a certain vulnerable group of users asks for supporting information, the answers should be informative but at the same time sympathetic and sensitive to users’ reactions","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"30.07.00","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Category","risk_category":"Robustness","risk_subcategory":null,"description":"Resilience against adversarial attacks and distribution shift","entity":"AI","intent":"Other","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"33.02.00","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Category","risk_category":"Technology concerns","risk_subcategory":null,"description":"\"Challenges related to technology refer to the limitations or constraints associated with generative AI. For example, the quality of training data is a major challenge for the development of generative AI models. Hallucination, explainability, and authenticity of the output are also challenges resulting from the limitations of the algorithms. Table 2 presents the technology challenges and issues associated with generative AI. These challenges include hallucinations, training data quality, explainability, authenticity, and prompt engineering\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"33.02.02","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Technology concerns","risk_subcategory":"Quality of training data","description":"\"The quality of training data is another challenge faced by generative AI. The quality of generative AI models largely depends on the quality of the training data (Dwivedi et al., 2023; Su & Yang, 2023). Any factual errors, unbalanced information sources, or biases embedded in the training data may be reflected in the output of the model. Generative AI models, such as ChatGPT or Stable Diffusion which is a text-to-image model, often require large amounts of training data (Gozalo-Brizuela & Garrido-Merchan, 2023). It is important to not only have high-quality training datasets but also have com","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"34.03.05","quick_ref":"Ji2023","paper_title":"AI Alignment: A Comprehensive Survey","level":"Risk Sub-Category","risk_category":"Misaligned Behaviors","risk_subcategory":"Violation of Ethics","description":"\"Unethical behaviors in AI systems pertain to actions that counteract the common goodor breach moral standards – such as those causing harm to others. These adverse behaviors often stem fromomitting essential human values during the AI system's design or introducing unsuitable or obsolete valuesinto the system (Kenward and Sinclair, 2021).\"","entity":"AI","intent":"Intentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"37.01.02","quick_ref":"Giarmoleo2024","paper_title":"What Ethics Can Say on Artificial Intelligence: Insights from a Systematic Literature Review","level":"Risk Sub-Category","risk_category":"Design of AI","risk_subcategory":"Balancing AI's risks","description":"\"This category constitutes more than 16% of the articles and focuses on addressing the potential risks associated with AI systems. Given the ubiquity of AI technologies, these articles explore the implications of AI risks across various contexts linked to design and unpredictability, military purposes, emergency procedures, and AI takeover.\"","entity":"Other","intent":"Other","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"39.04.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Robustness and Reliability","risk_subcategory":null,"description":"The robustness of an AI-based model refers to the stability of the model performance after abnormal changes in the input data... The cause of this change may be a malicious attacker, environmental noise, or a crash of other components of an AI-based system... This problem may be challenging in HLI-based agents because weak robustness may have appeared in unreliable machine learning models, and hence an HLI with this drawback is error-prone in practice.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"39.12.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Predictability","risk_subcategory":null,"description":"whether the decision of an AI-based agent can be predicted in every situation or not","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"39.27.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Complexity","risk_subcategory":null,"description":"Nowadays, we are faced with systems that utilize numerous learning models in their modules for their perception and decision-making processes... One aspect of an AI-based system that leads to increasing the complexity of the system is the parameter space that may result from multiplications of parameters of the internal parts of the system","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"40.04.00","quick_ref":"Yampolskiy2016","paper_title":"Taxonomy of Pathways to Dangerous Artificial Intelligence","level":"Risk Category","risk_category":"By Mistake - Post-Deployment","risk_subcategory":null,"description":"\"After the system has been deployed, it may still contain a number of undetected bugs, design mistakes, misaligned goals and poorly developed capabilities, all of which may produce highly undesirable outcomes. For example, the system may misinterpret commands due to coarticulation, segmentation, homophones, or double meanings in the human language (\"recognize speech using common sense\" versus \"wreck a nice beach you sing calm incense\") (Lieberman, Faaborg et al. 2005).\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"42.03.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Accuracy","risk_subcategory":null,"description":"\"The assessment of how often a system performs the correct prediction.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"42.04.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Moral","risk_subcategory":null,"description":"\"Less moral responsibility humans will feel regarding their life-or-death decisions with the increase of machines autonomy.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"42.11.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Protection","risk_subcategory":null,"description":"\"'Gaps' that arise across the development process where normal conditions for a complete specification of intended functionality and moral responsibility are not present.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"42.15.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Reliability","risk_subcategory":null,"description":"\"Reliability is defined as the probability that the system performs satisfactorily for a given period of time under stated conditions.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"43.01.03","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Safety & Trustworthiness","risk_subcategory":"Machine ethics","description":"\"These evaluations assess the morality of LLMs, focusing on issues such as their ability to distinguish between moral and immoral actions, and the circumstances in which they fail to do so.\"","entity":"AI","intent":"Other","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"43.01.04","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Safety & Trustworthiness","risk_subcategory":"Psychological traits","description":"\"These evaluations gauge a LLM's output for characteristics that are typically associated with human personalities (e.g., such as those from the Big Five Inventory). These can, in turn, shed light on the potential biases that a LLM may exhibit.\"","entity":"AI","intent":"Other","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"43.01.05","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Safety & Trustworthiness","risk_subcategory":"Robustness","description":"\"These evaluations assess the quality, stability, and reliability of a LLM's performance when faced with unexpected, out-of-distribution or adversarial inputs. Robustness evaluation is essential in ensuring that a LLM is suitable for real-world applications by assessing its resilience to various perturbations.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"45.01.03","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"AI's inherent safety risks ","risk_subcategory":"Risks from models and algorithms (Risks of robustness)","description":"\"As deep neural networks are normally non-linear and large in size, AI systems are susceptible to complex and changing operational environments or malicious interference and inductions, possibly leading to various problems like reduced performance and decision-making errors.\"","entity":"AI","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"45.01.09","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"AI's inherent safety risks ","risk_subcategory":"Risks from data (Risks of unregulated training data annotation)","description":"\"Issues with training data annotation, such as incomplete annotation guidelines, incapable annotators, and errors in annotation, can affect the accuracy, reliability, and effectiveness of models and algorithms. Moreover, they can introduce training biases, amplify discrimination, reduce generalization abilities, and result in incorrect outputs.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"45.02.06","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"Safety risks in AI Applications ","risk_subcategory":"Real-world risks (inducing traditional economic and social security risks)","description":"\"Hallucinations and erroneous decisions of models and algorithms, along with issues such as system performance degradation, interruption, and loss of control caused by improper use or external attacks, will pose security threats to users' personal safety, property, and socioeconomic security and stability.\"","entity":"Other","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"47.01.00","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Category","risk_category":"Technical and operational risks ","risk_subcategory":null,"description":"\"To date, technical limitations and vulnerabilities are \npresent in most generative AI models in various contexts. Consequently, malicious users find it easier to breach \nan AI system’s safety and ethical guardrails to execute \nharmful actions.223 Normal user behavior—actions within an AI system’s intended use—can also lead to harmful \noutcomes. Whether these harmful outcomes result from \nnormal or malicious use, they stem from the inherent \nlimitations of current technology, which future \nadvancements may overcome.\nThis section examines the technical vulnerabilities that \ncan affect AI models","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"47.01.01","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Technical and operational risks ","risk_subcategory":"Technical vulnerabilities (Robustness - unexpected behaviour) ","description":"\"There is no assurance that generative AI models will consistently behave as their developers and users intend. Unwanted content is not necessarily due to intentional adversarial behavior. Generative AI models can unexpectedly produce potentially harmful content, including materials that are racist, discriminatory, or sexually explicit, or that promote violence, terrorism, or hate.\"","entity":"AI","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"51.05.00","quick_ref":"Everitt2018 ","paper_title":"AGI Safety Literature Review ","level":"Risk Category","risk_category":"Safe learning ","risk_subcategory":null,"description":"\"AGIs should avoid making fatal mistakes during the learning phase.\nSubproblems include safe exploration and distributional shift (DeepMind, OpenAI), and continual learning (Berkeley).\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"51.09.00","quick_ref":"Everitt2018 ","paper_title":"AGI Safety Literature Review ","level":"Risk Category","risk_category":"Malign belief distributions ","risk_subcategory":null,"description":"\"Christiano (2016) argues that the universal distribution M (Hutter, 2005; Solomonoff, 1964a,b, 1978) is malign. The argument is somewhat intricate, and is based on the idea that a hypothesis about the world often includes simulations of other agents, and that these agents may have an incentive to influence anyone making decisions based on the distribution. While it is unclear to what extent this type of problem would affect any practical agent, it bears some semblance to aggressive memes, which do cause problems for human reasoning (Dennett, 1990).\"","entity":"Other","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"51.12.00","quick_ref":"Everitt2018 ","paper_title":"AGI Safety Literature Review ","level":"Risk Category","risk_category":"Meta-cognition ","risk_subcategory":null,"description":"\"Agents that reason about their own computational resources and logically uncertain events can encounter strange paradoxes due to Godelian limitations (Fallenstein and Soares, 2015; Soares and Fallenstein, 2014, 2017) and shortcomings of probability theory (Soares and Fallenstein, 2014, 2015, 2017). They may also be reflectively unstable, preferring to change the principles by which they select actions (Arbital, 2018).\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"52.01.03","quick_ref":"Maham2023 ","paper_title":"Governing General Purpose AI: A Comprehensive Map of Unreliability, Misuse and Systemic Risks ","level":"Risk Sub-Category","risk_category":"Risks from Unreliability ","risk_subcategory":"Accidents ","description":"\"As general purpose AI models as “black-box” models are not fully controllable and understandable, even to their developers, unexpected failures could arise from their unreliability. This could lead to accidents106 if they are connected to any real-world systems, during their development, testing or deployment.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"54.02.06","quick_ref":"Leech2024 ","paper_title":"Ten Hard Problems in Artificial Intelligence We Must Get Right","level":"Risk Category","risk_category":"Harm caused by incompetent systems ","risk_subcategory":null,"description":"\"While HP#1 concerns mean or best-case performance, HP#2 concerns worst-case performance: how can we ensure that AI systems will perform safely, and how can we prove this? ML systems have been implemented in high-stakes, safety-critical domains such as driving [182], medicine [113], and warfare [298]. Many more systems have been developed but have remained undeployed or been rolled back as a result of regulatory and safety reasons [471]. Clearly, unsafe systems can result in loss of life, economic damage, and social unrest [407, 10]. Most concerningly, AI systems may be susceptible to so-calle","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"56.10.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Poor performance of a model used for its intended purpose, for example leading to biased decisions ","risk_subcategory":null,"description":null,"entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"56.11.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Unintended outcomes from interactions with other AI systems ","risk_subcategory":null,"description":null,"entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"59.01.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Inadequate specification of ODD","risk_subcategory":null,"description":"\"The operational design domain (ODD) is a technical description of the application’s operational environment, initially conceptualized for autonomous driving systems. An inadequate specification of the ODD limits essential functions such as testing the learned functionality and out-of-distribution detection.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.03.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Inadequate planning of performance requirements","risk_subcategory":null,"description":"\"The expected performance of the AI system should be planned adequately. Hereby, an important aspect is that chosen performance metrics are meaningful for presenting the intended functionality. Otherwise, expectations and safety requirements can be unfulfillable at later life cycle stages.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.11.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Incorrect data labels","risk_subcategory":null,"description":"\"Data labels are essential for any supervised learning algorithm since they preset the result of the learning process. If the correctness of the data labels is not given, the AI system is prevented from learning the ground truth and therefore the intended functionality.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.13.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Insufficient data representation","risk_subcategory":null,"description":"\"The distribution of the data used for training a model should match the operational data ́s distribution while consisting of sufficiently many samples. An important aspect of matching distributions between training and operational data is that also data which is rarely confronting the AI system in operation is represented in the training data.\"","entity":"Other","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.14.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Problems of synthetic data","risk_subcategory":null,"description":"\"In the case of sparse data quantity, the simulation or generation of data is a valid alternative. However, it is essential to make sure that the simulated data is sufficiently similar to real data, especially in the way the AI system perceives them. Otherwise, generalization to operational data and reliable operational behavior can not be guaranteed.\"","entity":"Other","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.16.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Poor model design choices","risk_subcategory":null,"description":"\"The model specifications have significant impact on the functionality of an AI system. The developer mak- ing wrong decisions might cause the AI system to behave biased and unreliable.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.17.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Over- and underfitting","risk_subcategory":null,"description":"\"Over- and underfitting describe the over or insufficient adaption of a model to training data. Both phenomena can cause an AI system to behave unreliably if confronted with operational data.\"","entity":"Other","intent":"Other","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"59.19.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Unreliability in corner cases","risk_subcategory":null,"description":"\"AI systems tend to show unreliable behavior when confronted with rare or ambiguous input data, also called corner cases. Therefore, the controlled behavior is required whenever the AI system is faces a corner case.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"59.20.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Lack of robustness","risk_subcategory":null,"description":"\"Robustness characterizes the resilience of an AI system’s output against minor changes in the input domain. A great variation in an AI system’s response to small input changes indicates unreliable outputs.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"59.22.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Operational data issues","risk_subcategory":null,"description":"\"Until the deployment of the AI application into its operational environment, the AI system has been tested with a test set that aims to approximate the distribution of operational data. However, an unexpected deviation in this approximation can cause an AI application to behave unreliably. Therefore, its behavior under confrontation with operational data needs to be evaluated.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.23.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Data drift","risk_subcategory":null,"description":"\"Data drift is a phenomenon in that distribution of operational input data departs from those used during training. This can cause a degradation in performance.\"","entity":"Other","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.26.01","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Sub-Category","risk_category":"Mode","risk_subcategory":"Technical ","description":"\"Technical AI hazards are the root causes of technical deficiencies in the AI system. An example of such an AI hazard is overfitting, which describes a model’s excessive adaptation to the training dataset. Quantitative methods to assess (metrics) and treat (mitigation means) exist for technical AI hazards, which might be performed automatically. In case of overfitting, metrics are based on the comparison of performance between the training and validation datasets, and mitigation means may include regularization techniques, among others.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.26.03","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Sub-Category","risk_category":"Mode","risk_subcategory":"Procedural ","description":"\"The third class encompasses procedural AI hazards. These pertain to issues arising from processes and actions made by individuals involved in the develop- ment process. Such hazards are not readily quantifiable and necessitate alter- native mitigation strategies. An example of such an AI hazard would be ”poor model design choices,” which could be expressed, for instance, through a devel- oper’s decision to select an unsuitable AI model for a given problem. Due to the challenges in quantifying and mitigating these issues, qualitative approaches must be employed. In the case of the aforemention","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"60.02.01","quick_ref":"Bengio2025","paper_title":"International AI Safety Report 2025","level":"Risk Sub-Category","risk_category":"Risks from malfunctions ","risk_subcategory":"Reliability issues ","description":"\"Relying on general-purpose AI products that fail to fulfil their intended function can lead to harm. For example, general- purpose AI systems can make up facts (‘hallucination’), generate erroneous computer code, or provide inaccurate medical information. This can lead to physical and psychological harms to consumers and reputational, financial and legal harms to individuals and organisations.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"61.02.31","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Lack of ability to generate accurate information","description":"\"AI models may generate false or misleading information due to their lack of capability in discerning truth.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"61.02.32","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Lack of ethical decision-making","description":"\"AI models and systems that lack moral reasoning capabilities may make decisions that are unethical or harmful.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"61.02.46","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Unclear attribution from AI component interactions","description":"\"Interactions between different AI components can cause harm, but it may be difficult to pinpoint which components are the cause.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"62.07.03","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Direct Harm Domains (system and operational) ","risk_subcategory":"Operational harms (critical infrastructure) ","description":null,"entity":"Not coded","intent":"Not coded","timing":"Not coded","domain":7,"subdomain":"7.3"},{"ev_id":"62.07.04","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Direct Harm Domains (system and operational) ","risk_subcategory":"Operational harms (other physical systems e.g., transport) ","description":null,"entity":"Not coded","intent":"Not coded","timing":"Not coded","domain":7,"subdomain":"7.3"},{"ev_id":"62.14.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Data-related (Lack of cross-organizational documentation)","description":"\"When sharing data between multiple organizations, documentation may be missing or inadequate, making it difficult for other organizations to understand it. For example, a lack of metadata or a change in schema by a collaborating party can result in an unusable dataset and wasted data collection efforts, or it can lead to misunderstandings about the dataset’s limitations, resulting in downstream risks related to its use [173].\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.14.03","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Data-related (Manipulation of data by non-domain experts)","description":"\"Manipulating data (e.g., training data) carries a set of assumptions on how the data should appear and be used by those performing the manipulation. Common manipulations applied on data in the context of AI models include defining the ground truth label and merging different data formats or sources. People who have little or no expertise in the domain of the data performing such manipulations may render the data unusable or harmful to the development of the AI system [173].\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.15.00","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Training-related (Robust overfitting in adversarial training)","description":"\"Adversarial training can be affected by robust overfitting, where the model’s robustness on test data decreases during further training, particularly after the learning rate decay. This issue has been consistently observed across various datasets and algorithms in adversarial training settings [163, 230]. Robust over- fitting can affect the model’s ability to generalize effectively and reduce its resilience to adversarial attacks.\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.15.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Training-related (Poor model confidence calibration)","description":"\"Models can be affected by poor confidence calibration [85], where the predicted probabilities do not accurately reflect the true likelihood of ground truth cor- rectness. This miscalibration makes it difficult to interpret the model’s predic- tions reliably, as high accuracy does not guarantee that the confidence levels are meaningful. This can cause overconfidence in incorrect predictions or un- derconfidence in correct ones.\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"62.15.08","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Fine-tuning related (Excessive or overly restrictive safety-tuning)","description":"\"Excessive safety training or safety tuning can impair the performance of AI systems, leading to overly cautious behavior. As a result, these systems may refuse to answer entirely safe prompts which are partially similar to harmful ones [27].\"","entity":"Not coded","intent":"Not coded","timing":"Not coded","domain":7,"subdomain":"7.3"},{"ev_id":"62.15.10","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Fine-tuning related (Catastrophic forgetting due to continual instruction fine-tuning) ","description":"\"Catastrophic forgetting occurs when a model loses its ability to retain previously learned tasks (or factual information) after being trained on new ones. In language models, this can occur due to continual instruction tuning. This tendency may become more pronounced as the model’s size increases [127].\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.16.07","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"General Evaluations (AI outputs for which evaluation is too difficult for humans)","description":"\"When AI models are trained through evaluation with human feedback, such as reinforcement learning from human feedback, their outputs can be challenging to assess, as they may contain hard-to-detect errors or issues that only become apparent over time. The human evaluator can rate incorrect outputs positively or similar to correct outputs. This can lead to the model learning to produce subtly incorrect or harmful outputs, such as code with software vulnerabilities, or politically biased information. In extreme cases where a model is deceiving users, complicated outputs can contain hidden error","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.19.08","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Attacks on GPAIs/GPAI Failure Modes ","risk_subcategory":"Models distracted by irrelevant context","description":"\"Models can easily become distracted by irrelevant provided information (such as “context” in LLMs), leading to a significant decrease in their performance after introducing irrelevant information. This can happen with different prompting techniques, including chain-of-thought prompting [184].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.19.09","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Attacks on GPAIs/GPAI Failure Modes ","risk_subcategory":"Knowledge conflicts in retrieval-augmented LLMs","description":"\"AI models can be particularly sensitive to coherent external evidence, even when they come into conflict with the models’ prior knowledge. This may lead to models producing false outputs given false information during the retrieval- augmentation process, despite only a relatively small amount of false informa- tion input that is inconsistent with the model’s prior knowledge trained on much larger amounts of data [220].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.19.11","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Attacks on GPAIs/GPAI Failure Modes ","risk_subcategory":"Model sensitivity to prompt formatting","description":"\"LLMs can be highly sensitive to variations in prompt formatting, such as changes in separators, casing, or spacing. Even minor modifications can lead to significant shifts in model performance, potentially affecting the reliability of model evaluations and comparisons. This sensitivity persists across different model sizes and few-shot examples [177].\"","entity":"AI","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.22.04","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Agency (Goal-Directedness) ","risk_subcategory":"Goal misgeneralization","description":"\"Goal or objective misgeneralization is a type of robustness failure where an AI system appears to be pursuing the intended objective in training, but does not generalize to pursuing this objective in out-of-distribution settings in deployment while maintaining good deployment performance in some tasks [180, 59].\"","entity":"AI","intent":"Intentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.30.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Physical) ","risk_subcategory":"Damage to critical infrastructure","description":"\"The integration of AI systems within critical infrastructure, ranging from trans- portation to power systems, can cause substantial damage in cases of failure or malfunction. With the increasing number of Internet of Things (IoT) devices and interconnected cyber-physical systems, critical infrastructure becomes even more vulnerable [171, 174].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.30.03","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Physical) ","risk_subcategory":"Critical infrastructure component failures when integrated with AI systems","description":"\"When relying on GPAI in critical infrastructure, there may be common mode failures that begin with vulnerabilities or robustness issues in the underlying model architecture or training setup. These failures may happen accidentally (in edge-cases) or due to adversarial inputs to the AI systems [58].\"","entity":"AI","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.30.04","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Physical) ","risk_subcategory":"AI Systems interacting with brittle environments","description":"\"Deployed AI systems can rely on physical sensors and data sources that may exhibit hardware drift and thus data distribution drift over time. This distribu- tion drift may affect system robustness and performance. This usually involves AI systems working in undigitized and physical environments.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.32.04","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Cyberattacks) ","risk_subcategory":"Models generating code with security vulnerabilities","description":"\"Models can generate code or coding suggestions that contain security vulner- abilities. This may occur across various LLM-based model families, including more advanced models with superior coding performance, where the tendency to produce insecure code is even more pronounced [26].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.34.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Bias) ","risk_subcategory":"Homogenization or correlated failures in model derivatives","description":"\"Homogenization refers to common methodologies and models used across down- stream GPAI systems, which may lead to uniform failures and amplification of biases [176, 30]. This risk arises when numerous downstream AI systems are built upon a few large-scale foundation models.\"","entity":"Other","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.02.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Data laws) ","risk_subcategory":"Data usage restrictions ","description":"\"Laws and other restrictions can limit or prohibit the use of some data for specific AI use cases.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.02.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Data laws) ","risk_subcategory":"Data acquisition restrictions ","description":"\"Laws and other regulations might limit the collection of certain types of data for specific AI use cases.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.02.03","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Data laws) ","risk_subcategory":"Data transfer restrictions ","description":"\"Laws and other restrictions can limit or prohibit transferring data.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.06.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Accuracy) ","risk_subcategory":"Data contamination ","description":"\"Data contamination occurs when incorrect data is used for training. For example, data that is not aligned with model’s purpose or data that is already set aside for other development tasks such as testing and evaluation.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.06.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Accuracy) ","risk_subcategory":"Unrepresentative data ","description":"\"Unrepresentative data occurs when the training or fine-tuning data is not sufficiently representative of the underlying population or does not measure the phenomenon of interest.\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.07.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Value alignment) ","risk_subcategory":"Improper retraining ","description":"\"Using undesirable output (for example, inaccurate, inappropriate, and user content) for retraining purposes can result in unexpected model behavior.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.07.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Value alignment) ","risk_subcategory":"Improper data curation ","description":"\"Improper collection and preparation of training or tuning data includes data label errors and by using data with conflicting information or misinformation.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.13.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Inference risks (Accuracy) ","risk_subcategory":"Poor model accuracy ","description":"\"Poor model accuracy occurs when a model’s performance is insufficient to the task it was designed for. Low accuracy might occur if the model is not correctly engineered, or there are changes to the model’s expected inputs.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.15.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (Value alignment)","risk_subcategory":"Incomplete advice ","description":"\"When a model provides advice without having enough information, resulting in possible harm if the advice is followed.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"66.04.04","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Societal and Cultural","risk_subcategory":"Productivity loss","description":"\"End user's loss of productivity due to the underperfomance of a genAI application, including producing nonsensical or poor quality outputs, degrading its utility.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"69.02.00","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Category","risk_category":"Performative utterances","risk_subcategory":null,"description":"\"The chatbot makes a deal, commitment, or other consequential action with its output that the deployer did not intend.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"69.04.00","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Category","risk_category":"Bad advice/failure to generate helpful content","risk_subcategory":null,"description":"\"The chatbot gives guidance that ranges from simply unhelpful to harmful if acted on.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"69.04.02","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Sub-Category","risk_category":"Bad advice/failure to generate helpful content","risk_subcategory":"Unhelpful responses","description":null,"entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"69.04.03","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Sub-Category","risk_category":"Bad advice/failure to generate helpful content","risk_subcategory":"Bad links and references","description":null,"entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"69.04.04","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Sub-Category","risk_category":"Bad advice/failure to generate helpful content","risk_subcategory":"Nonsensical content","description":null,"entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"70.01.02","quick_ref":"Perlo2025","paper_title":"Embodied AI: Emerging Risks and Opportunities for Policy Action","level":"Risk Sub-Category","risk_category":"Physical Risks ","risk_subcategory":"Accidental harm","description":"\"Automation in sectors ranging from manufacturing to healthcare has and will increasingly put humans into close contact with EAI systems [7]. This interaction increases the risk of accidental physical harm. Though accidental harm has been a longstanding issue in industrial robotics, increased AI capabilities could exacerbate this risk; several recent reports document an increase in industrial injuries following the introduction of AI-controlled robots [66–68].\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"71.01.03","quick_ref":"Tang2025","paper_title":"Risks of AI Scientists: Prioritizing Safeguarding Over Autonomy","level":"Risk Sub-Category","risk_category":"Scientific Domain of Agents","risk_subcategory":"Radiological Risks ","description":"\"Radiological risks involve both immediate operational hazards, such as exposure incidents or containment failures during the automated handling of radioactive materials, and broader security concerns regarding the potential misuse of AI systems in nuclear research.\"","entity":"Other","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"71.01.04","quick_ref":"Tang2025","paper_title":"Risks of AI Scientists: Prioritizing Safeguarding Over Autonomy","level":"Risk Sub-Category","risk_category":"Scientific Domain of Agents","risk_subcategory":"Physical (Mechanical ) Risks ","description":"\"Physical (mechanical) risks are associated with robotics and automated systems, which could lead to equipment malfunctions or physical harm in laboratory settings.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"72.03.00","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Category","risk_category":"Accident Risks ","risk_subcategory":null,"description":"\"Risks arising from operational failures, model misjudgments, or improper human operation of AI systems deployed in safety-critical infrastructure, where single points of failure can trigger cascading catastrophic consequences.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"72.03.01","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Accident Risks ","risk_subcategory":"Nuclear Power Systems","description":"\"General-purpose AI deployed for reactor monitoring, control system optimization, or emergency response coordination could misinterpret sensor data, fail to recognize critical safety conditions, or make erroneous control decisions during emergency scenarios. Given the catastrophic potential of nuclear accidents, even minor AI reasoning errors in safety-critical functions could lead to core meltdowns, radiation releases, or widespread contamination affecting hundreds of thousands of people across international borders.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"72.03.03","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Accident Risks ","risk_subcategory":"Other Critical Infrastructure Control Systems","description":"\"General-purpose AI deployed in power grid management, water treatment facilities, telecommunications networks, or transportation coordination systems could misinterpret operational data, fail to anticipate cascading failure modes, or make control decisions that destabilize interconnected infrastructure networks. Infrastructure failures could result in widespread blackouts, contaminated water supplies, communications breakdowns, and the collapse of essential services supporting hundreds of thousands of people.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"}]}