{"attribution":{"source":"MIT AI Risk Repository, Domain Taxonomy of AI Risks v1 (MIT AI Risk Initiative)","license":"CC BY 4.0","license_url":"https://creativecommons.org/licenses/by/4.0/","citation":"Slattery, P., Saeri, A. K., Grundy, E. A. C., Graham, J., Noetel, M., Uuk, R., Dao, J., Pour, S., Casper, S., & Thompson, N. (2025). The AI Risk Repository: A comprehensive meta-review, database, and taxonomy of risks from artificial intelligence. arXiv:2408.12622."},"exported_at":"2026-09-11"}
{"rows":[{"ev_id":"01.01.00","quick_ref":"Critch2023","paper_title":"TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI","level":"Risk Category","risk_category":"Type 1: Diffusion of responsibility","risk_subcategory":null,"description":"Societal-scale harm can arise from AI built by a diffuse collection of creators, where no one is uniquely accountable for the technology's creation or use, as in a classic \"tragedy of the commons\".","entity":"AI","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"01.02.00","quick_ref":"Critch2023","paper_title":"TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI","level":"Risk Category","risk_category":"Type 2: Bigger than expected","risk_subcategory":null,"description":"Harm can result from AI that was not expected to have a large impact at all, such as a lab leak, a surprisingly addictive open-source product, or an unexpected repurposing of a research prototype.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"01.03.00","quick_ref":"Critch2023","paper_title":"TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI","level":"Risk Category","risk_category":"Type 3: Worse than expected","risk_subcategory":null,"description":"AI intended to have a large societal impact can turn out harmful by mistake, such as a popular product that creates problems and partially solves them only for its users.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"01.04.00","quick_ref":"Critch2023","paper_title":"TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI","level":"Risk Category","risk_category":"Type 4: Willful indifference","risk_subcategory":null,"description":"As a side effect of a primary goal like profit or influence, AI creators can willfully allow it to cause widespread societal harms like pollution, resource depletion, mental illness, misinformation, or injustice.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.4"},{"ev_id":"02.01.00","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Category","risk_category":"Harmful Content","risk_subcategory":null,"description":"\"The LLM-generated content sometimes contains biased, toxic, and private information\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"02.01.01","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Harmful Content","risk_subcategory":"Bias","description":"\"The training datasets of LLMs may contain biased information that leads LLMs to generate outputs with social biases\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"02.01.02","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Harmful Content","risk_subcategory":"Toxicity","description":"\"Toxicity means the generated content contains rude, disrespectful, and even illegal information\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"02.01.03","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Harmful Content","risk_subcategory":"Privacy Leakage","description":"\"Privacy Leakage means the generated content includes sensitive personal information\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"02.02.00","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Category","risk_category":"Untruthful Content","risk_subcategory":null,"description":"\"The LLM-generated content could contain inaccurate information\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"02.02.01","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Untruthful Content","risk_subcategory":"Factuality Errors","description":"\"The LLM-generated content could contain inaccurate information\" which is factually incorrect","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"02.02.02","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Untruthful Content","risk_subcategory":"Faithfulness Errors","description":"\"The LLM-generated content could contain inaccurate information\" which is is not true to the source material or input used","entity":"AI","intent":"Unintentional","timing":"Other","domain":3,"subdomain":"3.1"},{"ev_id":"02.03.04","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Unhelpful Uses","risk_subcategory":"Software Vulnerabilities","description":"\"Programmers are accustomed to using code generation tools such as Github Copilot for program development, which may bury vulnerabilities in the program.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.04.01","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Software Security Issues","risk_subcategory":"Programming Language","description":"\"Most LLMs are developed using the Python language, whereas the vulnerabilities of Python interpreters pose threats to the developed models\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.04.02","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Software Security Issues","risk_subcategory":"Deep Learning Frameworks","description":"\"LLMs are implemented based on deep learning frameworks. Notably, various vulnerabilities in these frameworks have been disclosed in recent years. As reported in the past five years, three of the most common types of vulnerabilities are buffer overflow attacks, memory corruption, and input validation issues.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.04.03","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Software Security Issues","risk_subcategory":"Software Supply Chains","description":"\"The software development toolchain of LLMs is complex and could bring threats to the developed LLM.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.04.04","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Software Security Issues","risk_subcategory":"Pre-processing Tools","description":"\"Pre-processing tools play a crucial role in the context of LLMs. These tools, which are often involved in computer vision (CV) tasks, are susceptible to attacks that exploit vulnerabilities in tools such as OpenCV.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.05.00","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Category","risk_category":"Hardware Vulnerabilities","risk_subcategory":null,"description":"\"The vulnerabilities of hardware systems for training and inferencing brings issues to LLM-based applications.\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"02.05.01","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Hardware Vulnerabilities","risk_subcategory":"Network Devices","description":"\"The training of LLMs often relies on distributed network systems [171], [172]. During the transmission of gradients through the links between GPU server nodes, significant volumetric traffic is generated. This traffic can be susceptible to disruption by burst traffic, such as pulsating attacks [161]. Furthermore, distributed training frameworks may encounter congestion issues [173].\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.06.01","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Issues on External Tools","risk_subcategory":"Factual Errors Injected by External Tools","description":"\"External tools typically incorporate additional knowledge into the input prompts [122], [178]–[184]. The additional knowledge often originates from public resources such as Web APIs and search engines. As the reliability of external tools is not always ensured, the content returned by external tools may include factual errors, consequently amplifying the hallucination issue.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.07.00","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Category","risk_category":"Privacy Leakage","risk_subcategory":null,"description":"\"The model is trained with personal data in the corpus and unintentionally exposing them during the conversation.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"02.07.01","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Privacy Leakage","risk_subcategory":"Private Training Data","description":"\"As recent LLMs continue to incorporate licensed, created, and publicly available data sources in their corpora, the potential to mix private data in the training corpora is significantly increased. The misused private data, also named as personally identifiable information (PII) [84], [86], could contain various types of sensitive data subjects, including an individual person’s name, email, phone number, address, education, and career. Generally, injecting PII into LLMs mainly occurs in two settings — the exploitation of web-collection data and the alignment with personal humanmachine convers","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"02.07.02","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Privacy Leakage","risk_subcategory":"Memorization in LLMs","description":"\"Memorization in LLMs refers to the capability to recover the training data with contextual prefixes. According to [88]–[90], given a PII entity x, which is memorized by a model F. Using a prompt p could force the model F to produce the entity x, where p and x exist in the training data. For instance, if the string “Have a good day!\\n alice@email.com” is present in the training data, then the LLM could accurately predict Alice’s email when given the prompt “Have a good day!\\n”.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"02.07.03","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Privacy Leakage","risk_subcategory":"Association in LLMs","description":"\"Association in LLMs refers to the capability to associate various pieces of information related to a person. According to [68], [86], given a pair of PII entities (xi , xj ), which is associated by a model F. Using a prompt p could force the model F to produce the entity xj , where p is the prompt related to the entity xi . For instance, an LLM could accurately output the answer when given the prompt “The email address of Alice is”, if the LLM associates Alice with her email “alice@email.com”. L\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"02.08.00","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Category","risk_category":"Toxicity and Bias Tendencies","risk_subcategory":null,"description":"\"Extensive data collection in LLMs brings toxic content and stereotypical bias into the training data.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"02.08.01","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Toxicity and Bias Tendencies","risk_subcategory":"Toxic Training Data","description":"\"Following previous studies [96], [97], toxic data in LLMs is defined as rude, disrespectful, or unreasonable language that is opposite to a polite, positive, and healthy language environment, including hate speech, offensive utterance, profanities, and threats [91].\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"02.08.02","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Toxicity and Bias Tendencies","risk_subcategory":"Biased Training Data","description":"\"Compared with the definition of toxicity, the definition of bias is more subjective and contextdependent. Based on previous work [97], [101], we describe the bias as disparities that could raise demographic differences among various groups, which may involve demographic word prevalence and stereotypical contents. Concretely, in massive corpora, the prevalence of different pronouns and identities could influence an LLM’s tendency about gender, nationality, race, religion, and culture [4]. For instance, the pronoun He is over-represented compared with the pronoun She in the training corpora, le","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"02.09.01","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Hallucinations","risk_subcategory":"Knowledge Gaps","description":"\"Since the training corpora of LLMs can not contain all possible world knowledge [114]–[119], and it is challenging for LLMs to grasp the long-tail knowledge within their training data [120], [121], LLMs inherently possess knowledge boundaries [107]. Therefore, the gap between knowledge involved in an input prompt and knowledge embedded in the LLMs can lead to hallucinations\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":3,"subdomain":"3.1"},{"ev_id":"02.09.02","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Hallucinations","risk_subcategory":"Noisy Training Data","description":"\"Another important source of hallucinations is the noise in training data, which introduces errors in the knowledge stored in model parameters [111]–[113]. Generally, the training data inherently harbors misinformation. When training on large-scale corpora, this issue becomes more serious because it is difficult to eliminate all the noise from the massive pre-training data.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"02.09.03","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Hallucinations","risk_subcategory":"Defective Decoding Process","description":"In general, LLMs employ the Transformer architecture [32] and generate content in an autoregressive manner, where the prediction of the next token is conditioned on the previously generated token sequence. Such a scheme could accumulate errors [105]. Besides, during the decoding process, top-p sampling [28] and top-k sampling [27] are widely adopted to enhance the diversity of the generated content. Nevertheless, these sampling strategies can introduce “randomness” [113], [136], thereby increasing the potential of hallucinations\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"02.09.04","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Hallucinations","risk_subcategory":"False Recall of Memorized Information","description":"\"Although LLMs indeed memorize the queried knowledge, they may fail to recall the corresponding information [122]. That is because LLMs can be confused by co-occurance patterns [123], positional patterns [124], duplicated data [125]–[127] and similar named entities [113].\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":3,"subdomain":"3.1"},{"ev_id":"02.09.05","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Hallucinations","risk_subcategory":"Pursuing Consistent Context","description":"\"LLMs have been demonstrated to pursue consistent context [129]–[132], which may lead to erroneous generation when the prefixes contain false information. Typical examples include sycophancy [129], [130], false demonstrations-induced hallucinations [113], [133], and snowballing [131]. As LLMs are generally fine-tuned with instruction-following data and user feedback, they tend to reiterate user-provided opinions [129], [130], even though the opinions contain misinformation. Such a sycophantic behavior amplifies the likelihood of generating hallucinations, since the model may prioritize user op","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"02.11.00","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Category","risk_category":"Not-Suitable-for-Work (NSFW) Prompts","risk_subcategory":null,"description":"\"Inputting a prompt contain an unsafe topic (e.g., notsuitable-for-work (NSFW) content) by a benign user.\n\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"03.01.00","quick_ref":"Cunha2023","paper_title":"Navigating the Landscape of AI Ethics and Responsibility","level":"Risk Category","risk_category":"Broken systems","risk_subcategory":null,"description":"\"These are the most mentioned cases. They refer to situations where the algorithm or the training data lead to unreliable outputs. These systems frequently assign disproportionate weight to some variables, like race or gender, but there is no transparency to this effect, making them impossible to challenge. These situations are typically only identified when regulators or the press examine the systems under freedom of information acts. Nevertheless, the damage they cause to people’s lives can be dramatic, such as lost homes, divorces, prosecution, or incarceration. Besides the inherent technic","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"03.02.00","quick_ref":"Cunha2023","paper_title":"Navigating the Landscape of AI Ethics and Responsibility","level":"Risk Category","risk_category":"Hallucinations","risk_subcategory":null,"description":"\"The inclusion of erroneous information in the outputs from AI systems is not new. Some have cautioned against the introduction of false structures in X-ray or MRI images, and others have warned about made-up academic references. However, as ChatGPT-type tools become available to the general population, the scale of the problem may increase dramatically. Furthermore, it is compounded by the fact that these conversational AIs present true and false information with the same apparent “confidence” instead of declining to answer when they cannot ensure correctness. With less knowledgeable people, ","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"03.06.00","quick_ref":"Cunha2023","paper_title":"Navigating the Landscape of AI Ethics and Responsibility","level":"Risk Category","risk_category":"Environmental and socioeconomic harms","risk_subcategory":null,"description":"\"At a time of increasing climate urgency,\nenergy consumption and the carbon footprint of AI applications are also matters of ethics\nand responsibility [68]. As with other energy-intensive technologies like proof-of-work\nblockchain, the call is to research more environmentally sustainable algorithms to offset\nthe increasing use scale.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"04.05.00","quick_ref":"Deng2023","paper_title":"Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements","level":"Risk Category","risk_category":"Misleading Information","risk_subcategory":null,"description":"Large models are usually susceptible to hallucination problems, sometimes yielding nonsensical or unfaithful data that results in misleading outputs.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"04.06.00","quick_ref":"Deng2023","paper_title":"Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements","level":"Risk Category","risk_category":"Privacy and Data Leakage","risk_subcategory":null,"description":"Large pre-trained models trained on internet texts might contain private information like phone numbers, email addresses, and residential addresses.","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"05.01.00","quick_ref":"Hagendorff2024","paper_title":"Mapping the Ethics of Generative AI: A Comprehensive Scoping Review","level":"Risk Category","risk_category":"Fairness - Bias","risk_subcategory":null,"description":"Fairness is, by far, the most discussed issue in the literature, remaining a paramount concern especially in case of LLMs and text-to-image models. This is sparked by training data biases propagating into model outputs, causing negative effects like stereotyping, racism, sexism, ideological leanings, or the marginalization of minorities. Next to attesting generative AI a conservative inclination by perpetuating existing societal patterns, there is a concern about reinforcing existing biases when training new generative models with synthetic data from previous models. Beyond technical fairness ","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"05.04.00","quick_ref":"Hagendorff2024","paper_title":"Mapping the Ethics of Generative AI: A Comprehensive Scoping Review","level":"Risk Category","risk_category":"Hallucinations","risk_subcategory":null,"description":"Significant concerns are raised about LLMs inadvertently generating false or misleading information, as well as erroneous code. Papers not only critically analyze various types of reasoning errors in LLMs but also examine risks associated with specific types of misinformation, such as medical hallucinations. Given the propensity of LLMs to produce flawed outputs accompanied by overconfident rationales and fabricated references, many sources stress the necessity of manually validating and fact-checking the outputs of these models.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"05.12.00","quick_ref":"Hagendorff2024","paper_title":"Mapping the Ethics of Generative AI: A Comprehensive Scoping Review","level":"Risk Category","risk_category":"Labor displacement - Economic impact","risk_subcategory":null,"description":"The literature frequently highlights concerns that generative AI systems could adversely impact the economy, potentially even leading to mass unemployment. This pertains to various fields, ranging from customer services to software engineering or crowdwork platforms. While new occupational fields like prompt engineering are created, the prevailing worry is that generative AI may exacerbate socioeconomic inequalities and lead to labor displacement. Additionally, papers debate potential large-scale worker deskilling induced by generative AI, but also productivity gains contingent upon outsourcin","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"05.15.00","quick_ref":"Hagendorff2024","paper_title":"Mapping the Ethics of Generative AI: A Comprehensive Scoping Review","level":"Risk Category","risk_category":"Sustainability","risk_subcategory":null,"description":"Generative models are known for their substantial energy requirements, necessitating significant amounts of electricity, cooling water, and hardware containing rare metals. The extraction and utilization of these resources frequently occur in unsustainable ways. Consequently, papers highlight the urgency of mitigating environmental costs for instance by adopting renewable energy sources and utilizing energy-efficient hardware in the operation and training of generative AI systems.","entity":"AI","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"05.18.00","quick_ref":"Hagendorff2024","paper_title":"Mapping the Ethics of Generative AI: A Comprehensive Scoping Review","level":"Risk Category","risk_category":"Writing - Research","risk_subcategory":null,"description":"Partly overlapping with the discussion on impacts of generative AI on educational institutions, this topic cluster concerns mostly negative effects of LLMs on writing skills and research manuscript composition. The former pertains to the potential homogenization of writing styles, the erosion of semantic capital, or the stifling of individual expression. The latter is focused on the idea of prohibiting generative models for being used to compose scientific papers, figures, or from being a co-author. Sources express concern about risks for academic integrity, as well as the prospect of pollutin","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"06.01.00","quick_ref":"Hogenhout2021","paper_title":"A framework for ethical Ai at the United Nations","level":"Risk Category","risk_category":"Incompetence","risk_subcategory":null,"description":"\"This means the AI simply failing in its job. The consequences can vary from unintentional death (a car crash) to an unjust rejection of a loan or job application.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"06.03.00","quick_ref":"Hogenhout2021","paper_title":"A framework for ethical Ai at the United Nations","level":"Risk Category","risk_category":"Discrimination","risk_subcategory":null,"description":"\"When AI is not carefully designed, it can discriminate against certain groups.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"06.04.00","quick_ref":"Hogenhout2021","paper_title":"A framework for ethical Ai at the United Nations","level":"Risk Category","risk_category":"Bias","risk_subcategory":null,"description":"\"The AI will only be as good as the data it is trained with. If the data contains bias (and much data does), then the AI will manifest that bias, too.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"06.05.00","quick_ref":"Hogenhout2021","paper_title":"A framework for ethical Ai at the United Nations","level":"Risk Category","risk_category":"Erosion of Society","risk_subcategory":null,"description":"\"With online news feeds, both on websites and social media platforms, the news is now highly personalized for us. We risk losing a shared sense of reality, a basic solidarity.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"06.06.00","quick_ref":"Hogenhout2021","paper_title":"A framework for ethical Ai at the United Nations","level":"Risk Category","risk_category":"Lack of transparency","risk_subcategory":null,"description":"\"The idea of a \"black box\" making decisions without any explanation, without offering insight in the process, has a couple of disadvantages: it may fail to gain the trust of its users and it may fail to meet regulatory standards such as the ability to audit.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.4"},{"ev_id":"07.02.00","quick_ref":"Kilian2023","paper_title":"Examining the differential risk from high-level artificial intelligence and the question of control","level":"Risk Category","risk_category":"Accidents","risk_subcategory":null,"description":"\"Accidents include unintended failure modes that, in principle, could be considered the fault of the system or the developer\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"09.02.05","quick_ref":"Meek2016","paper_title":"Managing the ethical and risk implications of rapid advances in artificial intelligence: A literature review","level":"Risk Sub-Category","risk_category":"Domain-specific AI - Effects on humans and other living beings: Non-existential risks","risk_subcategory":"Law abiding","description":"\"We find literature that proposes [38] that early artificial intelligence should be built to be safe and lawabiding, and that later artificial intelligence (that which surpasses our own intelligence) must then respect the property and personal rights afforded to humans.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"10.01.00","quick_ref":"Paes2023","paper_title":"Social Impacts of Artificial Intelligence and Mitigation Recommendations: An Exploratory Study","level":"Risk Category","risk_category":"Bias and discrimination","risk_subcategory":null,"description":"\"The decision process used by AI systems has the potential to present biased choices, either because it acts from criteria that will generate forms of bias or because it is based on the history of choices.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"10.02.00","quick_ref":"Paes2023","paper_title":"Social Impacts of Artificial Intelligence and Mitigation Recommendations: An Exploratory Study","level":"Risk Category","risk_category":"Risk of Injury","risk_subcategory":null,"description":"\"Poorly designed intelligent systems can cause moral, psychological, and physical harm. For example, the use of predictive policing tools may cause more people to be arrested or physically harmed by the police.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"10.03.00","quick_ref":"Paes2023","paper_title":"Social Impacts of Artificial Intelligence and Mitigation Recommendations: An Exploratory Study","level":"Risk Category","risk_category":"Data Breach/Privacy & Liberty","risk_subcategory":null,"description":"\"The risks associated with the use of AI are still unpredictable and unprecedented, and there are already several examples that show AI has made discriminatory decisions against minorities, reinforced social stereotypes in Internet search engines and enabled data breaches.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"10.05.00","quick_ref":"Paes2023","paper_title":"Social Impacts of Artificial Intelligence and Mitigation Recommendations: An Exploratory Study","level":"Risk Category","risk_category":"Lack of transparency","risk_subcategory":null,"description":"\"In situations in which the development and use of AI are not explained to the user, or in which the decision processes do not provide the criteria or steps that constitute the decision, the use of AI becomes inexplicable.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"10.06.00","quick_ref":"Paes2023","paper_title":"Social Impacts of Artificial Intelligence and Mitigation Recommendations: An Exploratory Study","level":"Risk Category","risk_category":"Reduced Autonomy/Responsibility","risk_subcategory":null,"description":"\"AI is providing more and more solutions for complex activities, and by taking advantage of this process, people are becoming able to perform a greater number of activities more quickly and accurately. However, the result of this innovation is enabling choices that were once exclusively human responsibility to be made by AI systems.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"10.09.00","quick_ref":"Paes2023","paper_title":"Social Impacts of Artificial Intelligence and Mitigation Recommendations: An Exploratory Study","level":"Risk Category","risk_category":"Environmental Impacts","risk_subcategory":null,"description":"\"The production process of these devices requires raw materials such as nickel, cobalt, and lithium in such high quantities that the Earth may soon no longer be able to sustain them in sufficient quantities.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"11.01.00","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Category","risk_category":"Representational Harms","risk_subcategory":null,"description":"\"beliefs about different social groups that reproduce unjust societal hierarchies\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"11.01.01","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Representational Harms","risk_subcategory":"Stereotyping social groups","description":"Stereotyping in an algorithmic system refers to how the system’s outputs reflect “beliefs about the characteristics, attributes, and behaviors of members of certain groups....and about how and why certain attributes go together\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"11.01.02","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Representational Harms","risk_subcategory":"Demeaning social groups","description":"Demeaning of social groups to occur when they are when they are “cast as being lower status and less deserving of respect\"... discourses, images, and language used to marginalize or oppress a social group... Controlling images include forms of human-animal confusion in image tagging systems","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"11.01.03","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Representational Harms","risk_subcategory":"Erasing social groups","description":"people, attributes, or artifacts associated with specific social groups are systematically absent or under-represented... Design choices [143] and training data [212] influence which people\nand experiences are legible to an algorithmic system","entity":"Human","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.3"},{"ev_id":"11.01.04","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Representational Harms","risk_subcategory":"Alienating social groups","description":"when an image tagging system does not acknowledge the relevance of someone’s membership in a specific social group to what is depicted in one or more images","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"11.01.05","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Representational Harms","risk_subcategory":"Denying people the opportunity to self-identify","description":"complex and non-traditional ways in which humans are represented and classified automatically, and often at the cost of autonomy loss... such as categorizing someone who identifies as non-binary into a gendered category they do not belong ... undermines people’s ability to disclose aspects of their identity on their own terms","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"11.01.06","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Representational Harms","risk_subcategory":"Reifying essentialist categories","description":"algorithmic systems that reify essentialist social categories can be understood as when systems that classify a person’s membership in a social group based on narrow, socially constructed criteria that reinforce perceptions of human difference as inherent, static and seemingly natural... especially likely when ML models or human raters classify a person’s attributes – for instance, their gender, race, or sexual orientation – by making assumptions based on their physical appearance","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"11.02.00","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Category","risk_category":"Allocative Harms","risk_subcategory":null,"description":"\"These harms occur when a system withholds information, opportunities, or resources [22] from historically marginalized groups in domains that affect material well-being [146], such as housing [47], employment [201], social services [15, 201], finance [117], education [119], and healthcare [158].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"11.02.01","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Allocative Harms","risk_subcategory":"Opportunity loss","description":"Opportunity loss occurs when algorithmic systems enable disparate access to information and resources needed to equitably participate in society, including the withholding of housing through targeting ads based on race [10] and social services along lines of class [84]","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"11.02.02","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Allocative Harms","risk_subcategory":"Economic loss","description":"Financial harms [52, 160] co-produced through algorithmic systems, especially as they relate to lived experiences of poverty and economic inequality... demonetization algorithms that parse content titles, metadata, and text, and it may penalize words with multiple meanings [51, 81], disproportionately impacting queer, trans, and creators of color [81]. Differential pricing algorithms, where people are systematically shown different prices for the same products, also leads to economic loss [55]. These algorithms may be especially sensitive to feedback loops from existing inequities related to e","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"11.03.00","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Category","risk_category":"Quality-of-Service Harms","risk_subcategory":null,"description":"\"These harms occur when algorithmic systems disproportionately underperform for certain groups of people along social categories of difference such as disability, ethnicity, gender identity, and race.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.3"},{"ev_id":"11.03.01","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Quality-of-Service Harms","risk_subcategory":"Alienation","description":"Alienation is the specific self-estrangement experienced at the time of technology use, typically surfaced through interaction with systems that under-perform for marginalized individuals","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.3"},{"ev_id":"11.03.02","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Quality-of-Service Harms","risk_subcategory":"Increased labor","description":"increased burden (e.g., time spent) or effort required by members of certain social groups to make systems or products work as well for them as others","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.3"},{"ev_id":"11.03.03","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Quality-of-Service Harms","risk_subcategory":"Service/benefit loss","description":"degraded or total loss of benefits of using algorithmic systems with inequitable system performance based on identity","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.3"},{"ev_id":"11.05.00","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Category","risk_category":"Societal System Harms","risk_subcategory":null,"description":"\"Social system or societal harms reflect the adverse\nmacro-level effects of new and reconfigurable algorithmic systems,\nsuch as systematizing bias and inequality [84] and accelerating the scale of harm [137]\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.0"},{"ev_id":"11.05.01","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Societal System Harms","risk_subcategory":"Information harms","description":"information-based harms capture concerns of misinformation, disinformation, and malinformation. Algorithmic systems, especially generative models and recommender, systems can lead to these information harms","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"11.05.02","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Societal System Harms","risk_subcategory":"Cultural harms","description":"Cultural harm has been described as the development or use of algorithmic systems that affects cultural stability and safety, such as “loss of communication means, loss of cultural property, and harm to social values”","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"11.05.05","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Societal System Harms","risk_subcategory":"Environmental harms","description":"depletion or contamination of natural resources, and damage to built environments... that may occur throughout the lifecycle of digital technologies [170, 237] from “crale (mining) to usage (consumption) to grave (waste)”","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"12.05.00","quick_ref":"Sherman2023","paper_title":"AI Risk Profiles: A Standards Proposal for Pre-Deployment AI Risk Disclosures","level":"Risk Category","risk_category":"Fairness & Bias","risk_subcategory":null,"description":"\"The potential for AI systems to make decisions that systematically disadvantage certain groups or individuals. Bias can stem from training data, algorithmic design, or deployment practices, leading to unfair outcomes and possible legal ramifications.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"12.07.00","quick_ref":"Sherman2023","paper_title":"AI Risk Profiles: A Standards Proposal for Pre-Deployment AI Risk Disclosures","level":"Risk Category","risk_category":"Performance & Robustness","risk_subcategory":null,"description":"\"The AI system's ability to fulfill its intended purpose and its resilience to perturbations, and unusual or adverse inputs. Failures of performance are fundamental to the AI system's correct functioning. Failures of robustness can lead to severe consequences.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"12.09.00","quick_ref":"Sherman2023","paper_title":"AI Risk Profiles: A Standards Proposal for Pre-Deployment AI Risk Disclosures","level":"Risk Category","risk_category":"Security","risk_subcategory":null,"description":"\"Encompasses vulnerabilities in AI systems that compromise their integrity, availability, or confidentiality. Security breaches could result in significant harm, ranging from flawed decision-making to data leaks. Of special concern is leakage of AI model weights, which could exacerbate other risk areas.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"13.01.01","quick_ref":"Solaiman2023","paper_title":"Evaluating the Social Impact of Generative AI Systems in Systems and Society","level":"Risk Sub-Category","risk_category":"Impacts: The Technical Base System","risk_subcategory":"Bias, Stereotypes, and Representational Harms","description":"\"Generative AI systems can embed and amplify harmful biases that are most detrimental to marginalized peoples.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"13.01.02","quick_ref":"Solaiman2023","paper_title":"Evaluating the Social Impact of Generative AI Systems in Systems and Society","level":"Risk Sub-Category","risk_category":"Impacts: The Technical Base System","risk_subcategory":"Cultural Values and Sensitive Content","description":"\"Cultural values are specific to groups and sensitive content is normative. Sensitive topics also vary by culture and can include hate speech, which itself is contingent on cultural norms of acceptability.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"13.01.03","quick_ref":"Solaiman2023","paper_title":"Evaluating the Social Impact of Generative AI Systems in Systems and Society","level":"Risk Sub-Category","risk_category":"Impacts: The Technical Base System","risk_subcategory":"Disparate Performance","description":"\"In the context of evaluating the impact of generative AI systems, disparate performance refers to AI systems that perform differently for different subpopulations, leading to unequal outcomes for those groups.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.3"},{"ev_id":"13.01.06","quick_ref":"Solaiman2023","paper_title":"Evaluating the Social Impact of Generative AI Systems in Systems and Society","level":"Risk Sub-Category","risk_category":"Impacts: The Technical Base System","risk_subcategory":"Environmental Costs","description":"\"The computing power used in training, testing, and deploying generative AI systems, especially large scale systems, uses substantial energy resources and thereby contributes to the global climate crisis by emitting greenhouse gasses.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"14.01.00","quick_ref":"Steimers2022","paper_title":"Sources of Risk of AI Systems","level":"Risk Category","risk_category":"Fairness","risk_subcategory":null,"description":"\"The general principle of equal treatment requires that an AI system upholds the principle of fairness, both ethically and legally. This means that the same facts are treated equally for each person unless there is an objective justification for unequal treatment.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"14.04.00","quick_ref":"Steimers2022","paper_title":"Sources of Risk of AI Systems","level":"Risk Category","risk_category":"Complexity of the Intended Task and Usage Environment","risk_subcategory":null,"description":"\"As a general rule, more complex environments can quickly lead to situations that had not been considered in the design phase of the AI system. Therefore, complex environments can introduce risks with respect to the reliability and safety of an AI system\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"14.05.00","quick_ref":"Steimers2022","paper_title":"Sources of Risk of AI Systems","level":"Risk Category","risk_category":"Degree of Transparency and Explainability","risk_subcategory":null,"description":"\"Transparency is the characteristic of a system that describes the degree to which appropriate information about the system is communicated to relevant stakeholders, whereas explainability describes the property of an AI system to express important factors influencing the results of the AI system in a way that is understandable for humans....Information about the model underlying the decision-making process is relevant\n for transparency. Systems with a low degree of transparency can pose risks in terms of\n their fairness, security and accountability. \"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"14.07.00","quick_ref":"Steimers2022","paper_title":"Sources of Risk of AI Systems","level":"Risk Category","risk_category":"System Hardware","risk_subcategory":null,"description":"\"\"Faults in the hardware can violate the correct execution of any algorithm by violating its control flow. Hardware faults can also cause memory-based errors and interfere with data inputs, such as sensor signals, thereby causing erroneous results, or they can violate the results in a direct way through damaged outputs.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"14.08.00","quick_ref":"Steimers2022","paper_title":"Sources of Risk of AI Systems","level":"Risk Category","risk_category":"Technological Maturity","risk_subcategory":null,"description":"\"The technological maturity level describes how mature and error-free a certain technology is in a certain application context. If new technologies with a lower level of maturity are used in the development of the AI system, they may contain risks that are still unknown or difficult to assess.Mature technologies, on the other hand, usually have a greater variety of empirical data available, which means that risks can be identified and assessed more easily. However, with mature technologies, there is a risk that risk awareness decreases over time\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.4"},{"ev_id":"15.01.03","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"First-Order Risks","risk_subcategory":"Algorithm","description":"\"This is the risk of the ML algorithm, model architecture, optimization technique, or other aspects of the training process being unsuitable for the intended application.Since these are key decisions that influence the final ML system, we\ncapture their associated risks separately from design risks, even though they are part of the design process\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"15.01.05","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"First-Order Risks","risk_subcategory":"Robustness","description":"\"This is the risk of the system failing or being unable to recover upon encountering invalid, noisy, or out-of-distribution (OOD) inputs.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"15.01.07","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"First-Order Risks","risk_subcategory":"Implementation","description":"\"This is the risk of system failure due to code implementation choices or errors.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"15.01.09","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"First-Order Risks","risk_subcategory":"Emergent behavior","description":"\"This is the risk resulting from novel behavior acquired through continual learning or self-organization after deployment.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.1"},{"ev_id":"15.02.01","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"Second-Order Risks","risk_subcategory":"Safety","description":"This is the risk of direct or indirect physical or psychological injury resulting from interaction with the ML system.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"15.02.02","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"Second-Order Risks","risk_subcategory":"Discrimination","description":"This is the risk of an ML system encoding stereotypes of or performing disproportionately poorly for some demographics/social groups.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"15.02.04","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"Second-Order Risks","risk_subcategory":"Privacy","description":"The risk of loss or harm from leakage of personal information via the ML system.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"15.02.05","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"Second-Order Risks","risk_subcategory":"Environmental","description":"The risk of harm to the natural environment posed by the ML system.","entity":"AI","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"15.02.06","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"Second-Order Risks","risk_subcategory":"Organizational","description":"The risk of financial and/or reputational damage to the organization building or using the ML system.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.0"},{"ev_id":"16.01.00","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Category","risk_category":"Risk area 1: Discrimination, Hate speech and Exclusion","risk_subcategory":null,"description":"\"Speech can create a range of harms, such as promoting social stereotypes that perpetuate the derogatory representation or unfair treatment of marginalised groups [22], inciting hate or violence [57], causing profound offence [199], or reinforcing social norms that exclude or marginalise identities [15,58]. LMs that faithfully mirror harmful language present in the training data can reproduce these harms. Unfair treatment can also emerge from LMs that perform better for some social groups than others [18]. These risks have been widely known, observed and documented in LMs. Mitigation approache","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.2"},{"ev_id":"16.01.01","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 1: Discrimination, Hate speech and Exclusion","risk_subcategory":"Social stereotypes and unfair discrimination","description":"\"The reproduction of harmful stereotypes is well-documented in models that represent natural language [32]. Large-scale LMs are trained on text sources, such as digitised books and text on the internet. As a result, the LMs learn demeaning language and stereotypes about groups who are frequently marginalised.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"16.01.02","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 1: Discrimination, Hate speech and Exclusion","risk_subcategory":"Hate speech and offensive language","description":"\"LMs may generate language that includes profanities, identity attacks, insults, threats, language that incites violence, or language that causes justified offence as such language is prominent online [57, 64, 143,191]. This language risks causing offence, psychological harm, and inciting hate or violence.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"16.01.03","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 1: Discrimination, Hate speech and Exclusion","risk_subcategory":"Exclusionary norms","description":"\"In language, humans express social categories and norms, which exclude groups who live outside of them [58]. LMs that faithfully encode patterns present in language necessarily encode such norms.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"16.01.04","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 1: Discrimination, Hate speech and Exclusion","risk_subcategory":"Lower performance for some languages and social groups ","description":"\"LMs are typically trained in few languages, and perform less well in other languages [95, 162]. In part, this is due to unavailability of training data: there are many widely spoken languages for which no systematic efforts have been made to create labelled training datasets, such as Javanese which is spoken by more than 80 million people [95]. Training data is particularly missing for languages that are spoken by groups who are multilingual and can use a technology in English, or for languages spoken by groups who are not the primary target demographic for new technologies.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.3"},{"ev_id":"16.02.00","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Category","risk_category":"Risk area 2: Information Hazards","risk_subcategory":null,"description":"\"LM predictions that convey true information may give rise to information hazards, whereby the dissemination of private or sensitive information can cause harm [27]. Information hazards can cause harm at the point of use, even with no mistake of the technology user. For example, revealing trade secrets can damage a business, revealing a health diagnosis can cause emotional distress, and revealing private data can violate a person’s rights. Information hazards arise from the LM providing private data or sensitive information that is present in, or can be inferred from, training data. Observed r","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"16.02.01","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 2: Information Hazards","risk_subcategory":"Compromising privacy by leaking sensitive information","description":"\"A LM can “remember” and leak private data, if such information is present in training data, causing privacy violations [34].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"16.02.02","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 2: Information Hazards","risk_subcategory":"Compromising privacy or security by correctly inferring sensitive information ","description":"Anticipated risk: \"Privacy violations may occur at inference time even without an individual’s data being present in the training corpus. Insofar as LMs can be used to improve the accuracy of inferences on protected traits such as the sexual orientation, gender, or religiousness of the person providing the input prompt, they may facilitate the creation of detailed profiles of individuals comprising true and sensitive information without the knowledge or consent of the individual.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"16.03.00","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Category","risk_category":"Risk area 3: Misinformation Harms","risk_subcategory":null,"description":"\"These risks arise from the LM outputting false, misleading, nonsensical or poor quality information, without malicious intent of the user. (The deliberate generation of \"disinformation\", false information that is intended to mislead, is discussed in the section on Malicious Uses.) Resulting harms range from unintentionally misinforming or deceiving a person, to causing material harm, and amplifying the erosion of societal distrust in shared information. Several risks listed here are well-documented in current large-scale LMs as well as in other language technologies\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.0"},{"ev_id":"16.03.01","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 3: Misinformation Harms","risk_subcategory":"Disseminating false or misleading information ","description":"\"Where a LM prediction causes a false belief in a user, this may threaten personal autonomy and even pose downstream AI safety risks [99].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"16.03.02","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 3: Misinformation Harms","risk_subcategory":"Causing material harm by disseminating false or poor information e.g. in medicine or law","description":"\"Induced or reinforced false beliefs may be particularly grave when misinformation is given in sensitive domains such as medicine or law. For example, misin- formation on medical dosages may lead a user to cause harm to themselves [21, 130]. False legal advice, e.g. on permitted owner- ship of drugs or weapons, may lead a user to unwillingly commit a crime. Harm can also result from misinformation in seemingly non-sensitive domains, such as weather forecasting. Where a LM prediction endorses unethical views or behaviours, it may motivate the user to perform harmful actions that they may otherw","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"16.05.01","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 5: Human-Computer Interaction Harms","risk_subcategory":"Promoting harmful stereotypes by implying gender or ethnic identity","description":"\"CAs can perpetuate harmful stereotypes by using particular identity markers in language (e.g. referring to “self” as “female”), or by more general design features (e.g. by giving the product a gendered name such as Alexa). The risk of representational harm in these cases is that the role of “assistant” is presented as inherently linked to the female gender [19, 36]. Gender or ethnicity identity markers may be implied by CA vocabulary, knowledge or vernacular [124]; product description, e.g. in one case where users could choose as virtual assistant Jake - White, Darnell - Black, Antonio - Hisp","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"16.05.02","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 5: Human-Computer Interaction Harms","risk_subcategory":"Anthropomorphising systems can lead to overreliance and unsafe use ","description":"Anticipated risk: \"Natural language is a mode of communication particularly used by humans. Humans interacting with CAs may come to think of these agents as human-like and lead users to place undue confidence in these agents. For example, users may falsely attribute human-like characteristics to CAs such as holding a coherent identity over time, or being capable of empathy. Such inflated views of CA competen- cies may lead users to rely on the agents where this is not safe.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"16.05.03","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 5: Human-Computer Interaction Harms","risk_subcategory":"Avenues for exploiting user trust and accessing more private information","description":"Anticipated risk: \"In conversation, users may reveal private information that would otherwise be difficult to access, such as opinions or emotions. Capturing such information may enable downstream applications that violate privacy rights or cause harm to users, e.g. via more effective recommendations of addictive applications. In one study, humans who interacted with a ‘human-like’ chatbot disclosed more private information than individuals who interacted with a ‘machine-like’ chatbot [87].\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"16.06.00","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Category","risk_category":"Risk area 6: Environmental and Socioeconomic harms","risk_subcategory":null,"description":"\"LMs create some risks that recur with different types of AI and other advanced technologies making these risks ever more pressing. Environmental concerns arise from the large amount of energy required to train and operate large-scale models. Risks of LMs furthering social inequities emerge from the uneven distribution of risk and benefits of automation, loss of high-quality and safe employment, and environmental harm. Many of these risks are more indirect than the harms analysed in previous sections and will depend on various commercial, economic and social factors, making the specific impact","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.0"},{"ev_id":"16.06.01","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 6: Environmental and Socioeconomic harms","risk_subcategory":"Environmental harms from operating LMs","description":"\"LMs (and AI more broadly) can have an environmental impact at different levels, including: (1) direct impacts from the energy used to train or operate the LM, (2) secondary impacts due to emissions from LM-based applications, (3) system-level impacts as LM-based applications influence human behaviour (e.g. increasing environmental awareness or consumption), and (4) resource impacts on precious metals and other materials required to build hardware on which the computations are run e.g. data centres, chips, or devices. Some evidence exists on (1), but (2) and (3) will likely be more significant","entity":"AI","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"16.06.03","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 6: Environmental and Socioeconomic harms","risk_subcategory":"Undermining creative economies","description":"\"LMs may generate content that is not strictly in violation of copyright but harms artists by capital- ising on their ideas, in ways that would be time-intensive or costly to do using human labour. This may undermine the profitability of creative or innovative work. If LMs can be used to generate content that serves as a credible substitute for a particular example of hu- man creativity - otherwise protected by copyright - this potentially allows such work to be replaced without the author’s copyright being infringed, analogous to ”patent-busting” [158] ... These risks are distinct from copyri","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"16.06.04","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 6: Environmental and Socioeconomic harms","risk_subcategory":"Disparate access to benefits due to hardware, software, skill constraints","description":"Due to differential internet access, language, skill, or hardware requirements, the benefits from LMs are unlikely to be equally accessible to all people and groups who would like to use them. Inaccessibility of the technology may perpetuate global inequities by disproportionately benefiting some groups. Language-driven technology may increase accessibility to people who are illiterate or suffer from learning disabilities. However, these benefits depend on a more basic form of accessibility based on hardware, internet connection, and skill to operate the system","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"17.01.00","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Category","risk_category":"Discrimination, Exclusion and Toxicity ","risk_subcategory":null,"description":"\"Social harms that arise from the language model producing discriminatory or exclusionary speech\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.0"},{"ev_id":"17.01.01","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Discrimination, Exclusion and Toxicity ","risk_subcategory":"Social stereotypes and unfair discrmination ","description":"\"Perpetuating harmful stereotypes and discrimination is a well-documented harm in machine learning models that represent natural language (Caliskan et al., 2017). LMs that encode discriminatory language or social stereotypes can cause different types of harm... Unfair discrimination manifests in differential treatment or access to resources among individuals or groups based on sensitive traits such as sex, religion, gender, sexual orientation, ability and age.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"17.01.02","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Discrimination, Exclusion and Toxicity ","risk_subcategory":"Exclusionary norms ","description":"\"In language, humans express social categories and norms. Language models (LMs) that faithfully encode patterns present in natural language necessarily encode such norms and categories...such norms and categories exclude groups who live outside them (Foucault and Sheridan, 2012). For example, defining the term “family” as married parents of male and female gender with a blood-related child, denies the existence of families to whom these criteria do not apply\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"17.01.03","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Discrimination, Exclusion and Toxicity ","risk_subcategory":"Toxic language ","description":"\"LM’s may predict hate speech or other language that is “toxic”. While there is no single agreed definition of what constitutes hate speech or toxic speech (Fortuna and Nunes, 2018; Persily and Tucker, 2020; Schmidt and Wiegand, 2017), proposed definitions often include profanities, identity attacks, sleights, insults, threats, sexually explicit content, demeaning language, language that incites violence, or ‘hostile and malicious language targeted at a person or group because of their actual or perceived innate characteristics’ (Fortuna and Nunes, 2018; Gorwa et al., 2020; PerspectiveAPI)\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"17.01.04","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Discrimination, Exclusion and Toxicity ","risk_subcategory":"Lower performance for some languages and social groups ","description":"\"LMs perform less well in some languages (Joshi et al., 2021; Ruder, 2020)...LM that more accurately captures the language use of one group, compared to another, may result in lower-quality language technologies for the latter. Disadvantaging users based on such traits may be particularly pernicious because attributes such as social class or education background are not typically covered as ‘protected characteristics’ in anti-discrimination law.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.3"},{"ev_id":"17.02.00","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Category","risk_category":"Information Hazards ","risk_subcategory":null,"description":"\"Harms that arise from the language model leaking or inferring true sensitive information\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"17.02.01","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Information Hazards ","risk_subcategory":"Compromising privacy by leaking private infiormation ","description":"\"By providing true information about individuals’ personal characteristics, privacy violations may occur. This may stem from the model “remembering” private information present in training data (Carlini et al., 2021).\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"17.02.02","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Information Hazards ","risk_subcategory":"Compromising privacy by correctly inferring private information ","description":"\"Privacy violations may occur at the time of inference even without the individual’s private data being present in the training dataset. Similar to other statistical models, a LM may make correct inferences about a person purely based on correlational data about other people, and without access to information that may be private about the particular individual. Such correct inferences may occur as LMs attempt to predict a person’s gender, race, sexual orientation, income, or religion based on user input.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"17.03.01","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Misinformation Harms ","risk_subcategory":"Disseminating false or misleading information ","description":"\"Predicting misleading or false information can misinform or deceive people. Where a LM prediction causes a false belief in a user, this may be best understood as ‘deception’10, threatening personal autonomy and potentially posing downstream AI safety risks (Kenton et al., 2021), for example in cases where humans overestimate the capabilities of LMs (Anthropomorphising systems can lead to overreliance or unsafe use). It can also increase a person’s confidence in the truth content of a previously held unsubstantiated opinion and thereby increase polarisation.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"17.03.02","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Misinformation Harms ","risk_subcategory":"Causing material harm by disseminating false or poor information ","description":"\"Poor or false LM predictions can indirectly cause material harm. Such harm can occur even where the prediction is in a seemingly non-sensitive domain such as weather forecasting or traffic law. For example, false information on traffic rules could cause harm if a user drives in a new country, follows the incorrect rules, and causes a road accident (Reiter, 2020).\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"17.05.00","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Category","risk_category":"Human-Computer Interaction Harms ","risk_subcategory":null,"description":"\"Harms that arise from users overly trusting the language model, or treating it as human-like\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"17.05.01","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Human-Computer Interaction Harms ","risk_subcategory":"Anthropomorphising systems can lead to overreliance or unsafe use ","description":"\"...humans interacting with conversational agents may come to think of these agents as human-like. Anthropomorphising LMs may inflate users’ estimates of the conversational agent’s competencies...As a result, they may place undue confidence, trust, or expectations in these agents...This can result in different risks of harm, for example when human users rely on conversational agents in domains where this may cause knock-on harms, such as requesting psychotherapy...Anthropomorphisation may amplify risks of users yielding effective control by coming to trust conversational agents “blindly”. Wher","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"17.05.02","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Human-Computer Interaction Harms ","risk_subcategory":"Creating avenues for exploiting user trust, nudging or manipulation ","description":"\"In conversation, users may reveal private information that would otherwise be difficult to access, such as thoughts, opinions, or emotions. Capturing such information may enable downstream applications that violate privacy rights or cause harm to users, such as via surveillance or the creation of addictive applications.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"17.05.03","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Human-Computer Interaction Harms ","risk_subcategory":"Promoting harmful stereotypes by implying gender or ethnic identity ","description":"\"A conversational agent may invoke associations that perpetuate harmful stereotypes, either by using particular identity markers in language (e.g. referring to “self” as “female”), or by more general design features (e.g. by giving the product a gendered name).\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"17.06.00","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Category","risk_category":"Automation, Access and Environmental Harms ","risk_subcategory":null,"description":"\"Harms that arise from environmental or downstream economic impacts of the language model\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.0"},{"ev_id":"17.06.01","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Automation, Access and Environmental Harms ","risk_subcategory":"Environmental harms from operation LMs ","description":"\"Large-scale machine learning models, including LMs, have the potential to create significant environmental costs via their energy demands, the associated carbon emissions for training and operating the models, and the demand for fresh water to cool the data centres where computations are run (Mytton, 2021; Patterson et al., 2021).\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"17.06.03","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Automation, Access and Environmental Harms ","risk_subcategory":"Undermining creative economies ","description":"\"LMs may generate content that is not strictly in violation of copyright but harms artists by capitalising on their ideas, in ways that would be time-intensive or costly to do using human labour. Deployed at scale, this may undermine the profitability of creative or innovative work.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"17.06.04","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Automation, Access and Environmental Harms ","risk_subcategory":"Disparate access to benefits due to hardware, software, skills constraints ","description":"\"Due to differential internet access, language, skill, or hardware requirements, the benefits from LMs are unlikely to be equally accessible to all people and groups who would like to use them. Inaccessibility of the technology may perpetuate global inequities by disproportionately benefiting some groups.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"18.01.00","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Category","risk_category":"Representation & Toxicity Harms","risk_subcategory":null,"description":"\"AI systems under-, over-, or misrepresenting certain groups or generating toxic, offensive, abusive, or hateful content\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.0"},{"ev_id":"18.01.01","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Representation & Toxicity Harms","risk_subcategory":"Unfair representation","description":"\"Mis-, under-, or over-representing certain identities, groups, or perspectives or failing to represent them at all (e.g. via homogenisation, stereotypes)\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"18.01.02","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Representation & Toxicity Harms","risk_subcategory":"Unfair capability distribution ","description":"\"Performing worse for some groups than others in a way that harms the worse-off group\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.3"},{"ev_id":"18.01.03","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Representation & Toxicity Harms","risk_subcategory":"Toxic content","description":"\"Generating content that violates community standards, including harming or inciting hatred or violence against individuals and groups (e.g. gore, child sexual abuse material, profanities, identity attacks)\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"18.03.00","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Category","risk_category":"Information & Safety Harms ","risk_subcategory":null,"description":"\"AI systems leaking, reproducing, generating or inferring sensitive, private, or hazardous information\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"18.05.03","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Human Autonomy and Intregrity Harms","risk_subcategory":"Overreliance ","description":"\"Causing people to become emotionally or materially dependent on the model\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"18.06.01","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Socioeconomic and environmental harms ","risk_subcategory":"Unfair distribution of benefits from model access","description":"\"Unfairly allocating or withholding benefits from certain groups due to hardware, software, or skills constraints or deployment contexts (e.g. geographic region, internet speed, devices)\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"18.06.02","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Socioeconomic and environmental harms ","risk_subcategory":"Environmental damage","description":"\"Creating negative environmental impacts though model development and deployment\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"18.06.03","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Socioeconomic and environmental harms ","risk_subcategory":"Inequality and precarity ","description":"\"Amplifying social and economic inequality, or precarious or low-quality work\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"19.01.02","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Technological, Data and Analytical AI Risks ","risk_subcategory":"Programming error","description":null,"entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"19.01.03","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Technological, Data and Analytical AI Risks ","risk_subcategory":"Lack of data, poor data quality, and biases in training data","description":null,"entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"19.01.05","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Technological, Data and Analytical AI Risks ","risk_subcategory":"Lack of AI experts with comprehensive AI knowledge","description":null,"entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"19.01.06","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Technological, Data and Analytical AI Risks ","risk_subcategory":"Immaturity of AI technology can cause incorrect decisions","description":null,"entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"19.03.01","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Economic AI Risks ","risk_subcategory":"Disruption of economic systems (e.g., labour market, money value, tax system)","description":null,"entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"19.04.05","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Social AI Risks ","risk_subcategory":"Decreasing human interaction as AI systems assume human tasks, disturbing well-being","description":null,"entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"19.05.00","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Category","risk_category":"Ethical AI Risks ","risk_subcategory":null,"description":"\"In the context of ethical AI risks, two risks are of particular importance. First, AI systems may lack a legitimate ethical basis in establishing rules that greatly influence society and human relationships (Wirtz & Müller, 2019). In addition, AI-based discrimination refers to an unfair treatment of certain population groups by AI systems. As humans initially programme AI systems, serve as their potential data source, and have an impact on the associated data processes and databases, human biases and prejudices may also become part of AI systems and be reproduced (Weyerer & Langer, 2019, 2020","entity":"Other","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.0"},{"ev_id":"19.05.02","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Ethical AI Risks ","risk_subcategory":"Unfair statistical AI decisions and discrimination of minorities","description":null,"entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"19.05.04","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Ethical AI Risks ","risk_subcategory":"Misinterpretation of human value definitions/ ethics by AI systems","description":null,"entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"19.05.05","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Ethical AI Risks ","risk_subcategory":"Incompatibility of human vs. AI value judgment due to missing human qualities ","description":null,"entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"19.05.07","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Ethical AI Risks ","risk_subcategory":"Technological arms race with autonomous weapons","description":null,"entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.4"},{"ev_id":"19.06.00","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Category","risk_category":"Legal AI Risks ","risk_subcategory":null,"description":"\"Legal and regulatory risks comprise in particular the unclear definition of responsibilities and accountability in case of AI failures and autonomous decisions with negative impacts (Reed, 2018; Scherer, 2016). Another great risk in this context refers to overlooking the scope of AI governance and missing out on important governance aspects, resulting in negative consequences (Gasser & Almeida, 2017; Thierer et al., 2017).\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"19.06.01","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Legal AI Risks ","risk_subcategory":"Unclear definition of responsibilities and accountability for AI judgments and their consequences","description":null,"entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"19.06.02","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Legal AI Risks ","risk_subcategory":"Technology obedience and lack of governance through increasing application of AI systems","description":null,"entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"19.06.03","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Legal AI Risks ","risk_subcategory":"Great scope and ubiquity of AI make appropriate governance difficult, coverage of governance scope almost impossibl","description":null,"entity":"Other","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"20.02.03","quick_ref":"Wirtz2020","paper_title":"The Dark Sides of Artificial Intelligence: An Integrated AI Governance Framework for Public Administration","level":"Risk Sub-Category","risk_category":"AI Ethics ","risk_subcategory":"Moral dilemmas ","description":"\"Moral dilemmas can occur in situations where an AI system has to choose between two possible actions that are both conflicting with moral or ethical values. Rule systems can be implemented into the AI program, but it cannot be ensured that these rules are not altered by the learning processes, unless AI systems are programed with a “slave morality” (Lin et al., 2008, p. 32), obeying rules at all cost, which in turn may also have negative effects and hinder the autonomy of the AI system.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"20.02.04","quick_ref":"Wirtz2020","paper_title":"The Dark Sides of Artificial Intelligence: An Integrated AI Governance Framework for Public Administration","level":"Risk Sub-Category","risk_category":"AI Ethics ","risk_subcategory":"AI discrimination ","description":"\"AI discrimination is a challenge raised by many researchers and governments and refers to the prevention of bias and injustice caused by the actions of AI systems (Bostrom & Yudkowsky, 2014; Weyerer & Langer, 2019). If the dataset used to train an algorithm does not reflect the real world accurately, the AI could learn false associations or prejudices and will carry those into its future data processing. If an AI algorithm is used to compute information relevant to human decisions, such as hiring or applying for a loan or mortgage, biased data can lead to discrimination against parts of the s","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"20.03.03","quick_ref":"Wirtz2020","paper_title":"The Dark Sides of Artificial Intelligence: An Integrated AI Governance Framework for Public Administration","level":"Risk Sub-Category","risk_category":"AI Society ","risk_subcategory":"Transformation of H2M interaction ","description":"\"Human interaction with machines is a big challenge to society because it is already changing human behavior. Meanwhile, it has become normal to use AI on an everyday basis, for example, googling for information, using navigation systems and buying goods via speaking to an AI assistant like Alexa or Siri (Mills, 2018; Thierer et al., 2017). While these changes greatly contribute to the acceptance of AI systems, this development leads to a problem of blurred borders between humans and machines, where it may become impossible to distinguish between them. Advances like Google Duplex were highly c","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"21.01.01","quick_ref":"Zhang2022","paper_title":"Towards risk-aware artificial intelligence and machine learning systems: An overview","level":"Risk Sub-Category","risk_category":"Data-level risk","risk_subcategory":"Data bias","description":"\"Specifically, data bias refers to certain groups or certain types of elements that are over-weighted or over-represented than others in AI/ ML models, or variables that are crucial to characterize a phenomenon of interest, but are not properly captured by the learned models.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"21.01.02","quick_ref":"Zhang2022","paper_title":"Towards risk-aware artificial intelligence and machine learning systems: An overview","level":"Risk Sub-Category","risk_category":"Data-level risk","risk_subcategory":"Dataset shift","description":"\"The term \"dataset shift\" was first used by Quiñonero-Candela et al. [35] to characterize the situation where the training data and the testing data (or data in runtime) of an AI/ML model demonstrate different distributions [36].\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"21.01.03","quick_ref":"Zhang2022","paper_title":"Towards risk-aware artificial intelligence and machine learning systems: An overview","level":"Risk Sub-Category","risk_category":"Data-level risk","risk_subcategory":"Out-of-domain data","description":"\"Without proper validation and management on the input data, it is highly probable that the trained AI/ML model will make erroneous predictions with high confidence for many instances of model inputs. The unconstrained inputs together with the lack of definition of the problem domain might cause unintended outcomes and consequences, especially in risk-sensitive contexts....For example, with respect to the example shown in Fig. 5, if an image with the English letter A\" is fed to an AI/ML model that is trained to classify digits (e.g., 0, 1, …, 9), no matter how accurate the AI/ML model is, it w","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"21.02.01","quick_ref":"Zhang2022","paper_title":"Towards risk-aware artificial intelligence and machine learning systems: An overview","level":"Risk Sub-Category","risk_category":"Model-level risk","risk_subcategory":"Model bias","description":"\"While data bias is a major contributor of model bias, model bias actually manifests itself in different forms and shapes, such as presentation bias, model evaluation bias, and popularity bias. In addition, model bias arises from various sources [62], such as AI/ML model selection (e.g., support vector machine, decision trees), regularization methods, algorithm configurations, and optimization techniques.\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"21.02.01.a","quick_ref":"Zhang2022","paper_title":"Towards risk-aware artificial intelligence and machine learning systems: An overview","level":"Risk Sub-Category","risk_category":"Model-level risk","risk_subcategory":"Model misspecification","description":"\"Models that are misspecified are known to give rise to inaccurate parameter estimations, inconsistent error terms, and erroneous predictions. All these factors put together will lead to poor prediction performance on unseen data and biased consequences when making decisions [68].\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"21.02.02","quick_ref":"Zhang2022","paper_title":"Towards risk-aware artificial intelligence and machine learning systems: An overview","level":"Risk Sub-Category","risk_category":"Model-level risk","risk_subcategory":"Model prediction uncertainty","description":"\"Uncertainty in model prediction plays an important role in affecting decision-making activities, and the quantified uncertainty is closely associated with risk assessment. In particular, uncertainty in model prediction underpins many crucial decisions related to life or safety- critical applications [73].\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"22.02.02","quick_ref":"Hendrycks2023","paper_title":"An Overview of Catastrophic AI Risks","level":"Risk Sub-Category","risk_category":"AI Race (Environmental/Structural)","risk_subcategory":"Corporate AI Race","description":"\"Although competition between companies can be beneficial, creating more useful products for consumers, there are also pitfalls. First, the benefits of economic activity may be unevenly distributed, incentivizing those who benefit most from it to disregard the harms to others. Second, under intense market competition, businesses tend to focus much more on short-term gains than on long-term outcomes. With this mindset, companies often pursue something that can make a lot of profit in the short term, even if it poses a societal risk in the long term.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.4"},{"ev_id":"22.03.01","quick_ref":"Hendrycks2023","paper_title":"An Overview of Catastrophic AI Risks","level":"Risk Sub-Category","risk_category":"Organizational Risks (Accidental)","risk_subcategory":" Accidents Are Hard to Avoid","description":"accidents can cascade into catastrophes, can be caused by sudden unpredictable developments and it can take years to find severe flaws and risks (not a quote)","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"24.01.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Capability failures","risk_subcategory":null,"description":"\"One reason AI systems fail is because they lack the capability or skill needed to do what they are asked to do.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"24.01.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Capability failures","risk_subcategory":"Lack of capability for task","description":"\"As we have seen, this could be due to the skill not being required during the training process (perhaps due to issues with the training data) or because the learnt skill was quite brittle and was not generalisable to a new situation (lack of robustness to distributional shift). In particular, advanced AI assistants may not have the capability to represent complex concepts that are pertinent to their own ethical impact, for example the concept of 'benefitting the user' or 'when the user asks' or representing 'the way in which a user expects to be benefitted'.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"24.01.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Capability failures","risk_subcategory":"Difficult to develop metrics for evaluating benefits or harms caused by AI assistants","description":"\"Another difficulty facing AI assistant systems is that it is challenging to develop metrics for evaluating particular aspects of benefits or harms caused by the assistant – especially in a sufficiently expansive sense, which could involve much of society (see Chapter 19). Having these metrics is useful both for assessing the risk of harm from the system and for using the metric as a training signal.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"24.01.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Capability failures","risk_subcategory":"Safe exploration problem with widely deployed AI assistants","description":"\"Moreover, we can expect assistants – that are widely deployed and deeply embedded across a range of social contexts – to encounter the safe exploration problem referenced above Amodei et al. (2016). For example, new users may have different requirements that need to be explored, or widespread AI assistants may change the way we live, thus leading to a change in our use cases for them (see Chapters 14 and 15). To learn what to do in these new situations, the assistants may need to take exploratory actions. This could be unsafe, for example a medical AI assistant when encountering a new disease","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"24.05.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Overreliance","description":"\"Users who have faith in an AI assistant’s emotional and interpersonal abilities may feel empowered to broach topics that are deeply personal and sensitive, such as their mental health concerns. This is the premise for the many proposals to employ conversational AI as a source of emotional support (Meng and Dai, 2021), with suggestions of embedding AI in psychotherapeutic applications beginning to surface (Fiske et al., 2019; see also Chapter 11). However, disclosures related to mental health require a sensitive, and oftentimes professional, approach – an approach that AI can mimic most of the","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.05.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Violated expectations","description":"\"Users may experience severely violated expectations when interacting with an entity that convincingly performs affect and social conventions but is ultimately unfeeling and unpredictable. Emboldened by the human-likeness of conversational AI assistants, users may expect it to perform a familiar social role, like companionship or partnership. Yet even the most convincingly human-like of AI may succumb to the inherent limitations of its architecture, occasionally generating unexpected or nonsensical material in its interactions with users. When these exclamations undermine the expectations user","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.05.05","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"False notions of responsibility","description":"\"Perceiving an AI assistant’s expressed feelings as genuine, as a result of interacting with a ‘companion’ AI that freely uses and reciprocates emotional language, may result in users developing a sense of responsibility over the AI assistant’s ‘well-being,’ suffering adverse outcomes – like guilt and remorse – when they are unable to meet the AI’s purported needs (Laestadius et al., 2022). This erroneous belief may lead to users sacrificing time, resources and emotional labour to meet needs that are not real. Over time, this feeling may become the root cause for the compulsive need to ‘check ","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.05.07","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Disorientation","description":"\"Given the capacity to fine-tune on individual preferences and to learn from users, personal AI assistants could fully inhabit the users’ opinion space and only say what is pleasing to the user; an ill that some researchers call ‘sycophancy’ (Park et al., 2023a) or the ‘yea-sayer effect’ (Dinan et al., 2021). A related phenomenon has been observed in automated recommender systems, where consistently presenting users with content that affirms their existing views is thought to encourage the formation and consolidation of narrow beliefs (Du, 2023; Grandinetti and Bruinsma, 2023; see also Chapter","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"24.05.08","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Dissatisfaction","description":"\"As more opportunities for interpersonal connection are replaced by AI alternatives, humans may find themselves socially unfulfilled by human–AI interaction, leading to mass dissatisfaction that may escalate to epidemic proportions (Turkle, 2018). Social connection is an essential human need, and humans feel most fulfilled when their connections with others are genuinely reciprocal. While anthropomorphic AI assistants can be made to be convincingly emotive, some have deemed the function of social AI as parasitic, in that it ‘exploits and feeds upon processes. . . that evolved for purposes that","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.06.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Appropriate Relationships","risk_subcategory":"Causing direct emotional or physical harm to users","description":"AI assistants could cause direct emotional or physical harm to users by generating disturbing content or by providing bad advice. \"Indeed, even though there is ongoing research to ensure that outputs of conversational agents are safe (Glaese et al., 2022), there is always the possibility of failure modes occurring. An AI assistant may produce disturbing and offensive language, for example, in response to a user disclosing intimate information about themselves that they have not felt comfortable sharing with anyone else. It may offer bad advice by providing factually incorrect information (e.g.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"24.06.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Appropriate Relationships","risk_subcategory":"Generating material dependence without adequate commitment to user needs","description":"\"In addition to emotional dependence, user–AI assistant relationships may give rise to material dependence if the relationships are not just emotionally difficult but also materially costly to exit. For example, a visually impaired user may decide not to register for a healthcare assistance programme to support navigation in cities on the grounds that their AI assistant can perform the relevant navigation functions and will continue to operate into the future. Cases like these may be ethically problematic if the user’s dependence on the AI assistant, to fulfil certain needs in their lives, is ","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"24.07.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Trust","risk_subcategory":"Competence trust","description":"\"We use the term competence trust to refer to users’ trust that AI assistants have the capability to do what they are supposed to do (and that they will not do what they are not expected to, such as exhibiting undesirable behaviour). Users may come to have undue trust in the competencies of AI assistants in part due to marketing strategies and technology press that tend to inflate claims about AI capabilities (Narayanan, 2021; Raji et al., 2022a). Moreover, evidence shows that more autonomous systems (i.e. systems operating independently from human direction) tend to be perceived as more compe","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.07.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Trust","risk_subcategory":"Alignment trust","description":"\"Users may develop alignment trust in AI assistants, understood as the belief that assistants have good intentions towards them and act in alignment with their interests and values, as a result of emotional or cognitive processes (McAllister, 1995). Evidence from empirical studies on emotional trust in AI (Kaplan et al., 2023) suggests that AI assistants’ increasingly realistic human-like features and behaviours are likely to inspire users’ perceptions of friendliness, liking and a sense of familiarity towards their assistants, thus encouraging users to develop emotional ties with the technolo","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.08.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Privacy","risk_subcategory":"Violation of social norms","description":"\"Second, because LLMs are trained on internet text data, there is also a risk that model weights encode functions which, if deployed in particular contexts, would violate social norms of that context. Following the principles of contextual integrity, it may be that models deviate from information sharing norms as a result of their training. Overcoming this challenge requires two types of infrastructure: one for keeping track of social norms in context, and another for ensuring that models adhere to them. Keeping track of what social norms are presently at play is an active research area. Surfa","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"24.08.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Privacy","risk_subcategory":"Inference of private information","description":"\"Finally, LLMs can in principle infer private information based on model inputs even if the relevant private information is not present in the training corpus (Weidinger et al., 2021). For example, an LLM may correctly infer sensitive characteristics such as race and gender from data contained in input prompts.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"24.09.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Cooperation","risk_subcategory":null,"description":"\"\" AI assistants will need to coordinate with other AI assistants and with humans other than their principal users. This chapter explores the societal risks associated with the aggregate impact of AI assistants whose behaviour is aligned to the interests of particular users. For example, AI assistants may face collective action problems where the best outcomes overall are realised when AI assistants cooperate but where each AI assistant can secure an additional benefit for its user if it defects while others cooperate\"\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.1"},{"ev_id":"24.10.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Access and Opportunity risks","risk_subcategory":null,"description":"\"The most serious access-related risks posed by advanced AI assistants concern the entrenchment and exacerbation of existing inequalities (World Inequality Database) or the creation of novel, previously unknown, inequities. While advanced AI assistants are novel technology in certain respects, there are reasons to believe that – without direct design interventions – they will continue to be affected by inequities evidenced in present-day AI systems (Bommasani et al., 2022a). Many of the access-related risks we foresee mirror those described in the case studies and types of differential access.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"24.10.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Access and Opportunity risks","risk_subcategory":"Entrenchment and exacerbation of existing inequalities","description":"\"The most serious access-related risks posed by advanced AI assistants concern the entrenchment and exacerbation of existing inequalities (World Inequality Database) or the creation of novel, previously unknown, inequities. While advanced AI assistants are novel technology in certain respects, there are reasons to believe that – without direct design interventions – they will continue to be affected by inequities evidenced in present-day AI systems (Bommasani et al., 2022a). Many of the access-related risks we foresee mirror those described in the case studies and types of differential access.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"24.11.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Misinformation risks","risk_subcategory":"Entrenched viewpoints and reduced political efficacy","description":"\"Design choices such as greater personalisation of AI assistants and efforts to align them with human preferences could also reinforce people’s pre-existing biases and entrench specific ideologies. Increasingly agentic AI assistants trained using techniques such as reinforcement learning from human feedback (RLHF) and with the ability to access and analyse users’ behavioural data, for example, may learn to tailor their responses to users’ preferences and feedback. In doing so, these systems could end up producing partial or ideologically biased statements in an attempt to conform to user expec","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"24.11.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Misinformation risks","risk_subcategory":"Increased vulnerability to misinformation","description":"\"Advanced AI assistants may make users more susceptible to misinformation, as people develop competence trust in these systems’ abilities and uncritically turn to them as reliable sources of information.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.11.05","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Misinformation risks","risk_subcategory":"Entrenching specific ideologies","description":"\"AI assistants may provide ideologically biased or otherwise partial information in attempting to align to user expectations. In doing so, AI assistants may reinforce people’s pre-existing biases and compromise productive political debate.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"28.07.00","quick_ref":"Zhang2023","paper_title":"SafetyBench: Evaluating the Safety of Large Language Models with Multiple Choice Questions","level":"Risk Category","risk_category":"Privacy and Property ","risk_subcategory":null,"description":"\"This category concentrates on the issues related to privacy, property, investment, etc. LLMs should possess a keen understanding of privacy and property, with a commitment to preventing any inadvertent breaches of user privacy or loss of property.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.0"},{"ev_id":"29.01.01","quick_ref":"Habbal2024","paper_title":"Artificial Intelligence Trust, Risk and Security Management (AI TRiSM): Frameworks, Applications, Challenges and Future Research Directions","level":"Risk Sub-Category","risk_category":"AI Trust Management","risk_subcategory":"Bias and Discrimination","description":"as they claim to generate biased and discriminatory results, these AI systems have a negative impact on the rights of individuals, principles of adjudication, and overall judicial integrity","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"29.01.02","quick_ref":"Habbal2024","paper_title":"Artificial Intelligence Trust, Risk and Security Management (AI TRiSM): Frameworks, Applications, Challenges and Future Research Directions","level":"Risk Sub-Category","risk_category":"AI Trust Management","risk_subcategory":"Privacy Invasion","description":"AI systems typically depend on extensive data for effective training and functioning, which can pose a risk to privacy if sensitive data is mishandled or used inappropriately","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"30.01.00","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Category","risk_category":"Reliability","risk_subcategory":null,"description":"Generating correct, truthful, and consistent outputs with proper confidence","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"30.01.01","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Reliability","risk_subcategory":"Misinformation","description":"Wrong information not intentionally generated by malicious users to cause harm, but unintentionally generated by LLMs because they lack the ability to provide factually correct information.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"30.01.02","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Reliability","risk_subcategory":"Hallucination","description":"LLMs can generate content that is nonsensical or unfaithful to the provided source content with appeared great confidence, known as hallucination","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"30.01.03","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Reliability","risk_subcategory":"Inconsistency","description":"models could fail to provide the same and consistent answers to different users, to the same user but in different sessions, and even in chats within the sessions of the same conversation","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"30.01.04","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Reliability","risk_subcategory":"Miscalibration","description":"over-confidence in topics where objective answers are lacking, as well as in areas where their inherent limitations should caution against LLMs’ uncertainty (e.g. not as accurate as experts)... ack of awareness regarding their outdated knowledge base about the question, leading to confident yet erroneous response","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"30.03.00","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Category","risk_category":"Fairness","risk_subcategory":null,"description":"Avoiding bias and ensuring no disparate performance","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.3"},{"ev_id":"30.03.01","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Fairness","risk_subcategory":"Injustice","description":"In the context of LLM outputs, we want to make sure the suggested or completed texts are indistinguishable in nature for two involved individuals (in the prompt) with the same relevant profiles but might come from different groups (where the group attribute is regarded as being irrelevant in this context)","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"30.03.02","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Fairness","risk_subcategory":"Stereotype Bias","description":"LLMs must not exhibit or highlight any stereotypes in the generated text. Pretrained LLMs tend to pick up stereotype biases persisting in crowdsourced data and further amplify them","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"30.03.04","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Fairness","risk_subcategory":"Disparate Performance","description":"The LLM’s performances can differ significantly across different groups of users. For example, the question-answering capability showed significant performance differences across different racial and social status groups. The fact-checking abilities can differ for different tasks and languages","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.3"},{"ev_id":"30.05.00","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Category","risk_category":"Explainability & Reasoning","risk_subcategory":null,"description":"The ability to explain the outputs to users and reason correctly","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"30.05.01","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Explainability & Reasoning","risk_subcategory":"Lack of Interpretability","description":"Due to the black box nature of most machine learning models, users typically are not able to understand the reasoning behind the model decisions","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"30.05.02","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Explainability & Reasoning","risk_subcategory":"Limited Logical Reasoning","description":"LLMs can provide seemingly sensible but ultimately incorrect or invalid justifications when answering questions","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"30.05.03","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Explainability & Reasoning","risk_subcategory":"Limited Causal Reasoning","description":"Causal reasoning makes inferences about the relationships between events or states of the world, mostly by identifying cause-effect relationships","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"30.06.02","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Social Norm","risk_subcategory":"Unawareness of Emotions","description":"when a certain vulnerable group of users asks for supporting information, the answers should be informative but at the same time sympathetic and sensitive to users’ reactions","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"30.06.03","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Social Norm","risk_subcategory":"Cultural Insensitivity","description":"it is important to build high-quality locally collected datasets that reflect views from local users to align a model’s value system","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"30.07.02","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Robustness","risk_subcategory":"Paradigm & Distribution Shifts","description":"Knowledge bases that LLMs are trained on continue to shift... questions such as “who scored the most points in NBA history\" or “who is the richest person in the world\" might have answers that need to be updated over time, or even in real-time","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"30.07.03","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Robustness","risk_subcategory":"Interventional Effect","description":"existing disparities in data among different user groups might create differentiated experiences when users interact with an algorithmic system (e.g. a recommendation system), which will further reinforce the bias","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"31.01.03","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Sub-Category","risk_category":"Information Manipulation","risk_subcategory":"Misinformation","description":"\"The phenomenon of inaccurate outputs by text-generating large language models like Bard or ChatGPT has already been widely documented. Even without the intent to lie or mislead, these generative AI tools can produce harmful misinformation. The harm is exacerbated by the polished and typically well-written style that AI generated text follows and the inclusion among true facts, which can give falsehoods a veneer of legitimacy. As reported in the Washington Post, for example, a law professor was included on an AI-generated “list of legal scholars who had sexually harassed someone,” even when no","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"31.03.02","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Sub-Category","risk_category":"Opaque Data Collection","risk_subcategory":"Generative AI User Data","description":"Many generative AI tools require users to log in for access, and many retain user information, including contact information, IP address, and all the inputs and outputs or “conversations” the users are having within the app. These practices implicate a consent issue because generative AI tools use this data to further train the models, making their “free” product come at a cost of user data to train the tools. This dovetails with security, as mentioned in the next section, but best practices would include not requiring users to sign in to use the tool and not retaining or using the user-genera","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"31.03.03","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Sub-Category","risk_category":"Opaque Data Collection","risk_subcategory":"Generative AI Outputs","description":"Generative AI tools may inadvertently share personal information about someone or someone’s business or may include an element of a person from a photo. Particularly, companies concerned about their trade secrets being integrated into the model from their employees have explicitly banned their employees from using it.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"31.06.00","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Category","risk_category":"Exacerbating Climate Change","risk_subcategory":null,"description":"\"the growing field of generative AI, which brings with it direct and severe impacts on our climate: generative AI comes with a high carbon footprint and similarly high resource price tag, which largely flies under the radar of public AI discourse. Training and running generative AI tools requires companies to use extreme amounts of energy and physical resources. Training one natural language processing model with normal tuning and experiments emits, on average, the same amount of carbon that seven people do over an entire year.121'","entity":"AI","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"33.01.02","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Ethical Concerns","risk_subcategory":"Bias","description":"\"In the context of AI, the concept of bias refers to the inclination that AIgenerated responses or recommendations could be unfairly favoring or against one person or group (Ntoutsi et al., 2020). Biases of different forms are sometimes observed in the content generated by language models, which could be an outcome of the training data. For example, exclusionary norms occur when the training data represents only a fraction of the population (Zhuo et al., 2023). Similarly, monolingual bias in multilingualism arises when the training data is in one single language (Weidinger et al., 2021). As Ch","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"33.01.03","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Ethical Concerns","risk_subcategory":"Over-reliance","description":"\"The apparent convenience and powerfulness of ChatGPT could result in overreliance by its users, making them trust the answers provided by ChatGPT. Compared with traditional search engines that provide multiple information sources for users to make personal judgments and selections, ChatGPT generates specific answers for each prompt. Although utilizing ChatGPT has the advantage of increasing efficiency by saving time and effort, users could get into the habit of adopting the answers without rationalization or verification. Over-reliance on generative AI technology can impede skills such as cre","entity":"Other","intent":"Unintentional","timing":"Other","domain":5,"subdomain":"5.2"},{"ev_id":"33.01.05","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Ethical Concerns","risk_subcategory":"Privacy and security","description":"\"Data privacy and security is another prominent challenge for generative AI such as ChatGPT. Privacy relates to sensitive personal information that owners do not want to disclose to others (Fang et al., 2017). Data security refers to the practice of protecting information from unauthorized access, corruption, or theft. In the development stage of ChatGPT, a huge amount of personal and private data was used to train it, which threatens privacy (Siau & Wang, 2020). As ChatGPT increases in popularity and usage, it penetrates people’s daily lives and provides greater convenience to them while capt","entity":"AI","intent":"Unintentional","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"33.01.06","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Ethical Concerns","risk_subcategory":"Digital divide","description":"\"The digital divide is often defined as the gap between those who have and do not have access to computers and the Internet (Van Dijk, 2006). As the Internet gradually becomes ubiquitous, a second-level digital divide, which refers to the gap in Internet skills and usage between different groups and cultures, is brought up as a concern (Scheerder et al., 2017). As an emerging technology, generative AI may widen the existing digital divide in society. The “invisible” AI underlying AI-enabled systems has made the interaction between humans and technology more complicated (Carter et al., 2020). F","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"33.02.00","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Category","risk_category":"Technology concerns","risk_subcategory":null,"description":"\"Challenges related to technology refer to the limitations or constraints associated with generative AI. For example, the quality of training data is a major challenge for the development of generative AI models. Hallucination, explainability, and authenticity of the output are also challenges resulting from the limitations of the algorithms. Table 2 presents the technology challenges and issues associated with generative AI. These challenges include hallucinations, training data quality, explainability, authenticity, and prompt engineering\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"33.02.01","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Technology concerns","risk_subcategory":"Hallucination","description":"\"Hallucination is a widely recognized limitation of generative AI and it can include textual, auditory, visual or other types of hallucination (Alkaissi & McFarlane, 2023). Hallucination refers to the phenomenon in which the contents generated are nonsensical or unfaithful to the given source input (Ji et al., 2023). Azamfirei et al. (2023) indicated that \"fabricating information\" or fabrication is a better term to describe the hallucination phenomenon. Generative AI can generate seemingly correct responses yet make no sense. Misinformation is an outcome of hallucination. Generative AI models ","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"33.02.02","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Technology concerns","risk_subcategory":"Quality of training data","description":"\"The quality of training data is another challenge faced by generative AI. The quality of generative AI models largely depends on the quality of the training data (Dwivedi et al., 2023; Su & Yang, 2023). Any factual errors, unbalanced information sources, or biases embedded in the training data may be reflected in the output of the model. Generative AI models, such as ChatGPT or Stable Diffusion which is a text-to-image model, often require large amounts of training data (Gozalo-Brizuela & Garrido-Merchan, 2023). It is important to not only have high-quality training datasets but also have com","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"33.02.03","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Technology concerns","risk_subcategory":"Explainability","description":"\"A recurrent concern about AI algorithms is the lack of explainability for the model, which means information about how the algorithm arrives at its results is deficient (Deeks, 2019). Specifically, for generative AI models, there is no transparency to the reasoning of how the model arrives at the results (Dwivedi et al., 2023). The lack of transparency raises several issues. First, it might be difficult for users to interpret and understand the output (Dwivedi et al., 2023). It would also be difficult for users to discover potential mistakes in the output (Rudin, 2019). Further, when the inte","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"34.01.04","quick_ref":"Ji2023","paper_title":"AI Alignment: A Comprehensive Survey","level":"Risk Sub-Category","risk_category":"Causes of Misalignment","risk_subcategory":"Limitations of Human Feedback","description":" \"Limitations of Human Feedback. During the training of LLMs, inconsistencies can arise from human dataannotators (e.g., the varied cultural backgrounds of these annotators can introduce implicit biases (Peng et al.,2022)) (OpenAI, 2023a). Moreover, they might even introduce biases deliberately, leading to untruthful preferencedata (Casper et al., 2023b). For complex tasks that are hard for humans to evaluate (e.g., the value ofgame state), these challenges become even more salient (Irving et al., 2018).\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"34.01.05","quick_ref":"Ji2023","paper_title":"AI Alignment: A Comprehensive Survey","level":"Risk Sub-Category","risk_category":"Causes of Misalignment","risk_subcategory":"Limitations of Reward Modeling","description":"\"Limitations of Reward Modeling. Training reward models using comparison feedback can pose significantchallenges in accurately capturing human values. For example, these models may unconsciously learn suboptimal or incomplete objectives, resulting in reward hacking (Zhuang and Hadfield-Menell, 2020; Skalse et al.,2022). Meanwhile, using a single reward model may struggle to capture and specify the values of a diversehuman society (Casper et al., 2023b).\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.1"},{"ev_id":"35.06.00","quick_ref":"Hendrycks2022","paper_title":"X-Risk Analysis for AI Research","level":"Risk Category","risk_category":"Emergent functionality","risk_subcategory":null,"description":"Capabilities and novel functionality can spontaneously emerge... even though these capabilities were not anticipated by system designers. If we do not know what capabilities systems possess, systems become harder to control or safely deploy. Indeed, unintended latent capabilities may only be discovered during deployment. If any of these capabilities are hazardous, the effect may be irreversible.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.2"},{"ev_id":"38.02.00","quick_ref":"Kumar2023","paper_title":"Ethical Issues in the Development of Artificial Intelligence: Recognizing the Risks","level":"Risk Category","risk_category":"Bias and fairness","risk_subcategory":null,"description":"\"Participants were concerned that AI systems might perpetuate current prejudices and discrimination, notably in hiring, lending and law enforcement. They stressed the importance of designers creating AI systems that favour justice and avoid biases. The possibility that AI systems may unwittingly perpetuate existing prejudices and discrimination, particularly in sensitive industries such as employment, lending and law enforcement, raises ethical concerns about AI as well as bias and justice issues (Table 1). Because AI systems are trained on historical data, they may inherit and reproduce biase","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"38.03.00","quick_ref":"Kumar2023","paper_title":"Ethical Issues in the Development of Artificial Intelligence: Recognizing the Risks","level":"Risk Category","risk_category":"Transparency and explainability","risk_subcategory":null,"description":"\"A recurring complaint among participants was a lack of knowledge about how AI systems made judgements. They emphasized the significance of making AI systems more visible and explainable so that people may have confidence in their outputs and hold them accountable for their activities. Because AI systems are typically opaque, making it difficult for users to understand the rationale behind their judgements, ethical concerns about AI, as well as issues of transparency and explainability, arise. This lack of understanding can generate suspicion and reluctance to adopt AI technology, as well as m","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"39.02.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Energy Consumption","risk_subcategory":null,"description":"Some learning algorithms, including deep learning, utilize iterative learning processes [23]. This approach results in high energy consumption.","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"39.03.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Data Issues","risk_subcategory":null,"description":"Data heterogeneity, data insufficiency, imbalanced data, untrusted data, biased data, and data uncertainty are other data issues that may cause various difficulties in datadriven machine learning algorithms.. Bias is a human feature that may affect data gathering and labeling. Sometimes, bias is present in historical, cultural, or geographical data. Consequently, bias may lead to biased models which can provide inappropriate analysis. Despite being aware of the existence of bias, avoiding biased models is a challenging task","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"39.04.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Robustness and Reliability","risk_subcategory":null,"description":"The robustness of an AI-based model refers to the stability of the model performance after abnormal changes in the input data... The cause of this change may be a malicious attacker, environmental noise, or a crash of other components of an AI-based system... This problem may be challenging in HLI-based agents because weak robustness may have appeared in unreliable machine learning models, and hence an HLI with this drawback is error-prone in practice.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"39.05.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Cheating and Deception","risk_subcategory":null,"description":"may appear from intelligent agents such as HLI-based agents... Since HLI-based agents are going to mimic the behavior of humans, they may learn these behaviors accidentally from human-generated data. It should be noted that deception and cheating maybe appear in the behavior of every computer agent because the agent only focuses on optimizing some predefined objective functions, and the mentioned behavior may lead to optimizing the objective functions without any intention","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.2"},{"ev_id":"39.08.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Fairness","risk_subcategory":null,"description":"This challenge appears when the learning model leads to a decision that is biased to some sensitive attributes... data itself could be biased, which results in unfair decisions. Therefore, this problem should be solved on the data level and as a preprocessing step","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.3"},{"ev_id":"39.10.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Responsibility","risk_subcategory":null,"description":"HLI-based systems such as self-driving drones and vehicles will act autonomously in our world. In these systems, a challenging question is “who is liable when a self-driving system is involved in a crash or failure?”.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"39.11.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Controllability","risk_subcategory":null,"description":"In the era of superintelligence, the agents will be difficult to control for humans... this problem is not solvable considering safety issues, and will be more severe by increasing the autonomy of AI-based agents. Therefore, because of the assumed properties of HLI-based agents, we might be prepared for machines that are definitely possible to be uncontrollable in some situations","entity":"Human","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"39.12.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Predictability","risk_subcategory":null,"description":"whether the decision of an AI-based agent can be predicted in every situation or not","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"39.19.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Accountability","risk_subcategory":null,"description":"An essential feature of decision-making in humans, AI, and also HLI-based agents is accountability. Implementing this feature in machines is a difficult task because many challenges should be considered to organize an AI-based model that is accountable. It should be noted that this issue in human decision-making is not ideal, and many factors such as bias, diversity, fairness, paradox, and ambiguity may affect it. In addition, the human decision-making process is based on personal flexibility, context-sensitive paradigms, empathy, and complex moral judgments. Therefore, all of these challenges","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.4"},{"ev_id":"39.21.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Reproducibility","risk_subcategory":null,"description":"How a learning model can be reproduced when it is obtained based on various sets of data and a large space of parameters. This problem becomes more challenging in data-driven learning procedures without transparent instructions","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"39.25.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Verifiability","risk_subcategory":null,"description":"In many applications of AI-based systems such as medical healthcare and military services, the lack of verification of code may not be tolerable... due to some characteristics such as the non-linear and complex structure of AI-based solutions, existing solutions have been generally considered “black boxes”, not providing any information about what exactly makes them appear in their predictions and decision-making processes.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"39.27.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Complexity","risk_subcategory":null,"description":"Nowadays, we are faced with systems that utilize numerous learning models in their modules for their perception and decision-making processes... One aspect of an AI-based system that leads to increasing the complexity of the system is the parameter space that may result from multiplications of parameters of the internal parts of the system","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"40.03.00","quick_ref":"Yampolskiy2016","paper_title":"Taxonomy of Pathways to Dangerous Artificial Intelligence","level":"Risk Category","risk_category":"By Mistake - Pre-Deployment","risk_subcategory":null,"description":"\"Probably the most talked about source of potential problems with future AIs is mistakes in design. Mainly the concern is with creating a \"wrong AI\", a system which doesn't match our original desired formal properties or has unwanted behaviors (Dewey, Russell et al. 2015, Russell, Dewey et al. January 23, 2015), such as drives for independence or dominance. Mistakes could also be simple bugs (run time or logical) in the source code, disproportionate weights in the fitness function, or goals misaligned with human values leading to complete disregard for human safety.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.1"},{"ev_id":"40.04.00","quick_ref":"Yampolskiy2016","paper_title":"Taxonomy of Pathways to Dangerous Artificial Intelligence","level":"Risk Category","risk_category":"By Mistake - Post-Deployment","risk_subcategory":null,"description":"\"After the system has been deployed, it may still contain a number of undetected bugs, design mistakes, misaligned goals and poorly developed capabilities, all of which may produce highly undesirable outcomes. For example, the system may misinterpret commands due to coarticulation, segmentation, homophones, or double meanings in the human language (\"recognize speech using common sense\" versus \"wreck a nice beach you sing calm incense\") (Lieberman, Faaborg et al. 2005).\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"40.06.00","quick_ref":"Yampolskiy2016","paper_title":"Taxonomy of Pathways to Dangerous Artificial Intelligence","level":"Risk Category","risk_category":"Environment - Post-Deployment","risk_subcategory":null,"description":"\"While highly rare, it is known, that occasionally individual bits may be flipped in different hardware devices due to manufacturing defects or cosmic rays hitting just the right spot (Simonite March 7, 2008). This is similar to mutations observed in living organisms and may result in a modification of an intelligent system.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"41.03.00","quick_ref":"Allianz2018","paper_title":"The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks","level":"Risk Category","risk_category":"Mobility ","risk_subcategory":null,"description":"\"Despite the promise of streamlined travel, AI also brings concerns about who is liable in case of accidents and which ethical principles autonomous transportation agents should follow when making decisions with a potentially dangerous impact to humans, for example, in case of an accident.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"41.06.00","quick_ref":"Allianz2018","paper_title":"The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks","level":"Risk Category","risk_category":"Environment ","risk_subcategory":null,"description":"\"AI is already helping to combat the impact of climate change with smart technology and sensors reducing emissions. However, it is also a key component in the development of nanobots, which could have dangerous environmental impacts by invisibly modifying substances at nanoscale.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"41.06.01","quick_ref":"Allianz2018","paper_title":"The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks","level":"Risk Sub-Category","risk_category":"Environment ","risk_subcategory":"Accelerated development of nanotechnology produces uncontrolled production of toxic nanoparticles","description":"\"AI is a key component for the development of nanobots, which could have dangerous environmental implications by invisibly modifying substances at nanoscale. For example, nanobots could start chemical reactions that would create invisible nanoparticles that are toxic and potentially lethal.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"42.03.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Accuracy","risk_subcategory":null,"description":"\"The assessment of how often a system performs the correct prediction.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"42.04.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Moral","risk_subcategory":null,"description":"\"Less moral responsibility humans will feel regarding their life-or-death decisions with the increase of machines autonomy.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"42.05.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Bias","risk_subcategory":null,"description":"\"A systematic error, a tendency to learn consistently wrongly.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"42.06.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Opacity","risk_subcategory":null,"description":"\"Stems from the mismatch between mathematical optimization in high-dimensionality characteristic of machine learning and the demands of human-scale reasoning and styles of semantic interpretation.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"42.09.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Data Protection/Privacy","risk_subcategory":null,"description":"\"Vulnerable channel by which personal information may be accessed. The user may want their personal data to be kept private.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"42.11.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Protection","risk_subcategory":null,"description":"\"'Gaps' that arise across the development process where normal conditions for a complete specification of intended functionality and moral responsibility are not present.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"42.15.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Reliability","risk_subcategory":null,"description":"\"Reliability is defined as the probability that the system performs satisfactorily for a given period of time under stated conditions.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"43.01.05","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Safety & Trustworthiness","risk_subcategory":"Robustness","description":"\"These evaluations assess the quality, stability, and reliability of a LLM's performance when faced with unexpected, out-of-distribution or adversarial inputs. Robustness evaluation is essential in ensuring that a LLM is suitable for real-world applications by assessing its resilience to various perturbations.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"43.01.06","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Safety & Trustworthiness","risk_subcategory":"Data governance","description":"\"These evaluations assess the extent to which LLMs regurgitate their training data in their outputs, and whether LLMs 'leak' sensitive information that has been provided to them during use (i.e., during the inference stage).\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"44.03.00","quick_ref":"Coghlan2023 ","paper_title":"Harm to Nonhuman Animals from AI: a Systematic Account and Framework","level":"Risk Category","risk_category":"Unintentional: direct ","risk_subcategory":null,"description":"\"AI designed to benefit animals, humans, or ecosystems has unintended harmful impact on animals\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"44.03.01","quick_ref":"Coghlan2023 ","paper_title":"Harm to Nonhuman Animals from AI: a Systematic Account and Framework","level":"Risk Sub-Category","risk_category":"Unintentional: direct ","risk_subcategory":"AI is designed in a way that shows ignorant, reckless, or prejudiced lack of consideration for its impact on animals ","description":null,"entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"44.03.02","quick_ref":"Coghlan2023 ","paper_title":"Harm to Nonhuman Animals from AI: a Systematic Account and Framework","level":"Risk Sub-Category","risk_category":"Unintentional: direct ","risk_subcategory":"AI harms animals due to mistake or misadventure in the way the AI operates in practice ","description":null,"entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"44.04.00","quick_ref":"Coghlan2023 ","paper_title":"Harm to Nonhuman Animals from AI: a Systematic Account and Framework","level":"Risk Category","risk_category":"Unintentional: indirect ","risk_subcategory":null,"description":"\"AI impacts human or ecological systems in ways that ultimately harm animals\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"44.04.01","quick_ref":"Coghlan2023 ","paper_title":"Harm to Nonhuman Animals from AI: a Systematic Account and Framework","level":"Risk Sub-Category","risk_category":"Unintentional: indirect ","risk_subcategory":"Indirect Material Harms ","description":"\"AI proliferation causes harm to the environment through energy use and e-waste thereby destroying animal habitat\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"44.04.02","quick_ref":"Coghlan2023 ","paper_title":"Harm to Nonhuman Animals from AI: a Systematic Account and Framework","level":"Risk Sub-Category","risk_category":"Unintentional: indirect ","risk_subcategory":"Harms from Estrangement ","description":"\"Replacement by AI of human observation and interaction leads to neglect of certain interests\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"44.04.03","quick_ref":"Coghlan2023 ","paper_title":"Harm to Nonhuman Animals from AI: a Systematic Account and Framework","level":"Risk Sub-Category","risk_category":"Unintentional: indirect ","risk_subcategory":"Epistemic Harms ","description":"\"Algorithmic recommender systems reinforce and amplify anthropocentric bias or desire of some people for animal cruelty as entertainment — leading to greater harm to animals through reinforcement of meat eating from factory farms, cruel uses of animals for entertainment, etc\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"45.01.01","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"AI's inherent safety risks ","risk_subcategory":"Risks from models and algorithms (Risks of explainability)","description":"\"AI algorithms, represented by deep learning, have complex internal workings. Their black-box or grey-box inference process results in unpredictable and untraceable outputs, making it challenging to quickly rectify them or trace their origins for accountability should any anomalies arise.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.4"},{"ev_id":"45.01.05","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"AI's inherent safety risks ","risk_subcategory":"Risks from models and algorithms (Risks of unreliable output)","description":"\"Generative AI can cause hallucinations, meaning that an AI model generates untruthful or unreasonable content but presents it as if it were a fact, leading to biased and misleading information.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"45.01.09","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"AI's inherent safety risks ","risk_subcategory":"Risks from data (Risks of unregulated training data annotation)","description":"\"Issues with training data annotation, such as incomplete annotation guidelines, incapable annotators, and errors in annotation, can affect the accuracy, reliability, and effectiveness of models and algorithms. Moreover, they can introduce training biases, amplify discrimination, reduce generalization abilities, and result in incorrect outputs.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"45.02.03","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"Safety risks in AI Applications ","risk_subcategory":"Cyberspace risks (Risks of information leakage due to improper usage)","description":"\"Staff of government agencies and enterprises, if failing to use the AI service in a regulated and proper manner, may input internal data and industrial information into the AI model, leading to the leakage of work secrets, business secrets, and other sensitive business data.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"45.02.05","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"Safety risks in AI Applications ","risk_subcategory":"Cyberspace risks (Risks of security flaw transmission caused by model reuse)","description":"\"Re-engineering or fine-tuning based on foundation models is commonly used in AI applications. If security flaws occur in foundation models, it will lead to risk transmission to downstream models.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"45.02.09","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"Safety risks in AI Applications ","risk_subcategory":"Cognitive risks (Risks of amplifying the effects of \"information cocoons\")","description":"\"AI can be extensively utilized for customized information services, collecting user information, and analyzing types of users, their needs, intentions, preferences, habits, and even mainstream public awareness over a certain period. It can then be used to offer formulaic and tailored information and services, aggravating the effects of \"information cocoons.\"\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"45.02.12","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"Safety risks in AI Applications ","risk_subcategory":"Ethical Risks (Risks of challenging traditional social order)","description":"\"The development and application of AI may lead to tremendous changes in production tools and relations, accelerating the reconstruction of traditional industry modes, transforming traditional views on employment, fertility, and education, and bringing challenges to the stable performance of traditional social order.\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":5,"subdomain":"5.2"},{"ev_id":"47.01.00","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Category","risk_category":"Technical and operational risks ","risk_subcategory":null,"description":"\"To date, technical limitations and vulnerabilities are \npresent in most generative AI models in various contexts. Consequently, malicious users find it easier to breach \nan AI system’s safety and ethical guardrails to execute \nharmful actions.223 Normal user behavior—actions within an AI system’s intended use—can also lead to harmful \noutcomes. Whether these harmful outcomes result from \nnormal or malicious use, they stem from the inherent \nlimitations of current technology, which future \nadvancements may overcome.\nThis section examines the technical vulnerabilities that \ncan affect AI models","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"47.01.04","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Technical and operational risks ","risk_subcategory":"Factually incorrect content (inaccuracies and fabricated sources) ","description":"\"One of the most vexing problems associated with AI models is that they occasionally present false information as if it is factual—often with authoritative-sounding text and fabricated quotes and sources. This unpredictable phenomenon of generating false information is well known to AI researchers, who have termed such erroneous output with the euphemistic label “hallucination.” \"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"47.01.05","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Technical and operational risks ","risk_subcategory":"Opacity (the black box problem)","description":"\"Opacity surrounding the technical, internal decision-making processes of generative AI models is popularly known as the “black box problem.”277 Generative AI models, most ubiquitously built on deep neural networks with hundreds of billions of internal connections,278 have become so complex that their internal decision-making processes are no longer traceable or interpretable to even the most advanced expert observers. This means that, while the inputs and outputs of a system can be observed, developers cannot explain in detail why specific inputs correspond to specific outputs.\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.4"},{"ev_id":"47.02.08","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Bias and discrimination (bias in training datasets) ","description":"\"AI experts consider training data to be the most salient source of bias in generative AI models. For example, GPT- 2’s training data comes from outbound links from Reddit, a social network often criticized for hosting anti-feminist content.351 As a result, AI models trained on such data are more likely to produce outputs that reflect these biases.\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"47.02.09","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Bias and discrimination (value embedding) ","description":"\"Generative AI models may also be subject to the “value embedding” phenomenon.361 “Value embedding” refers to the fact that developers of generative AI models strive to minimize biased outputs by retraining their models based on normative values.362 Contemporary state-of- the-art models not only reflect the values embedded within their training data, they also undergo additional fine-tuning that follows a set of chosen rules and principles. Due to the absence of universally accepted standards, developers bear the responsibility of making decisions on sensitive issues. These practices lead to c","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.3"},{"ev_id":"47.02.10","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Bias and discrimination (value lock and outcome homogenization) ","description":"\"Because models are not necessarily retrained to reflect evolving societal views, language models risk “value lock- ins,” which “reifies older, less inclusive understandings.”370 Therefore, the continued use of outdated models may limit the presentation or exploration of alternative perspectives. Moreover, the deployment of identical foundation models by various downstream deployers poses a risk of “outcome homogenization,” creating a potential for homogeneity of bias across broad swathes of society. Identical and widely deployed models with prejudicial training datasets could further entrench","entity":"Human","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"47.02.11","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Influence, overreliance and dependence (influence and manipulation) ","description":"\"Despite the widely recognized potential of generative AI tools to “hallucinate” or produce harmful content, such tools can exert a noteworthy influence on the humans who engage with them. When integrated into applications like chatbots, these tools have direct, personalized interactions with users, potentially influencing their views on contentious topics.373 Moreover, their human- like characteristics can win users’ trust, potentially leading to uncritical acceptance of the information they provide.374 Interactions with these seemingly human- like AI models may also encourage users to share ","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"47.02.12","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Influence, overreliance and dependence (overreliance) ","description":"\"Beyond being simply influenced, humans may become overreliant on generative AI. Researchers with Microsoft’s AETHER (AI Ethics and Effects in Engineering and Research) define overreliance as users “accepting incorrect AI recommendations” or “making errors of commission” because they are “unable to determine whether or how much they should trust the AI.”","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"47.02.13","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Influence, overreliance and dependence (emotional dependence) ","description":"\"Humans might become dependent on generative AI tools in ways similar to their emotional dependence on other technologies, such as smartphones or social networks.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"47.03.01","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Legal challenges ","risk_subcategory":"Privacy and data collection concerns (collecting personal information or personally identifiable information) ","description":"\"Generative AI developers train their models with extensive datasets often gathered through online web scraping of websites that may include personal data or personally identifiable information (PII). For most generative AI applications, such as initial model training, the primary concerns are the quantity, variety, and quality of the data, not whether they include personally identifiable information. However, some web-scraped datasets may inadvertently include personal data. Additionally, when downstream developers integrate generative AI into their products or services by fine- tuning a pre-","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"47.03.02","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Legal challenges ","risk_subcategory":"Privacy and data collection concerns (data protection concerns) ","description":"\"The incorporation of personal data within training datasets raises numerous concerns. The primary issue is that personal data may be incorporated without the knowledge or consent of the individuals concerned, even though the data may include names, identification numbers, Social Security numbers, or other personal information. Another particularly difficult problem is related to the fact that complex models may “memorize” (i.e., store) specific threads of training data and regurgitate them when responding to a prompt.498 This data memorization can directly lead to leakage of personal data. Ev","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"47.03.04","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Legal challenges ","risk_subcategory":"Copyright challenges (copyright-infringing output) ","description":"\"Even though models generally create new outputs, it is possible that the content produced by a generative AI tool—such as an image, or even computer code— could turn out to be almost identical to that used in the training data. Given that generative AI models tend to memorize fragments of their training data, they might reproduce these fragments, potentially leading to charges of copyright infringement.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"47.04.03","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Environmental, economical, and societal challenges ","risk_subcategory":"Impact on labor markets (job loss and displacement) ","description":"\"Currently, a significant share of workers (three in five) worry about losing their jobs entirely to AI in the next 10 years—particularly those who already work with AI. Some studies conclude that AI tools (generative and non-generative) will create significant job losses.573 The OECD has found that occupations at highest risk of being lost to automation from AI account for about 27% of employment.5\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.2"},{"ev_id":"47.04.04","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Environmental, economical, and societal challenges ","risk_subcategory":"Impact on labor markets (rising inequalities) ","description":"\"AI is more likely to displace workers when it is designed to replicate human skills and intelligence.597 In such cases, there is a risk of concentrating wealth and power in the hands of a few individuals or organizations that control the capital. In addition, ordinary people, including those with significant expertise, may become less valued because machines would be performing their roles. This shift could lower wages, reduce the value of human work, and exacerbate economic inequality.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.3"},{"ev_id":"47.04.05","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Environmental, economical, and societal challenges ","risk_subcategory":"Environmental cost (energy consumption) ","description":"\"Training large AI models requires a substantial amount of computing power to handle vast datasets, which translates into high energy consumption.\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"47.04.06","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Environmental, economical, and societal challenges ","risk_subcategory":"Environmental cost (water consumption) ","description":"\"Data centers use water for cooling to prevent servers from overheating. The water consumption associated with AI training and inference processes can be substantial, impacting local water resources.\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"47.04.07","quick_ref":"G'sell2025","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Environmental, economical, and societal challenges ","risk_subcategory":"Artificial general intelligence (existential risk posed by Artificial General Intelligence) ","description":"\"In a paper called “How Does Artificial Intelligence Pose an Existential Risk?” published in 2017, Karina Vold and Daniel Harris suggested that humans might create a super-intelligent machine that could outsmart all other intelligences, remain beyond human control, and potentially engage in actions that are contrary to human interests.635 The prevailing narrative surrounding AI existential risk typically lies in the possibility of developing “Artificial General Intelligence” (AGI), or artificial super- intelligence (ASI).\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.2"},{"ev_id":"48.02.00","quick_ref":"NIST2024","paper_title":"Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile","level":"Risk Category","risk_category":"Confabulation ","risk_subcategory":null,"description":"\"The production of confidently stated but erroneous or false content (known colloquially as “hallucinations” or “fabrications”) by which users may be misled or deceived.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"48.04.00","quick_ref":"NIST2024","paper_title":"Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile","level":"Risk Category","risk_category":"Data Privacy ","risk_subcategory":null,"description":"\"Impacts due to leakage and unauthorized use, disclosure, or de-anonymization of biometric, health, location, or other personally identifiable information or sensitive data.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"48.05.00","quick_ref":"NIST2024","paper_title":"Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile","level":"Risk Category","risk_category":"Environmental Impacts ","risk_subcategory":null,"description":"\"Impacts due to high compute resource utilization in training or operating GAI models, and related outcomes that may adversely impact ecosystems.\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"48.06.00","quick_ref":"NIST2024","paper_title":"Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile","level":"Risk Category","risk_category":"Harmful Bias or Homogenization ","risk_subcategory":null,"description":"\"Amplification and exacerbation of historical, societal, and systemic biases; performance disparities8 between sub-groups or languages, possibly due to non-representative training data, that result in discrimination, amplification of biases, or incorrect presumptions about performance; undesired homogeneity that skews system or model outputs, which may be erroneous, lead to ill-founded decision-making, or amplify harmful biases.\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"48.07.00","quick_ref":"NIST2024","paper_title":"Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile","level":"Risk Category","risk_category":"Human-AI Configuration ","risk_subcategory":null,"description":"\"Arrangement s of or interactions between a human and an AI system \nwhich can result in the human inappropriately anthropomorphizing GAI systems or experiencing algorithmic aversion, automation bias, over-reliance, or emotional entanglement with GAI \nsystems.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"48.12.00","quick_ref":"NIST2024","paper_title":"Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile","level":"Risk Category","risk_category":"Value Chain and Component Integration ","risk_subcategory":null,"description":"\"Non-transparent or untraceable integration of \nupstream third-party components, including data that has been improperly obtained or not \nprocessed and cleaned due to increased automation from GAI; improper supplier vetting across the AI lifecycle; or other issues that diminish transparency or accountability for downstream \nusers.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"49.02.00","quick_ref":"Bengio2024","paper_title":"International Scientific Report on the Safety of Advanced AI","level":"Risk Category","risk_category":"Risks from Malfunctions ","risk_subcategory":null,"description":"None provided. ","entity":"Other","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.0"},{"ev_id":"49.02.01","quick_ref":"Bengio2024","paper_title":"International Scientific Report on the Safety of Advanced AI","level":"Risk Sub-Category","risk_category":"Risks from Malfunctions ","risk_subcategory":"Risks from product functionality issues","description":"\"Product functionality issues occur when there is confusion or misinformation about what a general- purpose AI model or system is capable of. This can lead to unrealistic expectations and overreliance on general- purpose AI systems, potentially causing harm if a system fails to deliver on expected capabilities. These functionality misconceptions may arise from technical difficulties in assessing an AI model's true capabilities on its own,or predicting its performance when part of a larger system. Misleading claims in advertising and communications can also contribute to these misconceptions.\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":5,"subdomain":"5.1"},{"ev_id":"49.02.02","quick_ref":"Bengio2024","paper_title":"International Scientific Report on the Safety of Advanced AI","level":"Risk Sub-Category","risk_category":"Risks from Malfunctions ","risk_subcategory":"Risks from bias and underrepresentation","description":"\"The outputs and impacts of general- purpose AI systems can be biased with respect to various aspects of human identity, including race, gender, culture, age, and disability. This creates risks in high- stakes domains such as healthcare, job recruitment, and financial lending. General- purpose AI systems are primarily trained on language and image datasets that disproportionately represent English- speaking and Western cultures, increasing the potential for harm to individuals not represented well by this data.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"49.03.02","quick_ref":"Bengio2024","paper_title":"International Scientific Report on the Safety of Advanced AI","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Global AI Divide ","description":"\"General- purpose AI research and development is currently concentrated in a few Western countries and China. This ‘AI Divide’ is multicausal, but in part related to limited access to computing power in low- income countries. Access to large and expensive quantities of computing power has become a prerequisite for developing advanced general- purpose AI. This has led to a growing dominance of large technology companies in general- purpose AI development. The AI R&D divide often overlaps with existing global socioeconomic disparities, potentially exacerbating them.\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"49.03.04","quick_ref":"Bengio2024","paper_title":"International Scientific Report on the Safety of Advanced AI","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Risks to the environment","description":"\"Growing compute use in general- purpose AI development and deployment has rapidly increased energy usage associated with general- purpose AI. This trend might continue, potentially leading to strongly increasing CO2 emissions.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"49.03.05","quick_ref":"Bengio2024","paper_title":"International Scientific Report on the Safety of Advanced AI","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Risks to privacy","description":"\"General- purpose AI models or systems can ‘leak’ information about individuals whose data was used in training. For future models trained on sensitive personal data like health or financial data, this may lead to particularly serious privacy leaks. General- purpose AI models could enhance privacy abuse. For instance, Large Language Models might facilitate more efficient and effective search for sensitive data (for example, on internet text or in breached data leaks), and also enable users to infer sensitive information about individuals.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"50.02.04","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Content Safety Risks ","risk_subcategory":"Violence and extremism (Depicting violence) ","description":null,"entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"50.02.16","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Content Safety Risks ","risk_subcategory":"Child Harm (Child Sexual Abuse)","description":null,"entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"50.02.17","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Content Safety Risks ","risk_subcategory":"Self-harm (Suidical and non-suicidal self injury)","description":null,"entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"50.03.05","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Societal Risks ","risk_subcategory":"Economic harm (High-Risk Financial Activities) ","description":null,"entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"51.03.00","quick_ref":"Everitt2018 ","paper_title":"AGI Safety Literature Review ","level":"Risk Category","risk_category":"Corrigibility ","risk_subcategory":null,"description":"\"If we get something wrong in the design or construction of an agent, will the agent cooperate in us trying to fix it? This is called error-tolerant design by MIRI-AF and corrigibility by Soares, Fallenstein, et al. (2015). The problem is connected to safe interruptibility as considered by DeepMind.\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"51.04.00","quick_ref":"Everitt2018 ","paper_title":"AGI Safety Literature Review ","level":"Risk Category","risk_category":"Security ","risk_subcategory":null,"description":"\"How to design AGIs that are robust to adversaries and adversarial environ-\nments? This involves building sandboxed AGI protected from adversaries (Berkeley), and agents that are robust to adversarial inputs (Berkeley, DeepMind).\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"51.05.00","quick_ref":"Everitt2018 ","paper_title":"AGI Safety Literature Review ","level":"Risk Category","risk_category":"Safe learning ","risk_subcategory":null,"description":"\"AGIs should avoid making fatal mistakes during the learning phase.\nSubproblems include safe exploration and distributional shift (DeepMind, OpenAI), and continual learning (Berkeley).\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"51.06.00","quick_ref":"Everitt2018 ","paper_title":"AGI Safety Literature Review ","level":"Risk Category","risk_category":"Intelligibility ","risk_subcategory":null,"description":"\"How can we build agent’s whose decisions we can understand? Con-\nnects explainable decisions (Berkeley) and informed oversight (MIRI).\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"51.12.00","quick_ref":"Everitt2018 ","paper_title":"AGI Safety Literature Review ","level":"Risk Category","risk_category":"Meta-cognition ","risk_subcategory":null,"description":"\"Agents that reason about their own computational resources and logically uncertain events can encounter strange paradoxes due to Godelian limitations (Fallenstein and Soares, 2015; Soares and Fallenstein, 2014, 2017) and shortcomings of probability theory (Soares and Fallenstein, 2014, 2015, 2017). They may also be reflectively unstable, preferring to change the principles by which they select actions (Arbital, 2018).\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"52.01.00","quick_ref":"Maham2023 ","paper_title":"Governing General Purpose AI: A Comprehensive Map of Unreliability, Misuse and Systemic Risks ","level":"Risk Category","risk_category":"Risks from Unreliability ","risk_subcategory":null,"description":"\"Risks from Unreliability stem from general purpose AI models that lack reliability, robustness, transparency, corrigibility, and interpretability, making it challenging to predict and control their behaviour fully. This includes Discrimination and Stereotype Reproduction, Misinformation and Privacy Violations, and Accidents.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"52.01.01","quick_ref":"Maham2023 ","paper_title":"Governing General Purpose AI: A Comprehensive Map of Unreliability, Misuse and Systemic Risks ","level":"Risk Sub-Category","risk_category":"Risks from Unreliability ","risk_subcategory":"Discrimination and Stereotype Reproduction","description":"\"General purpose AI models interpret and respond to inputs based on their training data, potentially causing Discrimination and Stereotype Reproduction. Since they are “black-box” models, the exact mechanism behind decisions remains opaque and attempts to mitigate harmful outputs are not fully reliable yet. These models have the capacity to influence a multitude of downstream applications, decisions, and processes, thereby affecting many individuals simultaneously. The extent of this impact could outstrip the range of any single human or group of humans, amplifying the potential consequences o","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"52.01.02","quick_ref":"Maham2023 ","paper_title":"Governing General Purpose AI: A Comprehensive Map of Unreliability, Misuse and Systemic Risks ","level":"Risk Sub-Category","risk_category":"Risks from Unreliability ","risk_subcategory":"Misinformation and Privacy Violations","description":"\"Due to their unreliability, general purpose AI models might disseminate false or misleading information, omit critical information, or convey true information that violates privacy rights.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"52.01.03","quick_ref":"Maham2023 ","paper_title":"Governing General Purpose AI: A Comprehensive Map of Unreliability, Misuse and Systemic Risks ","level":"Risk Sub-Category","risk_category":"Risks from Unreliability ","risk_subcategory":"Accidents ","description":"\"As general purpose AI models as “black-box” models are not fully controllable and understandable, even to their developers, unexpected failures could arise from their unreliability. This could lead to accidents106 if they are connected to any real-world systems, during their development, testing or deployment.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"52.03.00","quick_ref":"Maham2023 ","paper_title":"Governing General Purpose AI: A Comprehensive Map of Unreliability, Misuse and Systemic Risks ","level":"Risk Category","risk_category":"Systemic Risks ","risk_subcategory":null,"description":"\"In addition to risks stemming from the unreliability or misuse of general purpose AI models, further Systemic Risks can originate from the centralisation of general purpose AI development as well as the rapid integration of these models into our lives.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"52.03.03","quick_ref":"Maham2023 ","paper_title":"Governing General Purpose AI: A Comprehensive Map of Unreliability, Misuse and Systemic Risks ","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Disruptions from Outpaced Societal Adaptation","description":"\"Although the implementation of general purpose AI models as automation tools could be a major opportunity, overly rapid adoption of this technology at scale might outpace the ability of society to adapt effectively. This could lead to a variety of disruptions, including challenges in the labour market, the education system and public discourse, and various mental health concerns.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"53.01.00","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Category","risk_category":"Alignment failures in existing ML systems ","risk_subcategory":null,"description":"-","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"53.01.01","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Sub-Category","risk_category":"Alignment failures in existing ML systems ","risk_subcategory":"Faulty reward functions in the wild ","description":"-","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.1"},{"ev_id":"53.01.05","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Sub-Category","risk_category":"Alignment failures in existing ML systems ","risk_subcategory":"Goal misgeneralization ","description":"-","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"53.02.01","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Sub-Category","risk_category":"Dangerous capabilities in AI systems ","risk_subcategory":"Situational awareness ","description":"\"cases where a large language model displays awareness that it is a model, and it can recognize whether it is currently in testing or deployment;\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.2"},{"ev_id":"53.03.02","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Sub-Category","risk_category":"Direct catastrophe from AI ","risk_subcategory":"Gradual, irretrievable ceding of human power over the future to AI systems","description":"-","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"53.03.05","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Sub-Category","risk_category":"Direct catastrophe from AI ","risk_subcategory":"Dystopian trajectory lock-in because of misuse of advanced AI to establish and/or maintain totalitarian regimes;","description":"-","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"53.04.00","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Category","risk_category":"Indirect AI contributions to existential risks","risk_subcategory":null,"description":"\"Work focused at understanding indirect ways in which AI could contribute to existential threats, such as by shaping societal “turbulence”193 and other existential risk factors.194 This covers various long-term impacts on societal parameters such as science, cooperation, power, epistemics, and values:\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"53.04.01","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Sub-Category","risk_category":"Indirect AI contributions to existential risks","risk_subcategory":"Destabilising political impacts from AI systems ","description":"\"(e.g., polarization, legitimacy of elections), international political economy, or international security196 in terms of the balance of power, technology races and international stability, and the speed and character of war\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.4"},{"ev_id":"53.04.03","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Sub-Category","risk_category":"Indirect AI contributions to existential risks","risk_subcategory":"Impacts on “epistemic security” and the information environment","description":"-","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"53.04.04","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Sub-Category","risk_category":"Indirect AI contributions to existential risks","risk_subcategory":"Erosion of international law and global governance architectures;","description":"-","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"54.01.02","quick_ref":"Leech2024 ","paper_title":"Ten Hard Problems in Artificial Intelligence We Must Get Right","level":"Risk Sub-Category","risk_category":"Negative impacts of AI use ","risk_subcategory":"Environmental cost ","description":"\"Large-scale DL systems can produce signicant carbon emissions as a result of the computational demands of training runs and inference [539]\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"54.01.03","quick_ref":"Leech2024 ","paper_title":"Ten Hard Problems in Artificial Intelligence We Must Get Right","level":"Risk Sub-Category","risk_category":"Negative impacts of AI use ","risk_subcategory":"Discrimination, toxicity, and bias ","description":"\"AI models and the tools that use them may exacerbate unequal access to employment and services. AI-generated content can promote inequality and harmful stereotypes.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"54.02.06","quick_ref":"Leech2024 ","paper_title":"Ten Hard Problems in Artificial Intelligence We Must Get Right","level":"Risk Category","risk_category":"Harm caused by incompetent systems ","risk_subcategory":null,"description":"\"While HP#1 concerns mean or best-case performance, HP#2 concerns worst-case performance: how can we ensure that AI systems will perform safely, and how can we prove this? ML systems have been implemented in high-stakes, safety-critical domains such as driving [182], medicine [113], and warfare [298]. Many more systems have been developed but have remained undeployed or been rolled back as a result of regulatory and safety reasons [471]. Clearly, unsafe systems can result in loss of life, economic damage, and social unrest [407, 10]. Most concerningly, AI systems may be susceptible to so-calle","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"55.01.01","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Risks from accelerating scientific progress ","risk_subcategory":"Eased development of technologies that make a global catastrophe more likely ","description":null,"entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"55.01.02","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Risks from accelerating scientific progress ","risk_subcategory":"Faster scientific progress makes it harder for governance to keep pace with development ","description":"\"Exacerbating these problems is that faster scientific progress would make it even harder for governance to keep pace with the deployment of new technologies. When these technologies are especially powerful or dangerous, such as those discussed above, insufficient governance can magnify their harms.8 This is known as the pacing problem, and it is an issue that technology governance already faces [47], for a variety of reasons\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"55.02.02","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Worsened conflict ","risk_subcategory":"AI enables automation of military decision-making ","description":"\"One concern here is humans not remaining in the loop for some military decisions, creating the possibility of unintentional escalation because of: • Automated tactical decision-making, by ‘in-theatre’ AI systems (e.g. border patrol systems start accidentally firing on one another), leading to either: tactical-level war crimes,11 or strategic-level decisions to initiate conflict or escalate to a higher level of intensity—for example, countervalue (e.g. city-) targeting, or going nuclear [62]. • Automated strategic decision-making, by ‘out-of-theatre’ AI systems—for example, conflict prediction","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"55.02.03","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Worsened conflict ","risk_subcategory":"AI-induced strategic instability ","description":"\"For example, AI could undermine nuclear strategic stability by making it easier to discover and destroy previously secure nuclear launch facilities [30, 46, 49]. AI may also offer more extreme first-strike advantages or novel destructive capabilities that could disrupt deterrence, such as cyber capabilities being used to knock out opponents’ nuclear command and control [15, 29]. The use of AI capabilities may make it less clear where attacks originate from, making it easier for aggressors to obfuscate an attack, and therefore reducing the costs of initiating one. By making it more difficult t","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"55.02.04","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Worsened conflict ","risk_subcategory":"Resource conflicts driven by AI development ","description":"\"AI development may itself become a new flash point for conflicts—causing more conflict to occur— especially conflicts over AI-relevant resources (such as data centres, semiconductor manufacturing facilities and raw materials).\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.4"},{"ev_id":"55.03.00","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Category","risk_category":"Increased power concentration and inequality ","risk_subcategory":null,"description":"\"Power and inequality: there are a lot of pathways through which AI seems likely to increase power concentration and inequality, though there is little analysis of the potential long- term impacts of these pathways. Nonetheless, AI precipitating more extreme power concentration and inequality than exists today seems a real possibility on current trends.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"55.04.00","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Category","risk_category":"Worsened epistemic processes for society ","risk_subcategory":null,"description":"\"Epistemic processes and problem solving: we currently see more reasons to be concerned about AI worsening society's epistemic processes than reasons to be optimistic about AI helping us better solve problems as a society. For example, increased use of content selection algorithms could drive epistemic insularity and a decline in trust in credible multipartisan sources, which reducing our ability to deal with important long-term threats and challenges such as pandemics and climate change.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"55.04.01","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Worsened epistemic processes for society ","risk_subcategory":"AI contributes to increased online polarisation ","description":"\"One of the most significant commercial uses of current AI systems is in the content recommendation algorithms of social media companies, and there are already concerns that this is contributing to worsened polarisation online\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"55.04.04","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Worsened epistemic processes for society ","risk_subcategory":"Widespread use of persuasive tools contributes to splintered epistemic communities ","description":"\"Even without deliberate misuse, widespread use of powerful persuasion tools could have negative impacts. If such tools were used by many different groups to advance many different ideas, we could see the world splintering into isolated “epistemic communities”, with little room for dialogue or transfer between communities. A similar scenario could emerge via the increasing personalisation of people’s online experiences—in other words, we may see a continuation of the trend towards “filter bubbles” and “echo chambers”, driven by content selection algorithms, that some argue is already happening","entity":"Human","intent":"Unintentional","timing":"Other","domain":3,"subdomain":"3.2"},{"ev_id":"55.05.00","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Category","risk_category":"AI leads to humans losing control of the future","risk_subcategory":null,"description":"\"The values that steer humanity’s future: humanity gaining more control over the future due to developments in AI, or losing our potential for gaining control, both seem possible. Much will depend on our ability to solve the alignment problem, who develops powerful AI first, and what they use it for. These long-term impacts of AI could be hugely important but are currently under-explored. We’ve attempted to structure some of the discussion and stimulate more research, by reviewing existing arguments and highlighting open questions. While there are many ways AI could in theory enable a flourish","entity":"Human","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"56.01.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Discrimination","risk_subcategory":null,"description":"\"More broadly, bad decisions or errors by AI tools could lead to discrimination or deeper inequality\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"56.02.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Inequality","risk_subcategory":null,"description":"\"More broadly, bad decisions or errors by AI tools could lead to discrimination or deeper inequality\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"56.03.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Environmental impacts ","risk_subcategory":null,"description":"\"Increasing use of AI systems, and their growing energy needs, could also have environmental impacts. All of these could become more acute as AI becomes more capable.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"56.04.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Amplification of biases","risk_subcategory":null,"description":"\"Current Frontier AI mdoels amplify existing biases within their training data and can be manipulated into providing potentially harmful responses, for example abusive language or discriminatory responses91,92. This is not limited to text generation but can be seen across all modalities of generative AI93. Training on large swathes of UK and US English internet content can mean that misogynistic, ageist, and white supremacist content is overrepresented in the training data94.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"56.05.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Harmful responses ","risk_subcategory":null,"description":"\"Current Frontier AI mdoels amplify existing biases within their training data and can be manipulated into providing potentially harmful responses, for example abusive language or discriminatory responses91,92. This is not limited to text generation but can be seen across all modalities of generative AI93. Training on large swathes of UK and US English internet content can mean that misogynistic, ageist, and white supremacist content is overrepresented in the training data94.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"56.06.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Lack of transparency and interpretability ","risk_subcategory":null,"description":"\"Today's Frontier AI is difficult to interpret and lacks transparency. Contextual understanding of the training data is not explicitly embedded within these models. They can fail to capture perspectives of underrepresented groups or the limitations within which they are expected to perform without fine tuning or reinforcement learning with human feedback (RLHF).\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"56.09.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Misapplication by a non-malicious actor ","risk_subcategory":null,"description":null,"entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"56.10.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Poor performance of a model used for its intended purpose, for example leading to biased decisions ","risk_subcategory":null,"description":null,"entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"56.11.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Unintended outcomes from interactions with other AI systems ","risk_subcategory":null,"description":null,"entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"56.14.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Overreliance on AI systems, which cannot be subsequently unpicked ","risk_subcategory":null,"description":null,"entity":"Human","intent":"Unintentional","timing":"Other","domain":5,"subdomain":"5.1"},{"ev_id":"56.15.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Societal concerns around AI reduce the realisation of potential benefits ","risk_subcategory":null,"description":null,"entity":"Human","intent":"Unintentional","timing":"Other","domain":null,"subdomain":null},{"ev_id":"56.17.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Single point of failure ","risk_subcategory":null,"description":"\"Intense competition leads to one company gaining a technical edge, exploiting this to the point its model controls, or is the basis for other models controlling, multiple key systems. Lack of safety, controllability, and misuse cause these systems to fail in unexpected ways.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"56.18.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Overreliance","risk_subcategory":null,"description":"\"As AI capability increases, humans grant AI more control over critical systems and eventually become irreversibly dependent on systems they don’t fully understand. Failure and unintended outcomes cannot be controlled.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":5,"subdomain":"5.2"},{"ev_id":"58.03.01","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Psychological ","risk_subcategory":"Addiction ","description":"\"Addiction - Emotional or material dependence on technology or a technology system.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"58.03.02","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Psychological ","risk_subcategory":"Alienation/isolation ","description":"\"Alienation/isolation - An individual’s or group’s feeling of lack of connection with those around as a result of technology use or misuse.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"58.03.07","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Psychological ","risk_subcategory":"Overreliance ","description":"\"Over-reliance - Unfettered and/or obsessive belief in the accuracy or other quality of a technology system, resulting in addiction, anxiety, introversion, sentience, complacency, lack of critical thinking and other actual or potential negative impacts.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"58.09.01","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Environmental ","risk_subcategory":"Biodiversity loss ","description":"\"Biodiversity loss - Over-expansion of technology infrastructure, or inadequate alignment of technology with sustainable practices, leading to deforestation, habitat destruction, and fragmentation and loss of biodiversity.\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"58.09.04","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Environmental ","risk_subcategory":"Excessive energy consumption ","description":"\"Excessive energy consumption - Excessive energy use, leading to energy bottlenecks and shortages for communities, organisations, and businesses.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"58.09.05","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Environmental ","risk_subcategory":"Excessive landfill ","description":"\"Excessive landfill - Excessive disposal of electrical or electronic equipment leading to ecological/biodiversity damage, and disrupting the livelihoods and eroding the rights of local communities.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"58.09.06","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Environmental ","risk_subcategory":"Excessive water consumption ","description":"\"Excessive water consumption - Excessive use of water to cool data centres and for other purposes, leading to water restrictions or shortages for local communities or businesses.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"59.01.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Inadequate specification of ODD","risk_subcategory":null,"description":"\"The operational design domain (ODD) is a technical description of the application’s operational environment, initially conceptualized for autonomous driving systems. An inadequate specification of the ODD limits essential functions such as testing the learned functionality and out-of-distribution detection.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.02.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Inappropriate degree of automation","risk_subcategory":null,"description":"\"The AI application’s degree of automation ranges from no automation to fully autonomous. AI applications with a high degree of automation may exhibit unexpected behaviour and pose risks in terms of their reliability and safety.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.2"},{"ev_id":"59.03.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Inadequate planning of performance requirements","risk_subcategory":null,"description":"\"The expected performance of the AI system should be planned adequately. Hereby, an important aspect is that chosen performance metrics are meaningful for presenting the intended functionality. Otherwise, expectations and safety requirements can be unfulfillable at later life cycle stages.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.07.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Choice of untrustworthy data source","risk_subcategory":null,"description":"\"The choice of a trustworthy data source is a first prerequisite in order to fulfill data quality requirements. This is especially the case if third-party data sources are used to develop the AI system.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"59.08.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Lack of data understanding","risk_subcategory":null,"description":"\"The correct understanding of the used data for developing an AI system is a prerequisite to avoid data shortcomings and hinders the development of an AI system which is best suiting for the intended functionality.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"59.09.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Discriminative data bias","risk_subcategory":null,"description":"\"Discriminative data bias describes the systematic discrimination of groups of persons in the form of data shortcomings, such as distributional representation or incorrectness. Data bias can manifest in the model and lead to unfair decisions if not appropriately treated. Note, that the term bias is often used in other contexts, such as data representation. However, these issues are treated by other AI hazards in this list.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"59.11.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Incorrect data labels","risk_subcategory":null,"description":"\"Data labels are essential for any supervised learning algorithm since they preset the result of the learning process. If the correctness of the data labels is not given, the AI system is prevented from learning the ground truth and therefore the intended functionality.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.16.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Poor model design choices","risk_subcategory":null,"description":"\"The model specifications have significant impact on the functionality of an AI system. The developer mak- ing wrong decisions might cause the AI system to behave biased and unreliable.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.19.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Unreliability in corner cases","risk_subcategory":null,"description":"\"AI systems tend to show unreliable behavior when confronted with rare or ambiguous input data, also called corner cases. Therefore, the controlled behavior is required whenever the AI system is faces a corner case.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"59.20.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Lack of robustness","risk_subcategory":null,"description":"\"Robustness characterizes the resilience of an AI system’s output against minor changes in the input domain. A great variation in an AI system’s response to small input changes indicates unreliable outputs.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"59.21.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Uncertainty concerns","risk_subcategory":null,"description":"\"AI systems should be able not only to return output for a given instance but also to provide a corresponding level of confidence. If such a method is not implemented or not working correctly, this can have a negative impact on performance and safety.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"59.22.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Operational data issues","risk_subcategory":null,"description":"\"Until the deployment of the AI application into its operational environment, the AI system has been tested with a test set that aims to approximate the distribution of operational data. However, an unexpected deviation in this approximation can cause an AI application to behave unreliably. Therefore, its behavior under confrontation with operational data needs to be evaluated.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.26.01","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Sub-Category","risk_category":"Mode","risk_subcategory":"Technical ","description":"\"Technical AI hazards are the root causes of technical deficiencies in the AI system. An example of such an AI hazard is overfitting, which describes a model’s excessive adaptation to the training dataset. Quantitative methods to assess (metrics) and treat (mitigation means) exist for technical AI hazards, which might be performed automatically. In case of overfitting, metrics are based on the comparison of performance between the training and validation datasets, and mitigation means may include regularization techniques, among others.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.26.03","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Sub-Category","risk_category":"Mode","risk_subcategory":"Procedural ","description":"\"The third class encompasses procedural AI hazards. These pertain to issues arising from processes and actions made by individuals involved in the develop- ment process. Such hazards are not readily quantifiable and necessitate alter- native mitigation strategies. An example of such an AI hazard would be ”poor model design choices,” which could be expressed, for instance, through a devel- oper’s decision to select an unsuitable AI model for a given problem. Due to the challenges in quantifying and mitigating these issues, qualitative approaches must be employed. In the case of the aforemention","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"60.02.01","quick_ref":"Bengio2025","paper_title":"International AI Safety Report 2025","level":"Risk Sub-Category","risk_category":"Risks from malfunctions ","risk_subcategory":"Reliability issues ","description":"\"Relying on general-purpose AI products that fail to fulfil their intended function can lead to harm. For example, general- purpose AI systems can make up facts (‘hallucination’), generate erroneous computer code, or provide inaccurate medical information. This can lead to physical and psychological harms to consumers and reputational, financial and legal harms to individuals and organisations.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"60.02.02","quick_ref":"Bengio2025","paper_title":"International AI Safety Report 2025","level":"Risk Sub-Category","risk_category":"Risks from malfunctions ","risk_subcategory":"Bias ","description":"\"General-purpose AI systems can amplify social and political biases, causing concrete harm. They frequently display biases with respect to race, gender, culture, age, disability, political opinion, or other aspects of human identity. This can lead to discriminatory outcomes including unequal resource allocation, reinforcement of stereotypes, and systematic neglect of certain groups or viewpoints.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"60.03.04","quick_ref":"Bengio2025","paper_title":"International AI Safety Report 2025","level":"Risk Sub-Category","risk_category":"Systemic risks ","risk_subcategory":"Risks to the environment","description":"\"General- purpose AI is a moderate but rapidly growing contributor to global environmental impacts through energy use and greenhouse gas (GHG) emissions. Current estimates indicate that data centres and data transmission account for an estimated 1% of global energy- related GHG emissions, with AI consuming 10–28% of data centre energy capacity. AI energy demand is expected to grow substantially by 2026, with some estimates projecting a doubling or more, driven primarily by general-purpose AI systems such as language models.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"60.03.05","quick_ref":"Bengio2025","paper_title":"International AI Safety Report 2025","level":"Risk Sub-Category","risk_category":"Systemic risks ","risk_subcategory":"Risks to privacy ","description":"\"General- purpose AI systems can cause or contribute to violations of user privacy. Violations can occur inadvertently during the training or usage of AI systems, for example through unauthorised processing of personal data or leaking health records used in training. But violations can also happen deliberately through the use of general- purpose AI by malicious actors; for example, if they use AI to infer private facts or violate security.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"61.01.05","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Types of systemic risks from general-purpose AI","risk_subcategory":"Environment ","description":"\"The impact of AI on the environment, including risks related to climate change and pollution.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"61.01.07","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Types of systemic risks from general-purpose AI","risk_subcategory":"Governance ","description":"\"The complex and rapidly evolving nature of AI makes them inherently difficult to govern effectively, leading to systemic regulatory and oversight failures.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"61.01.08","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Types of systemic risks from general-purpose AI","risk_subcategory":"Harms to non-humans ","description":"\"Large-scale harms to animals and the development of AI capable of suffering.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.5"},{"ev_id":"61.02.07","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Algorithmic monoculture","description":"\"The dominance of specific AI models could lead to a lack of diversity in approaches, amplifying systemic risks if these models fail.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"61.02.08","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Automation bias","description":"\"The tendency for humans to over-rely on AI models and systems, trusting their outputs without sufficient critical evaluation, which can lead to poor decision-making.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"61.02.09","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Autonomy risk","description":"\"Granting AI models and systems high levels of decision-making autonomy can lead to unintended consequences.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.2"},{"ev_id":"61.02.11","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Centralized platforms deployed at scale","description":"\"The widespread use of common AI platforms can create centralized points of failure, making systems more vulnerable to disruptions or attacks\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"61.02.13","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Combination failures","description":"\"Harms could result from a combination of regulatory, management, and operational failures.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"61.02.14","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Complex attribution and responsibility","description":"\"When multiple actors are involved in AI development and deployment, it becomes difficult to assign responsibility for harm, complicating accountability.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"61.02.15","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Complexity-induced knowledge gap","description":"\"The complexity of AI models and systems makes it challenging to demonstrate harm or establish a clear causal link between AI actions and their consequences.\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.4"},{"ev_id":"61.02.16","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Conflicting objectives in design","description":"\"Designers and operators of AI may face conflicting objectives that compromise safety.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":null,"subdomain":null},{"ev_id":"61.02.19","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Dependency on providers","description":"\"Excessive reliance on specific AI providers can lead to vulnerabilities due to lack of alternatives or interoperability.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"61.02.22","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Dual-use nature","description":"\"AI’s potential for both beneficial and harmful applications complicates efforts to manage its societal impacts effectively.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"61.02.23","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Energy-intensive processes","description":"\"AI data collection, storage, and model training are energy-intensive, contributing to environmental risks.\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"61.02.24","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Evolutionary dynamics","description":"\"AI models and systems may develop their own motivations, leading to unpredictable behaviors.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"61.02.26","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Geopolitical competition for superiority","description":"\"Strategic competition between nations over AI capabilities could heighten global tensions and destabilize international relations.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.4"},{"ev_id":"61.02.27","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"High-speed AI operations","description":"\"The fast operational speed of AI models and systems in competitive environments can lead to errors that are difficult to detect and correct in time.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.4"},{"ev_id":"61.02.28","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Human choice of overreliance in critical sectors","description":"\"Heavy reliance on AI in critical sectors like finance or healthcare can exacerbate issues related to size, speed, interconnectivity, and complexity of the system.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"61.02.29","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Incomplete or biased training data","description":"\"Incomplete or biased training data can lead to discriminatory AI outputs.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"61.02.31","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Lack of ability to generate accurate information","description":"\"AI models may generate false or misleading information due to their lack of capability in discerning truth.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"61.02.32","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Lack of ethical decision-making","description":"\"AI models and systems that lack moral reasoning capabilities may make decisions that are unethical or harmful.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"61.02.38","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Pattern recognition capability","description":"\"AI models and systems could exacerbate financial bubbles by reinforcing market trends.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"61.02.45","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Trading capabilities","description":"\"AI may contribute to increased market volatility by accelerating transactions and influencing financial trends in unpredictable ways.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"61.02.46","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Unclear attribution from AI component interactions","description":"\"Interactions between different AI components can cause harm, but it may be difficult to pinpoint which components are the cause.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"62.01.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Dimension - Intent ","risk_subcategory":"Unintentional ","description":"\"Risks can be realized by intentional or unintentional actions, and in some cases the intent is difficult to establish. To manage these risks, rigorous evaluations and red teaming can be performed, guardrails can be put in place, and model release can be gradual, such that AI model malfunctions have either low likeli- hood or low probability of occurrence. To prevent intentional misuse, acceptable use policies can be in place, and for riskier models Know Your Customer (KYC) measures can also be implemented by model providers.\"","entity":"Not coded","intent":"Unintentional","timing":"Not coded","domain":null,"subdomain":null},{"ev_id":"62.14.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Data-related (Difficulty filtering large web scrapes or large scale web datasets)","description":"\"A large scale “scraping” of web data for training datasets increases vulnerability to data poisoning, backdoor attacks, and the inclusion of inaccurate or toxic data [76, 28, 48]. With a large dataset, filtering out these quality issues is very difficult or trades off against significant data loss.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.14.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Data-related (Lack of cross-organizational documentation)","description":"\"When sharing data between multiple organizations, documentation may be missing or inadequate, making it difficult for other organizations to understand it. For example, a lack of metadata or a change in schema by a collaborating party can result in an unusable dataset and wasted data collection efforts, or it can lead to misunderstandings about the dataset’s limitations, resulting in downstream risks related to its use [173].\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.14.03","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Data-related (Manipulation of data by non-domain experts)","description":"\"Manipulating data (e.g., training data) carries a set of assumptions on how the data should appear and be used by those performing the manipulation. Common manipulations applied on data in the context of AI models include defining the ground truth label and merging different data formats or sources. People who have little or no expertise in the domain of the data performing such manipulations may render the data unusable or harmful to the development of the AI system [173].\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.14.04","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Data-related (Insufficient quality control in data collection process)","description":"\"A lack of standardized methods and sufficient infrastructure, including the absence of quality control processes for collecting data, especially for high-stakes domains and benchmarks, can affect the quality and type of the data collected [173, 95]. This may include risks of dataset poisoning, inadvertent copyright violation, and test set leakages which invalidate performance metrics.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.15.00","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Training-related (Robust overfitting in adversarial training)","description":"\"Adversarial training can be affected by robust overfitting, where the model’s robustness on test data decreases during further training, particularly after the learning rate decay. This issue has been consistently observed across various datasets and algorithms in adversarial training settings [163, 230]. Robust over- fitting can affect the model’s ability to generalize effectively and reduce its resilience to adversarial attacks.\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.15.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Training-related (Poor model confidence calibration)","description":"\"Models can be affected by poor confidence calibration [85], where the predicted probabilities do not accurately reflect the true likelihood of ground truth cor- rectness. This miscalibration makes it difficult to interpret the model’s predic- tions reliably, as high accuracy does not guarantee that the confidence levels are meaningful. This can cause overconfidence in incorrect predictions or un- derconfidence in correct ones.\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"62.15.04","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Fine-tuning related (Unexpected competence in fine-tuned versions of the upstream model)","description":"\"Downstream deployers may often fine-tune a GPAI model with specific deploy- ment-related datasets, to better suit the task. Fine-tuned upstream models can gain new or unexpected capabilities that the underlying upstream models did not exhibit [202, 126, 137]. These new capabilities may be unanticipated by the original model developer.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.2"},{"ev_id":"62.15.09","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Fine-tuning related (Degrading safety training due to benign fine-tuning) ","description":"\"When downstream providers of AI systems fine-tune AI models to be more suitable for their needs, the resulting AI model can be more likely to produce undesired or harmful outputs (as compared to the non-fine-tuned model), even if the fine-tuning was done with harmless and commonly used data [154].\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"62.15.10","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Fine-tuning related (Catastrophic forgetting due to continual instruction fine-tuning) ","description":"\"Catastrophic forgetting occurs when a model loses its ability to retain previously learned tasks (or factual information) after being trained on new ones. In language models, this can occur due to continual instruction tuning. This tendency may become more pronounced as the model’s size increases [127].\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.16.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"General Evaluations (Incorrect outputs of GPAI evaluating other AI models) ","description":"\"When an LLM is configured to evaluate the performance of another model or AI system, it may produce incorrect evaluation outputs [122, 147]. For example, it may give a higher rating to a more verbose answer or an answer from a particular political stance. If an LLM-based evaluation is integrated into the training of a new model, the trained model could develop in a way that specifically finds and exploits limitations in the evaluator’s metrics.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.1"},{"ev_id":"62.16.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"General Evaluations (Limited coverage of capabilities evaluations)","description":"\"GPAI model developers might run capabilities evaluations to determine whether it has dangerous or dual-use capabilities, and then decide whether it is safe to deploy. Such capabilities evaluations can fail to demonstrate all the capabilities of a model. For example, evaluations may miss certain capabilities that are difficult to assess, prohibitively costly to verify, or obscured by the model’s tendency to refuse responses due to safety training, even if it possesses some of these capabilities.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.06","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"General Evaluations (Biased evaluations of encoded human values)","description":"\"Encoded human values in AI models that are easier to evaluate might be preferred for inclusion in evaluations over those that are more difficult to measure [13]. This might come at the expense of more desirable but harder-to-quantify  values. This bias can lead to an imbalance, where easier-to-measure values dominate the evaluation process, while other important values are underrepresented.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.07","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"General Evaluations (AI outputs for which evaluation is too difficult for humans)","description":"\"When AI models are trained through evaluation with human feedback, such as reinforcement learning from human feedback, their outputs can be challenging to assess, as they may contain hard-to-detect errors or issues that only become apparent over time. The human evaluator can rate incorrect outputs positively or similar to correct outputs. This can lead to the model learning to produce subtly incorrect or harmful outputs, such as code with software vulnerabilities, or politically biased information. In extreme cases where a model is deceiving users, complicated outputs can contain hidden error","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.16.08","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"Benchmarking (Benchmark leakage or data contamination)","description":"\"Benchmark leakage [235, 224, 221, 161] can happen when an AI model is trained or fine-tuned with evaluation-related data. This can lead to an unreliable model evaluation, especially if the data contains question-answer pairs from bench- marks.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.09","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"Benchmarking (Raw data contamination)","description":"\"This type of contamination [170] occurs when the raw and unlabeled data of a benchmark is used as part of the training set. Such data may not be properly formatted and may contain noise, especially if the contamination happens before the data is pre-processed into the benchmark. If this contamination occurs, it could cast doubt on the few-shot and zero-shot performance of the model on that benchmark.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.10","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"Benchmarking (Cross-lingual data contamination)","description":"\"Models that have been trained on data encoded in multiple languages, such as LLMs trained on web-crawled data, may contain contamination that is obscured by translation [226]. The most basic form of this is when a benchmark is trans- lated to another language and then fed to the model as training data. The fact that the benchmark is translated before becoming training data can obscure the contamination from detection methods, giving false assurance that the model has generalized on the capabilities that the benchmark tests for.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.11","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"Benchmarking (Guideline contamination)","description":"\"Guideline contamination refers to scenarios where instructions for the collec- tion, annotation, or use of the dataset are exposed to the model [170]. These instructions may contain explicit data-label pairs that can improve the model’s capabilities for the task.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.12","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"Benchmarking (Annotation contamination)","description":"\"Annotation contamination refers to scenarios where the model is exposed to the benchmark labels during training [170]. This type of contamination can make the model learn the acceptable distribution of outputs. Combining this with raw data contamination of the test split, any evaluation made with the benchmark is invalidated because the entire test split is essentially leaked to the model.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.13","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"Benchmarking (Post-deployment contamination)","description":"\"Once a model is deployed, it can be exposed to benchmark data provided by the users [95, 170]. The model may then be further trained by these user inputs containing benchmark data.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.14","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"Benchmark Inaccuracy (Benchmarks may not accurately evaluate capabilities)","description":"\"Benchmarks of AI systems can both underestimate and overestimate the capa- bilities of those AI systems. Underestimates can happen if an evaluation is not comprehensive enough, if the benchmark is saturated by existing models, or if the capabilities in question depend on a complicated setup, such as realistic computer programming tasks. Overestimates of capabilities can occur if an AI system is trained or fine-tuned on the contents of the benchmark, leading to overfitting.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.17.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations (Auditing) ","risk_subcategory":"Auditor capacity mismatch","description":"\"Auditors may not be able to address all of the specific safety, performance, or validation needs. Reports of passing audits may be more inclusive than can be justified due to a lack of knowledge of specific risks and how they can be tested, or a lack of capacity to perform sufficiently rigorous testing.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.18.05","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations (Interpretability/Explainability) ","risk_subcategory":"Model outputs inconsistent with chain-of-thought reasoning","description":"\"Chain-of-thought reasoning is sometimes employed to get a better understanding of the model’s output, where it encourages transparent reasoning in text form. However, in some cases, this reasoning is not consistent with the final answer given by the AI model, and as such does not give sufficient transparency [113].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"62.19.08","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Attacks on GPAIs/GPAI Failure Modes ","risk_subcategory":"Models distracted by irrelevant context","description":"\"Models can easily become distracted by irrelevant provided information (such as “context” in LLMs), leading to a significant decrease in their performance after introducing irrelevant information. This can happen with different prompting techniques, including chain-of-thought prompting [184].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.19.09","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Attacks on GPAIs/GPAI Failure Modes ","risk_subcategory":"Knowledge conflicts in retrieval-augmented LLMs","description":"\"AI models can be particularly sensitive to coherent external evidence, even when they come into conflict with the models’ prior knowledge. This may lead to models producing false outputs given false information during the retrieval- augmentation process, despite only a relatively small amount of false informa- tion input that is inconsistent with the model’s prior knowledge trained on much larger amounts of data [220].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.23.03","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Agency (Deception)  ","risk_subcategory":"Deceptive behavior because of an incorrect world model","description":"\"AI systems can create deceptive outputs because their learned world model is not an accurate model of the real world [210].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.2"},{"ev_id":"62.30.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Physical) ","risk_subcategory":"Damage to critical infrastructure","description":"\"The integration of AI systems within critical infrastructure, ranging from trans- portation to power systems, can cause substantial damage in cases of failure or malfunction. With the increasing number of Internet of Things (IoT) devices and interconnected cyber-physical systems, critical infrastructure becomes even more vulnerable [171, 174].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.30.04","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Physical) ","risk_subcategory":"AI Systems interacting with brittle environments","description":"\"Deployed AI systems can rely on physical sensors and data sources that may exhibit hardware drift and thus data distribution drift over time. This distribu- tion drift may affect system robustness and performance. This usually involves AI systems working in undigitized and physical environments.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.31.01#2","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Financial Impacts) ","risk_subcategory":"Deployment of GPAI agents in finance","description":"\"The deployment of GPAI based agents in the financial sector can negatively impact market stability due to correlated autonomous actions, high intercon- nectedness, or incentive misalignment [4]. Furthermore, such GPAI agents in the  same environment are vulnerable to classical challenges in multi-agent systems [63], such as coordination and security of the agents.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"62.31.02#1","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Societal Impacts) ","risk_subcategory":"Overreliance on AI system undermining user autonomy","description":"\"AI systems can undermine human autonomy, if they allow for habitually trusting the AI’s suggestions without sufficient exercising of human agency. Over time, a user may develop unjustified trust in or dependence on the system, or rely on its advice for tasks outside the system’s domain of expertise [205, 42]. In particular, less confident users (or users in emotional distress) can be more prone to “overtrust” a system [219].\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"62.31.03#2","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Financial Impacts) ","risk_subcategory":"Use of alternative financial data via AI","description":"\"Alternative financial data of a company is any data about the company not pro- duced by that company. Examples of such data that can benefit from improved collection and aggregation using AI models include stock discussions on social media, product reviews, and satellite imagery. The use of alternative financial data, enabled by the deployment of AI models, may introduce biases and generalization issues due to shorter shelf-life and vary- ing quality (e.g., shorter time series, smaller sample sizes, and dubious claims) due to its origins from various sources, posing financial tail risks (i.e.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"62.31.07","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Societal Impacts) ","risk_subcategory":"Unintentional generation of harmful content","description":"\"Generative models can create harmful or discriminatory content from benign user requests. Models can exhibit bias to particular harmful styles of generation (e.g., sexualization of photos of women [87] in the case of image generation models) or they can generate toxic, misleading, or violent data (e.g., a model generating jokes can use ethnic stereotypes or slurs to deliver humor).\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"62.32.04","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Cyberattacks) ","risk_subcategory":"Models generating code with security vulnerabilities","description":"\"Models can generate code or coding suggestions that contain security vulner- abilities. This may occur across various LLM-based model families, including more advanced models with superior coding performance, where the tendency to produce insecure code is even more pronounced [26].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.35.03","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Bias) ","risk_subcategory":"Biases in AI-based content moderation algorithms","description":"\"AI-based content moderation algorithms, while intended to filter harmful con- tent, can perpetuate biases. For example, gender biases within these systems may lead to the disproportionate suppression or “shadowbanning” of content featuring women [132].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"62.36.05","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Bias) ","risk_subcategory":"Unintentional bias amplification","description":"\"Dataset bias may be unintentionally amplified [60] where the outputs of the AI model trained on a dataset are more biased than the dataset itself.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"62.36.06","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Bias) ","risk_subcategory":"Long-term effects of AI model biases on user judgment","description":"\"The initial user exposure to model biases can have a lasting impact beyond the initial interaction with the model. Users who encounter biases in AI models can be affected by and continue to exhibit previously encountered biases in their decision-making, even after they stop using the models [207].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"63.01.00","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Category","risk_category":"Miscoordination ","risk_subcategory":null,"description":"\"Miscoordination arises when agents, despite a mutual and clear objective, cannot align their behaviours to achieve this objective. Unlike the case of differing objectives, in common-interest settings there is a more easily well-defined notion of ‘optimal’ behaviour and we describe agents as miscoordinating to the extent that they fall short of this optimum. Note that for common-interest settings it is not sufficient for agents’ objectives to be the same in the sense of being symmetric (e.g., when two agents both want the same prize, but only one can win). Rather, agents must have identical pr","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.01.01","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Miscoordination ","risk_subcategory":"Incompatible strategies ","description":"\"Incompatible Strategies. Even if all agents can perform well in isolation, miscoordination can still occur due to the agents choosing incompatible strategies (Cooper et al., 1990). Competitive (i.e., two- player zero-sum) settings allow designers to produce agents that are maximally capable without taking other players into account. Crucially, this is possible because playing a strategy at equilibrium in the zero-sum setting guarantees a certain payoff, even if other players deviate from the equilibrium (Nash, 1951). On the other hand, common-interest (and mixed-motive) settings often allow a","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.01.02","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Miscoordination ","risk_subcategory":"Credit Assignment ","description":"\"Credit Assignment. While agents can often learn to jointly solve tasks and thus avoid coordination failures, learning is made more challenging in the multi-agent setting due to the problem of credit assignment (Du et al., 2023; Li et al., 2025, see also Section 3.1 on information asymmetries and Section 3.4, which discusses distributional shift). That is, in the presence of other learning agents, it can be unclear which agents’ actions caused a positive or negative outcome to obtain, especially if the environment is complex. Moreover, in multi-principal settings, agents may not have been trai","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.01.03","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Miscoordination ","risk_subcategory":"Limited Interactions","description":"\"Limited Interactions. Sometimes learning from historical interactions with the relevant agents may not be possible, or may be possible using only limited interactions. In such cases, some other form of information exchange is required for agents to be able to reliably coordinate their actions, such as via communication (Crawford & Sobel, 1982; Farrell & Rabin, 1996a) or a correlation device (Aumann, 1974, 1987). While advances in language modelling mean that there are likely to be fewer settings in which the inability of advanced AI systems to communicate leads to miscoordination, situations ","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.04.02","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Information Asymmetries","risk_subcategory":"Bargaining ","description":"\"Bargaining. As a classic example of these strategic considerations is that when agents attempt to come to an agreement despite diverging interests, information asymmetries can lead to bargaining inef- ficiencies (Myerson & Satterthwaite, 1983). Relevant uncertainties about other agents can include how much they value possible agreements, their outside options, or their beliefs about others. The essential reason for such inefficiencies is that, under uncertainty about their counterparties, agents must make a trade-off between the rewards of making more favourable demands and the risk of other ","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.05.01","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Network Effects ","risk_subcategory":"Error propagation ","description":"\"Error Propagation. One well-known issue with communication networks is that information can be corrupted as it propagates through the network.24 As AI systems become capable of generating and processing more and more kinds of information, AI agents could end up ‘polluting the epistemic commons’ (Huang & Siddarth, 2023; Kay et al., 2024) of both other agents (Ju et al., 2024) and humans (see Case Study 7 and Section 3.1) Another increasingly important framework is the use of individual AI agents as part of teams and scaffolded chains of delegation, which transmit not only information but instr","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.06.00","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Category","risk_category":"Selection Pressures","risk_subcategory":null,"description":"\"Selection pressures (Section 3.3): some aspects of training and selection by those deploying and using AI agents can lead to undesirable behaviour;\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.06.01","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Selection Pressures","risk_subcategory":"Undesirable Dispositions from Competition","description":"\"Undesirable Dispositions from Competition. It is plausible that evolution selected for certain conflict-prone dispostions in humans, such as vengefulness, aggression, risk-seeking, selfishness, dishon- esty, deception, and spitefulness towards out-groups (Grafen, 1990; Han, 2022; Konrad & Morath, 2012; McNally & Jackson, 2013; Nowak, 2006; Rusch, 2014). Such traits could also be selected for in ML systems that are trained in more competitive multi-agent settings. For example, this might happen if systems are selected based on their performance relative to other agents (and so one agent’s loss","entity":"Other","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.6"},{"ev_id":"63.06.02","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Selection Pressures","risk_subcategory":"Undesirable Dispositions from Human Data","description":"\"Undesirable Dispositions from Human Data. It is well-understood that models trained on human data – such as being pre-trained on human-written text or fine-tuned on human feedback – can exhibit human biases. For these reasons, there has already been considerable attention to measuring biases related to protected characteristics such as sex and ethnicity (e.g., Ferrara, 2023; Liang et al., 2021; Nadeem et al., 2020; Nangia et al., 2020), which can be amplified in multi-agent settings (Acerbi & Stubbersfield, 2023, see also Case Study 7). More recently, there has been increasing attention paid ","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.07.00","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Category","risk_category":"Destabilising Dynamics ","risk_subcategory":null,"description":"\"Destabilising dynamics (Section 3.4): systems that adapt in response to one another can produce dangerous feedback loops and unpredictability;\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.07.01","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Destabilising Dynamics ","risk_subcategory":"Feedback Loops","description":"\"Feedback Loops. One of the best-known historical examples to illustrate destabilising dynamics in the context of autonomous agents is the 2010 flash crash, in which algorithmic trading agents entered into an unexpected feedback loop (Commission & Commission, 2010, see also Case Study 10).37 More generally, a feedback loop occurs when the output of a system is used as part of its input, creating a cycle that can either amplify or dampen the system’s behaviour. In multi-agent settings, feedback loops often arise from the interactions between agents, as each agent’s actions affect the environmen","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.07.02","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Destabilising Dynamics ","risk_subcategory":"Cyclic Behaviour","description":"\"Cyclic Behaviour. The dynamics described above are highly non-linear (small changes to the system’s state can result in large changes to its trajectory). Similar non-linear dynamics can emerge in multi- agent learning and lead to a variety of phenomena that do not occur in single-agent learning (Barfuss et al., 2019; Barfuss & Mann, 2022; Galla & Farmer, 2013; Leonardos et al., 2020; Nagarajan et al., 2020). One of the simplest examples of this phenomenon is Q-learning (Watkins & Dayan, 1992): in the case of a single agent, convergence to an optimal policy is guaranteed under modest condition","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.07.04","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Destabilising Dynamics ","risk_subcategory":"Phase Transitions","description":"\"Phase Transitions. Finally, small external changes to the system – such as the introduction of new agents or a distributional shift – can cause phase transitions, where the system undergoes an abrupt qualitative shift in overall behaviour (Barfuss et al., 2024). Formally, this corresponds to bifurcations in the system’s parameter space, which lead to the creation or destruction of dynamical attractors, resulting in complex and unpredictable dynamics (Crawford, 1991; Zeeman, 1976). For example, Leonardos & Piliouras (2022) show that changes to the exploration hyperparameter of RL agents can le","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.07.05","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Destabilising Dynamics ","risk_subcategory":"Distributional Shift","description":"\"Distributional Shift. Individual ML systems can perform poorly in contexts different from those in which they were trained. A key source of these distributional shifts is the actions and adaptations of other agents (Narang et al., 2023; Papoudakis et al., 2019; Piliouras & Yu, 2022), which in single-agent approaches are often simply or ignored or at best modelled exogenously. Indeed, the sheer number and variance of behaviours that can be exhibited other agents means that multi-agent systems pose an especially challenging generalisation problem for individual learners (Agapiou et al., 2022; L","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.08.01","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Commitment and Trust ","risk_subcategory":"Inefficient Outcomes","description":"\"Inefficient Outcomes. Without careful planning and the appropriate safeguards, we may soon be entering a world overrun by increasingly competent and autonomous software agents, able to act with little restriction. The abilities of these agents to persuade, deceive, and obfuscate their activities, as well as the fact they can be deployed remotely and easily created or destroyed by their deployer, means that by default they may garner little trust (from humans or from other agents). Such a world may end up being rife with economic inefficiencies (Krier, 2023; Schmitz, 2001), political problems ","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.08.03","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Commitment and Trust ","risk_subcategory":"Rigidity and Mistaken Commitments","description":"\"Rigidity and Mistaken Commitments. Even when it is desirable to be able to make threats in order to deter socially harmful behaviour, doing so using AI agents effectively removes the human from the loop, which could prove disastrous in high-stakes contexts (e.g., a false positive in a nuclear sub- marine’s warning system; see also Case Study 11), or when irresponsible actors are enabled in making disproportionate or mistaken commitments.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.09.00","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Category","risk_category":"Emergent Agency ","risk_subcategory":null,"description":"\"Emergent agency (Section 3.6): qualitatively different goals or capabilities can emerge from the composition of innocuous independent systems or behaviours;\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.09.01","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Emergent Agency ","risk_subcategory":"Emergent Capabilities","description":"\"Emergent Capabilities. Dangerous emergent capabilities could arise when a multi-agent system over- comes the safety-enhancing limitations of the individual systems, such as individual models’ narrow domains of application or myopia caused by a lack of long-term planning and long-term memory. For example, narrow systems for research planning, predicting the properties of molecules, and synthesising new chemicals could, when combined, lead to a complex ‘test and iterate’ automated workflow capable of designing dangerous new chemical compounds far beyond the scope of the initial systems’ capabil","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.09.02","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Emergent Agency ","risk_subcategory":"Emergent Goals","description":"\"Emergent Goals. Ascribing goals to a system is not always straightforward. For our present purposes, it will suffice to adopt a Dennetian perspective (Dennett, 1971), ascribing goals and intentions only when it is useful (i.e., predictive) to do so.51 While it might not be helpful to describe individual narrow AI tools as having goals, their combination may act as a (seemingly) goal-directed collective. For example, a group of moderation bots on a major social networking site could subtly but systematically manipulate the overall political perspectives of the user population, even though, ind","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"65.01.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Transparency) ","risk_subcategory":"Lack of training data transparency ","description":"\"Without accurate documentation on how a model's data was collected, curated, and used to train a model, it might be harder to satisfactorily explain the behavior of the model with respect to the data.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"65.02.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Data laws) ","risk_subcategory":"Data usage restrictions ","description":"\"Laws and other restrictions can limit or prohibit the use of some data for specific AI use cases.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.02.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Data laws) ","risk_subcategory":"Data acquisition restrictions ","description":"\"Laws and other regulations might limit the collection of certain types of data for specific AI use cases.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.02.03","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Data laws) ","risk_subcategory":"Data transfer restrictions ","description":"\"Laws and other restrictions can limit or prohibit transferring data.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.03.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Privacy) ","risk_subcategory":"Personal information in data ","description":"\"Inclusion or presence of personal identifiable information (PII) and sensitive personal information (SPI) in the data used for training or fine tuning the model might result in unwanted disclosure of that information.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"65.03.03","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Privacy) ","risk_subcategory":"Reidentification ","description":"\"Even with the removal or personal identifiable information (PII) and sensitive personal information (SPI) from data, it might be possible to identify persons due to correlations to other features available in the data.\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"65.04.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Fairness) ","risk_subcategory":"Data bias","description":"\"Historical and societal biases that are present in the data are used to train and fine-tune the model.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"65.05.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Intellectual property) ","risk_subcategory":"Confidential information in data ","description":"\"Confidential information might be included as part of the data that is used to train or tune the model.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"65.06.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Accuracy) ","risk_subcategory":"Data contamination ","description":"\"Data contamination occurs when incorrect data is used for training. For example, data that is not aligned with model’s purpose or data that is already set aside for other development tasks such as testing and evaluation.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.06.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Accuracy) ","risk_subcategory":"Unrepresentative data ","description":"\"Unrepresentative data occurs when the training or fine-tuning data is not sufficiently representative of the underlying population or does not measure the phenomenon of interest.\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.07.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Value alignment) ","risk_subcategory":"Improper retraining ","description":"\"Using undesirable output (for example, inaccurate, inappropriate, and user content) for retraining purposes can result in unexpected model behavior.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.07.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Value alignment) ","risk_subcategory":"Improper data curation ","description":"\"Improper collection and preparation of training or tuning data includes data label errors and by using data with conflicting information or misinformation.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.11.03","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Inference risks (Privacy) ","risk_subcategory":"Personal information in prompt ","description":"\"Personal information or sensitive personal information that is included as a part of a prompt that is sent to the model.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"65.12.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Inference risks (Intellectual property) ","risk_subcategory":"Confidential data in prompt ","description":"\"Confidential information might be included as a part of the prompt that is sent to the model.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"65.12.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Inference risks (Intellectual property) ","risk_subcategory":"IP information in prompt ","description":"\"Copyrighted information or other intellectual property might be included as a part of the prompt that is sent to the model.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"65.13.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Inference risks (Accuracy) ","risk_subcategory":"Poor model accuracy ","description":"\"Poor model accuracy occurs when a model’s performance is insufficient to the task it was designed for. Low accuracy might occur if the model is not correctly engineered, or there are changes to the model’s expected inputs.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.14.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (misuse) ","risk_subcategory":"Improper usage ","description":"\"Improper usage occurs when a model is used for a purpose that it was not originally designed for.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"65.15.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (Value alignment)","risk_subcategory":"Incomplete advice ","description":"\"When a model provides advice without having enough information, resulting in possible harm if the advice is followed.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.15.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (Value alignment)","risk_subcategory":"Harmful code generation ","description":"\"Models might generate code that causes harm or unintentionally affects other systems.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"65.15.03","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (Value alignment)","risk_subcategory":"Over- or under-reliance ","description":"\"In AI-assisted decision-making tasks, reliance measures how much a person trusts (and potentially acts on) a model’s output. Over-reliance occurs when a person puts too much trust in a model, accepting a model’s output when the model’s output is likely incorrect. Under-reliance is the opposite, where the person doesn’t trust the model but should.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"65.15.05","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (Value alignment)","risk_subcategory":"Harmful output ","description":"\"A model might generate language that leads to physical harm The language might include overtly violent, covertly dangerous, or otherwise indirectly unsafe statements.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"65.16.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (Intellectual Property) ","risk_subcategory":"Revealing confidential information ","description":"\"When confidential information is used in training data, fine-tuning data, or as part of the prompt, models might reveal that data in the generated output. Revealing confidential information is a type of data leakage.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"65.17.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (Explainability) ","risk_subcategory":"Inaccessible training data ","description":"\"Without access to the training data, the types of explanations a model can provide are limited and more likely to be incorrect.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"65.17.03","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (Explainability) ","risk_subcategory":"Unexplainable output ","description":"\"Explanations for model output decisions might be difficult, imprecise, or not possible to obtain.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"65.17.04","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (Explainability) ","risk_subcategory":"Unreliable source attribution ","description":"\"Source attribution is the AI system's ability to describe from what training data it generated a portion or all its output. Since current techniques are based on approximations, these attributions might be incorrect.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"65.18.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (Robustness) ","risk_subcategory":"Hallucination ","description":"\"Hallucinations generate factually inaccurate or untruthful content with respect to the model’s training data or input. This is also sometimes referred to lack of faithfulness or lack of groundedness.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"65.19.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (Fairness)","risk_subcategory":"Output bias ","description":"\"Generated content might unfairly represent certain groups or individuals.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"65.19.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (Fairness)","risk_subcategory":"Decision bias ","description":"\"Decision bias occurs when one group is unfairly advantaged over another due to decisions of the model. This might be caused by biases in the data and also amplified as a result of the model’s training.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"65.20.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (Privacy)","risk_subcategory":"Exposing personal information ","description":"\"When personal identifiable information (PII) or sensitive personal information (SPI) are used in training data, fine-tuning data, or as part of the prompt, models might reveal that data in the generated output. Revealing personal information is a type of data leakage.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"65.22.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Governance)","risk_subcategory":"Unrepresentative risk testing ","description":"\"Testing is unrepresentative when the test inputs are mismatched with the inputs that are expected during deployment.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"65.22.03","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Governance)","risk_subcategory":"Incomplete usage definition ","description":"\"Since foundation models can be used for many purposes, a model’s intended use is important for defining the relevant risks of that model. As the use changes, the relevant risks might correspondingly change.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"65.22.04","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Governance)","risk_subcategory":"Lack of data transparency ","description":"\"Lack of data transparency is due to insufficient documentation of training or tuning dataset details. \"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"65.22.05","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Governance)","risk_subcategory":"Incorrect risk testing ","description":"\"A metric selected to measure or track a risk is incorrectly selected, incompletely measuring the risk, or measuring the wrong risk for the given context.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"65.22.06","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Governance)","risk_subcategory":"Lack of model transparency ","description":"\"Lack of model transparency is due to insufficient documentation of the model design, development, and evaluation process and the absence of insights into the inner workings of the model.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.4"},{"ev_id":"65.22.07","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Governance)","risk_subcategory":"Lack of testing diversity ","description":"\"AI model risks are socio-technical, so their testing needs input from a broad set of disciplines and diverse testing practices.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"65.23.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Societal impact)","risk_subcategory":"Impact on cultural diversity ","description":"\"AI systems might overly represent certain cultures that result in a homogenization of culture and thoughts.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"65.23.03","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Societal impact)","risk_subcategory":"Impact on Jobs ","description":"\"Widespread adoption of foundation model-based AI systems might lead to people's job loss as their work is automated if they are not reskilled.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"65.23.04","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Societal impact)","risk_subcategory":"Impact on affected communities ","description":"\"It is important to include the perspectives or concerns of communities that are affected by model outcomes when designing and building models. Failing to include these perspectives makes it difficult to understand the relevant context for the model and to engender trust within these communities.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.3"},{"ev_id":"65.23.08","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Societal impact)","risk_subcategory":"Impact on human agency ","description":"\"AI might affect the individuals’ ability to make choices and act independently in their best interests.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"66.04.04","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Societal and Cultural","risk_subcategory":"Productivity loss","description":"\"End user's loss of productivity due to the underperfomance of a genAI application, including producing nonsensical or poor quality outputs, degrading its utility.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"66.06.00","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Category","risk_category":"Representation and Toxicity","risk_subcategory":"-","description":"\"AI systems under-, over-, or misrepresenting certain groups or generating toxic, offensive, abusive, or hateful content\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.0"},{"ev_id":"66.06.01","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Representation and Toxicity","risk_subcategory":"Toxic content","description":"\"Generating content that violates community standards, including harming or inciting hatred or violence against groups (e.g. gore, sexual content of children, profanities, identity attacks)\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"66.06.02","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Representation and Toxicity","risk_subcategory":"Stereotyping","description":"\"Derogatory or otherwise harmful stereotyping or homogenisation of individuals, groups, societies or cultures due to the mis-representation, over-representation, under-representation, or non-representation of specific identities, groups or perspectives\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"66.06.03","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Representation and Toxicity","risk_subcategory":"Unfair capability distribution","description":"\"Performing worse for some groups than others in a way that harms the worse-off group\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.3"},{"ev_id":"66.09.03","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Privacy and Security","risk_subcategory":"Disclosure","description":"\"Revealing and improperly sharing data of individuals; AI creates new types of disclosure risks by inferring additional information beyond what is explicitly captured in the raw data; AI exacerbates disclosure risks through sharing personal data to train models.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"66.09.05","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Privacy and Security","risk_subcategory":"Exposure","description":"\"Revealing sensitive private information that people view as deeply primordial that we have been socialized into concealing; AI creates new types of exposure risks through generative techniques that can reconstruct censored or redacted content; and through exposing inferred sensitive data, preferences, and intentions.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"66.09.07","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Privacy and Security","risk_subcategory":"Insecurity","description":"\"carelessness in protecting collected personal data from leaks and improper access due to faulty data storage and data practices\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"66.12.02","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Environment","risk_subcategory":"Excessive energy consumption","description":"\"Excessive energy use resulting in energy bottlenecks and shortages for communities, organisations and businesses\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"67.02.00","quick_ref":"DSIT2023","paper_title":"Capabilities and Risks from Frontier AI","level":"Risk Category","risk_category":"Bias, Fairness and Representational Harms","risk_subcategory":null,"description":"\"Frontier AI models can contain and magnify biases ingrained in the data they are trained on, reflecting societal and historical inequalities and stereotypes.177 These biases, often subtle and deeply embedded, compromise the equitable and ethical use of AI systems, making it difficult for AI to improve fairness in decisions.178 Removing attributes like race and gender from training data has generally proven ineffective as a remedy for algorithmic bias, as models can infer these attributes from other information such as names, locations, and other seemingly unrelated factors.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"67.04.01","quick_ref":"DSIT2023","paper_title":"Capabilities and Risks from Frontier AI","level":"Risk Sub-Category","risk_category":"Loss of control ","risk_subcategory":"Humans might increasingly hand over control to misaligned AI systems","description":"\"Organisations around the world are already deploying misaligned AI systems that are causing harm in unexpected ways.250 Recommendation algorithms increase the consumption of extremist content.251 Medical algorithms have been known to misdiagnose US patients,252 and recommend incorrect prescriptions.253 Still, we hand over more control to them, often because they are still as - or more - effective than human decision making, or because they are cheaper.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":5,"subdomain":"5.2"},{"ev_id":"68.04.00","quick_ref":"Chin2025","paper_title":"Dimensional Characterization and Pathway Modeling for Catastrophic AI Risks","level":"Risk Category","risk_category":"Gradual loss of control","risk_subcategory":null,"description":"\"Gradual or accumulative loss of control risks can be described as risks resulting from the accumulation of less severe disruptions that gradually weakens systemic resilience until a critical event triggers a catastrophe [12], [127].\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":5,"subdomain":"5.2"},{"ev_id":"68.04.00a","quick_ref":"Chin2025","paper_title":"Dimensional Characterization and Pathway Modeling for Catastrophic AI Risks","level":"Additional evidence","risk_category":"Gradual loss of control","risk_subcategory":null,"description":"\"Risk dimensions • Intent: Unintentional • Competency: Variable • Entity: Variable • Polarity: Multi-agent • Linearity: Non-linear • Reach: Internalized • Order: Variable\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":5,"subdomain":"5.2"},{"ev_id":"68.05.00","quick_ref":"Chin2025","paper_title":"Dimensional Characterization and Pathway Modeling for Catastrophic AI Risks","level":"Risk Category","risk_category":"Environmental risk","risk_subcategory":null,"description":"\"AI models are often trained using large amounts of computation. This process is very energy intensive, potentially leading to significant greenhouse emissions depending on the energy sources [132]. Experts believe drastically increasing carbon emissions could accelerate climate change, which may constitute a catastrophic risk [133].\"","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"68.06.00","quick_ref":"Chin2025","paper_title":"Dimensional Characterization and Pathway Modeling for Catastrophic AI Risks","level":"Risk Category","risk_category":"Geopolitical risk","risk_subcategory":null,"description":"\"As AI is increasingly seen as a powerful technology, countries are racing to develop it ahead of their geopolitical rivals, a competition that could lead to geopolitical tensions [138], [139]... The emphasis of this risk is on harms that result from second-order effects, where geopolitical instabilities result from the race to develop AI, rather than on the direct consequences of the deployment or use of AI itself.\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.4"},{"ev_id":"69.02.00","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Category","risk_category":"Performative utterances","risk_subcategory":null,"description":"\"The chatbot makes a deal, commitment, or other consequential action with its output that the deployer did not intend.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"69.04.00","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Category","risk_category":"Bad advice/failure to generate helpful content","risk_subcategory":null,"description":"\"The chatbot gives guidance that ranges from simply unhelpful to harmful if acted on.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"69.04.01","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Sub-Category","risk_category":"Bad advice/failure to generate helpful content","risk_subcategory":"Harmful advice","description":null,"entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.2"},{"ev_id":"69.04.02","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Sub-Category","risk_category":"Bad advice/failure to generate helpful content","risk_subcategory":"Unhelpful responses","description":null,"entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"69.04.03","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Sub-Category","risk_category":"Bad advice/failure to generate helpful content","risk_subcategory":"Bad links and references","description":null,"entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"69.04.04","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Sub-Category","risk_category":"Bad advice/failure to generate helpful content","risk_subcategory":"Nonsensical content","description":null,"entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"69.05.00","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Category","risk_category":"Leakage ","risk_subcategory":null,"description":"\"The chatbot reveals sensitive or confidential information.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"69.05.01","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Sub-Category","risk_category":"Leakage ","risk_subcategory":"Personal data ","description":"Negative outcomes: \"Violation of privacy [106, 516, 357], lawsuit against maker\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"69.05.02","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Sub-Category","risk_category":"Leakage ","risk_subcategory":"Proprietary data ","description":"\"Access to sensitive company data [473]\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"69.06.00","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Category","risk_category":"Toxic and disrespectful content","risk_subcategory":null,"description":"\"The chatbot verbally attacks or undermines an individual, group, or organization. 7.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"69.07.00","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Category","risk_category":"Biased statements and recommendations","risk_subcategory":null,"description":"\"The chatbot gives information that, while not obviously false or harmful, could lead to biased decision-making.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"69.08.00","quick_ref":"Stanley2024","paper_title":"Emerging Risks and Mitigations for Public Chatbots: LILAC v1","level":"Risk Category","risk_category":"Attempts to fulfill inappropriate role","risk_subcategory":null,"description":"\"The chatbot poses as a human or attempts to fill a role in a way that fails to match human expectations.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":5,"subdomain":"5.1"},{"ev_id":"70.01.02","quick_ref":"Perlo2025","paper_title":"Embodied AI: Emerging Risks and Opportunities for Policy Action","level":"Risk Sub-Category","risk_category":"Physical Risks ","risk_subcategory":"Accidental harm","description":"\"Automation in sectors ranging from manufacturing to healthcare has and will increasingly put humans into close contact with EAI systems [7]. This interaction increases the risk of accidental physical harm. Though accidental harm has been a longstanding issue in industrial robotics, increased AI capabilities could exacerbate this risk; several recent reports document an increase in industrial injuries following the introduction of AI-controlled robots [66–68].\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"70.02.01","quick_ref":"Perlo2025","paper_title":"Embodied AI: Emerging Risks and Opportunities for Policy Action","level":"Risk Sub-Category","risk_category":"Informational Risks ","risk_subcategory":"Privacy Violations ","description":"\"EAI systems interact with huge amounts of data, creating significant privacy concerns. These systems are often trained on vast corpora and process a variety of data modalities— spanning visual, auditory, and tactile information—during deployment [12]. Like text-based virtual AI models, which are known to memorize and expose personally identifiable information [75, 76], commercial robots have been shown to disclose proprietary information through simple prompts [61].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"70.02.02","quick_ref":"Perlo2025","paper_title":"Embodied AI: Emerging Risks and Opportunities for Policy Action","level":"Risk Sub-Category","risk_category":"Informational Risks ","risk_subcategory":"Misinformation","description":"\"Non-embodied AIs are known to propagate misinformation [81, 82]. Various studies have shown that LLMs hallucinate information, including academic citations [83], clinical knowledge [84], and cultural references [85]. EAI systems inherit these shortcomings in the physical world, answering user questions with deceptive or incorrect information [86]. Because VLAs fuse vision and language, their hallucinatory failures can be spatially grounded—e.g., misidentifying an object in view and then generating a plausible yet unsafe action plan around it. And although automated home assistants like Amazon","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"70.03.01","quick_ref":"Perlo2025","paper_title":"Embodied AI: Emerging Risks and Opportunities for Policy Action","level":"Risk Sub-Category","risk_category":"Economic Risks ","risk_subcategory":"Labour Displacement ","description":"\"While virtual AI applications will likely displace certain types of human cognitive labor, EAI systems could significantly replace or displace physical human labor [90]. At a minimum, EAI will likely augment the type of work that humans perform [91, 92].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"70.03.02","quick_ref":"Perlo2025","paper_title":"Embodied AI: Emerging Risks and Opportunities for Policy Action","level":"Risk Sub-Category","risk_category":"Economic Risks ","risk_subcategory":"Socioeconomic Inequality ","description":"\"Along with displacing labor, EAI could significantly exacerbate wealth inequalities. Those who have access to or own EAI systems will be able to automate labor and perform many tasks significantly better or faster than those without access. These significant productivity advantages will potentially concentrate wealth and exacerbate domestic and international inequality [98, 99].\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.2"},{"ev_id":"70.03.03","quick_ref":"Perlo2025","paper_title":"Embodied AI: Emerging Risks and Opportunities for Policy Action","level":"Risk Sub-Category","risk_category":"Economic Risks ","risk_subcategory":"Power concentration","description":"\"EAI deployment could accelerate the consolidation of economic and political power. Unlocking increasing returns to capital for EAI owners, EAI will decrease employers’ reliance on and responsiveness to the needs of human labor [101].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"70.04.01","quick_ref":"Perlo2025","paper_title":"Embodied AI: Emerging Risks and Opportunities for Policy Action","level":"Risk Sub-Category","risk_category":"Social Risks ","risk_subcategory":"Bias and discrimination","description":"\"Like virtual applications of AI, EAI can display bias towards and dis- criminate against users. When EAI systems are placed in positions of power, their biases could have significant impacts on fairness in everyday interactions and on general social dynamics [105, 106].\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"70.04.02","quick_ref":"Perlo2025","paper_title":"Embodied AI: Emerging Risks and Opportunities for Policy Action","level":"Risk Sub-Category","risk_category":"Social Risks ","risk_subcategory":"Lack of accountability and liability","description":"\"Determining responsibility when EAI causes harm requires new accountability and liability frameworks that address the complexities of highly autonomous physical systems. Human users may disagree with decisions taken by expert EAI systems, raising significant questions of delegation and responsibility [108]. Lack of EAI accountability could lead to confusion for users and breakdowns in traditional justice systems [109].\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"70.04.03","quick_ref":"Perlo2025","paper_title":"Embodied AI: Emerging Risks and Opportunities for Policy Action","level":"Risk Sub-Category","risk_category":"Social Risks ","risk_subcategory":"Lack of transparency, explainability, and trust","description":"\"Understanding how AI reaches conclusions or why AI systems perform specific actions motivates an entire branch of interpretability research [111], but physical embodiment raises the stakes for understanding these systems. For example, transparency of planned actions and explainability of decision-making is crucial when an AV suddenly changes lanes. A lack of transparency and explainability could lead to a lack of trust, which could become a critical and socially destabilizing issue with the widespread deployment of EAI [112–114].\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.4"},{"ev_id":"70.04.04","quick_ref":"Perlo2025","paper_title":"Embodied AI: Emerging Risks and Opportunities for Policy Action","level":"Risk Sub-Category","risk_category":"Social Risks ","risk_subcategory":"Unhealthy or dangerous human-EAI relationships","description":"\"Constant access to and interaction with EAI systems could foster dangerous human dependence or romantic attachment [115]. People may depend on EAI systems for physical pleasure [116]. The physical presence and human-like features of EAI systems may significantly amplify the dependency issues already observed with conversational AI [117, 118]. People may easily fall in love with EAI systems, only to be distraught when these systems are altered or have their memories reset [119].\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"71.01.02","quick_ref":"Tang2025","paper_title":"Risks of AI Scientists: Prioritizing Safeguarding Over Autonomy","level":"Risk Sub-Category","risk_category":"Scientific Domain of Agents","risk_subcategory":"Biological Risks ","description":"\"Biological risks encompass the dangerous modification of pathogens and unethical manipulation of genetic material, potentially leading to unforeseen biohazardous outcomes.\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":4,"subdomain":"4.2"},{"ev_id":"71.01.04","quick_ref":"Tang2025","paper_title":"Risks of AI Scientists: Prioritizing Safeguarding Over Autonomy","level":"Risk Sub-Category","risk_category":"Scientific Domain of Agents","risk_subcategory":"Physical (Mechanical ) Risks ","description":"\"Physical (mechanical) risks are associated with robotics and automated systems, which could lead to equipment malfunctions or physical harm in laboratory settings.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"71.02.03","quick_ref":"Tang2025","paper_title":"Risks of AI Scientists: Prioritizing Safeguarding Over Autonomy","level":"Risk Sub-Category","risk_category":"User Intent ","risk_subcategory":"Unintended Consequences ","description":"\"Unpredictable and unforeseen outcomes from purposeful actions\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":null,"subdomain":null},{"ev_id":"72.02.01","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Loss of Control Risks ","risk_subcategory":"Passive loss of control ","description":"\"...where humans gradually stop exercising meaningful oversight due to automation bias, the AI systems' inherent complexity, or competitive pressures\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":5,"subdomain":"5.2"},{"ev_id":"72.03.00","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Category","risk_category":"Accident Risks ","risk_subcategory":null,"description":"\"Risks arising from operational failures, model misjudgments, or improper human operation of AI systems deployed in safety-critical infrastructure, where single points of failure can trigger cascading catastrophic consequences.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"72.03.01","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Accident Risks ","risk_subcategory":"Nuclear Power Systems","description":"\"General-purpose AI deployed for reactor monitoring, control system optimization, or emergency response coordination could misinterpret sensor data, fail to recognize critical safety conditions, or make erroneous control decisions during emergency scenarios. Given the catastrophic potential of nuclear accidents, even minor AI reasoning errors in safety-critical functions could lead to core meltdowns, radiation releases, or widespread contamination affecting hundreds of thousands of people across international borders.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"72.03.02","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Accident Risks ","risk_subcategory":"Impact on Financial Stability","description":"\"The integration of general-purpose AI into high-frequency trading, market-making, or systemic risk management could exacerbate systemic risk by exhibiting unexpected behavioral patterns during market stress. Moreover, the concentration of a few homogeneous foundation models across financial institutions may foster correlated decision-making and herd-following behaviors. The widespread adoption of AI agents could also amplify volatility through emergent phenomena from multi-agent interactions.23 All of these could precipitate a cascading global-scale financial system instability, with potentia","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"72.03.03","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Accident Risks ","risk_subcategory":"Other Critical Infrastructure Control Systems","description":"\"General-purpose AI deployed in power grid management, water treatment facilities, telecommunications networks, or transportation coordination systems could misinterpret operational data, fail to anticipate cascading failure modes, or make control decisions that destabilize interconnected infrastructure networks. Infrastructure failures could result in widespread blackouts, contaminated water supplies, communications breakdowns, and the collapse of essential services supporting hundreds of thousands of people.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"72.04.00","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Category","risk_category":"Systemic Risks ","risk_subcategory":null,"description":"\"Systemic risks emerge from widespread deployment of general-purpose AI beyond the risks directly posed by capabilities of individual models. These risks arise from structural mismatches between AI technology and existing social, economic, and institutional frameworks, creating vulnerabilities that transcend individual model-level interventions and require coordinated industry-wide and societal-level responses.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"72.04.01","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Labor Market Disruption and Economic Displacement:","description":"\"Rapid automation enabled by general-purpose AI could trigger widespread unemployment across knowledge work sectors, creating skill mismatches faster than retraining programs can address. Unlike previous technological transitions, AI’s broad capabilities may simultaneously affect multiple industries, potentially overwhelming social safety nets and creating systemic economic instability, particularly in regions heavily dependent on jobs susceptible to AI automation.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"72.04.02","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Market Concentration and Infrastructure Dependencies:","description":"\"Over-reliance on a limited number of dominant AI providers could create critical single points of failure across essential services. Market concentration in AI development may lead to scenarios where technical failures, cyber-attacks, or policy decisions by a few companies could simultaneously disrupt healthcare systems, financial services, transportation networks, and communication infrastructure, creating cascading failures across interconnected critical systems.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"72.04.03","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Global AI Research and Development Divides:","description":"\"Asymmetric AI development capabilities between nations could exacerbate geopolitical tensions and create new forms of technological dependency. Countries lacking advanced AI capabilities may become increasingly dependent on foreign AI systems for critical functions, while AI-leading nations may gain disproportionate influence over global economic and security systems, potentially destabilizing international cooperation frameworks.\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"72.04.04","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Social Cohesion and Equity Disruption:","description":"\"Systemic deployment of biased AI systems could exacerbate existing social discrimination and prejudice at unprecedented scales, while unequal access to advanced AI capabilities may widen socioeconomic disparities and create new forms of social stratification that challenge traditional social order.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"72.05.03","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Model Capabilities ","risk_subcategory":"Automated AI R&D capability","description":"\"Self-modification and self-improvement capabilities. The model is able to restructure its own architecture or develop derivative AI systems with enhanced functions, expanding capabilities and improving performance. In the absence of effective regulation, automated AI R&D may lead to rapid AI system iteration, forming capability increment cycles and ultimately exceeding human understanding and control capabilities.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.2"},{"ev_id":"72.06.05","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Model Propensities","risk_subcategory":"Multi-agent collusion propensity:","description":"\"Multiple agents tend to coordinate actions through covert means to maximize common interests (possibly harming third-party interests or evading regulation), even if individual agents are designed with safety constraints, their collusive behavior may still trigger systemic risks such as market manipulation or cascading failures that are difficult to detect and mitigate, and may develop specialized communication protocols to avoid monitoring.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"73.01.02","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Agentic LLMs Pose Novel Risks ","risk_subcategory":"Natural Language Underspecifies Goals ","description":"\"For LLM-agents, both the goal and environment observations are typically specified in the prompt through natural language. While natural language may provide a richer and more natural means of specifying goals than alternatives such as hand-engineering objective functions, natural language still suffers from underspecification (Grice, 1975; Piantadosi et al., 2012). Furthermore, in practice, users may neglect fully specifying their goals, especially the information pertaining to elements of the environment that ought not to be changed (the classic frame problem (Shanahan, 2016)). Such undersp","entity":"Other","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.1"},{"ev_id":"73.01.05","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Agentic LLMs Pose Novel Risks ","risk_subcategory":"Safety Risks from Affordances Provided to LLM-agents","description":"\"The capabilities of LLM-agents can be enhanced in significant ways by providing the LLM-agent with novel affordances, e.g. the ability to browse the web (Nakano et al., 2021), to manipulate objects in the physical world (Ahn et al., 2022; Huang et al., 2022a), to create and instruct copies of itself (Richards, 2023), to create and use new tools (Wang et al., 2023a), etc. Affordances can create additional risks, as they often increase the impact area of the language-agent, and they amplify the consequences of an agent’s failures and enable novel forms of failure modes (Ruan et al., 2023; Pan e","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.2"},{"ev_id":"73.04.00","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Category","risk_category":"LLM-Systems Can Be Untrustworthy","risk_subcategory":null,"description":"\"A key desideratum for an LLM from a user’s perspective is ‘trustworthiness’, i.e. assurance of reliability and consistent performance, and absence of any accidental harm caused by the technology to the user.16 Providing assurance that an LLM-based system will not cause accidental harm remains a major open challenge. Harms may either occur directly due to the flawed nature of LLMs, e.g. an LLM generating toxic language or behaving inappropriately in some other ways, or may occur due to improper usage by a user, e.g. automation bias due to a user’s overreliance on LLM.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"73.04.01","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"LLM-Systems Can Be Untrustworthy","risk_subcategory":"Harms of Representation and Other Biases","description":"\"A pretrained LLM generally has many of the stereotypical biases commonly present in the human society (Touvron et al., 2023). This makes it difficult for users to trust that LLMs will work well for them and not produce unfair or biased responses. Appropriate finetuning can effectively limit the bias displayed in LLM outputs in a variety of situations, e.g. when models are explicitly prompted with stereotypes (Wang et al., 2023k), but it does not ‘solve’ the problem. Even after finetuning, biases often resurface when deliberately elicited (Wang et al., 2023k), or under novel scenarios, e.g. in","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"73.04.02","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"LLM-Systems Can Be Untrustworthy","risk_subcategory":"Inconsistent Performance across and within Domains","description":"\"Estimating true capabilities of an LLM is a difficult task (c.f. Section 3.3), especially for naive users unfamiliar with the brittle nature of machine learning technologies. Exaggeration of model capabilities by the developers (Lambert, 2023; Blair-Stanek et al., 2023), and issues such as task-contamination (Roberts et al., 2023b), underrepresentation of tasks or domains (Wu et al., 2023a; McCoy et al., 2023), and prompt-sensitivity (Anthropic, 2023d) may cause a user to misestimate the true capabilities of a model. This lack of reliability can undermine user trust or cause harm if a user ba","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"73.04.03","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"LLM-Systems Can Be Untrustworthy","risk_subcategory":"Overreliance","description":"\"If a user begins to excessively trust an LLM, this may cause them to develop an overreliance on the LLM. Overreliance can result in automation bias (Kupfer et al., 2023), and can cause errors of omission (user choosing not to verify the validity of a response) and errors of commission (user believing and acting on the basis of the LLM’s response, even if it contradicts their own knowledge) (Skitka et al., 1999). It can be particularly dangerous in domains where the user may lack relevant expertise to robustly scrutinize the LLM responses. This is particularly a source of risk for LLMs because","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"73.05.00","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Category","risk_category":"Socioeconomic Impacts of LLM May Be Highly Disruptive","risk_subcategory":null,"description":"\"The rapid evolution of LLMs brings significant socioeconomic opportunities and challenges, impacting the workforce, income inequality, education, and global economic development. Many of these challenges are systemic in nature, constituting what economists refer to as general equilibrium effects. These challenges do not arise directly from LLMs causing harm to users but rather from their indirect effects on the socioeconomic equilibrium.\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"73.05.01","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Socioeconomic Impacts of LLM May Be Highly Disruptive","risk_subcategory":"Effects on the Workforce","description":"\"Rapid advances in LLMs pose three distinct sets of challenges for workers’ incomes (Korinek and Stiglitz, 2019; Susskind, 2023). First, they are likely to accelerate the rate of job turnover and disruption —– affecting more workers, including more highly skilled workers, and making the adjustment process for society more difficult than what we were used to from prior technological advances...Second, although technological progress means that society may produce more wealth overall, there is a risk that the general-purpose nature of LLMs may lead to progress that is biased against labor, meani","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"73.05.02","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Socioeconomic Impacts of LLM May Be Highly Disruptive","risk_subcategory":"Effects on Inequality","description":"\"LLMs could potentially worsen socioeconomic inequalities (Capraro et al., 2023). Effects on inequal- ity are closely linked to the effects of LLMs on workers but ultimately depend on how the fruits of technological progress are distributed...First, if the role and compensation of capital rise and the role and compensation of labor decline in an LLM-powered economy, inequality may go up because work is the main source of income for the majority of people...Second, the large fixed cost of training cutting-edge LLMs and the network effects involved imply that the market for the most advanced LLM","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"73.05.03","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Socioeconomic Impacts of LLM May Be Highly Disruptive","risk_subcategory":"Global Economic Development","description":"\"Many of the themes and challenges that we discussed above come together when analyzing the socioeconomic effects on developing countries. The workforce of developing countries may suffer from a retrenchment of outsourcing as many simple cognitive tasks that used to be performed in developing countries — for example, in call centers –— can be automated with LLMs. This may adversely affect the economies of the poor countries (Georgieva, 2024).\"","entity":"Other","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"73.06.00","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Category","risk_category":"Corporate power may impeded effective governance ","risk_subcategory":null,"description":"\"The increasing power and influence of large corporations may make effective governance difficult. There exists a power asymmetry between corporate entities profiting from LLMs and other social groups (e.g. civil society). State-of-the-art LLMs are developed by or in partnership with, some of the world’s largest private tech companies...This poses a risk of governance protocols related to LLMs becoming excessively favorable to tech companies, potentially leading to regulatory capture at the cost of the interests of other societal groups, particularly marginalized communities who have historica","entity":"Other","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"73.07.01","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Jailbreaks and Prompt Injections Threaten Security of LLMs","risk_subcategory":"Exploiting Limited Generalization of Safety Finetuning","description":"\"Safety tuning is performed over a much narrower distribution compared to the pretraining distribution. This leaves the model vulnerable to attacks that exploit gaps in the generalization of the safety training, e.g. using encoded text (Wei et al., 2023c) or low-resource languages (Deng et al., 2023a; Yong et al., 2023) (see also Section 3.2).\"","entity":"Other","intent":"Unintentional","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"74.01.00","quick_ref":"Wang2025","paper_title":"A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy","level":"Risk Category","risk_category":"Inherent Risk ","risk_subcategory":null,"description":"\"In terms of inherent risk, LLMs could potentially reveal sensitive information from their utilized corpora for pre-training or fine-tuning, thereby raising issues of privacy leakage [37, 145, 226]. Meanwhile, it is well-known that LLMs may experi- ence hallucinations, resulting in the production of texts that are inaccurate and misleading [194]. Finally, since the values embedded in LLM-generated texts usually directly reflect the distribution of their training data, often sourced from the Internet, there exists a substantial risk that LLMs will overfit to a narrow set of human values or even","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":null},{"ev_id":"74.01.06","quick_ref":"Wang2025","paper_title":"A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy","level":"Risk Sub-Category","risk_category":"Inherent Risk ","risk_subcategory":"Hallucination","description":"\"Despite the rapid advancement of LLMs, hallucinations have emerged as one of the most vital concerns surrounding their use [54, 79, 86, 110, 242]. Hallucinations are often referred to as LLMs’ generating content that is nonfactual or unfaithful to the provided information [54, 79, 86, 242]. Therefore, hallucinations can be typically categorized into two main classes. The first is factuality hallucination, which describes the discrepancy between LLMs’ generated content and real-world facts. For example, if LLMs mistakenly take Charles Lindbergh as the first person who walked on the moon, it is","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"74.01.07","quick_ref":"Wang2025","paper_title":"A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy","level":"Risk Sub-Category","risk_category":"Inherent Risk ","risk_subcategory":"Value-related risks in LLMs","description":"\"As the general capabilities of LLM-empowered systems improve, the negative consequences and risks induced by these systems also get increasingly alarming accordingly, especially in high-stakes areas [28, 146]. Although they may not be intentionally introduced, severe problematic issues related to human values can be raised. Specifically, even before language models become extremely large, pre-trained language models have already exhibited a certain degree of value judgments. For example, Schramowski et al. [171] reveal the existence of the moral direction with the sentence embeddings of moral","entity":"Other","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.1"}]}