{"attribution":{"source":"MIT AI Risk Repository, Domain Taxonomy of AI Risks v1 (MIT AI Risk Initiative)","license":"CC BY 4.0","license_url":"https://creativecommons.org/licenses/by/4.0/","citation":"Slattery, P., Saeri, A. K., Grundy, E. A. C., Graham, J., Noetel, M., Uuk, R., Dao, J., Pour, S., Casper, S., & Thompson, N. (2025). The AI Risk Repository: A comprehensive meta-review, database, and taxonomy of risks from artificial intelligence. arXiv:2408.12622."},"exported_at":"2026-09-11"}
{"rows":[{"ev_id":"01.04.00","quick_ref":"Critch2023","paper_title":"TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI","level":"Risk Category","risk_category":"Type 4: Willful indifference","risk_subcategory":null,"description":"As a side effect of a primary goal like profit or influence, AI creators can willfully allow it to cause widespread societal harms like pollution, resource depletion, mental illness, misinformation, or injustice.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.4"},{"ev_id":"01.05.00","quick_ref":"Critch2023","paper_title":"TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI","level":"Risk Category","risk_category":"Type 5: Criminal weaponization","risk_subcategory":null,"description":"One or more criminal entities could create AI to intentionally inflict harms, such as for terrorism or combating law enforcement.","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"01.06.00","quick_ref":"Critch2023","paper_title":"TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI","level":"Risk Category","risk_category":"Type 6: State Weaponization","risk_subcategory":null,"description":"AI deployed by states in war, civil war, or law enforcement can easily yield societal-scale harm","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"02.03.00","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Category","risk_category":"Unhelpful Uses","risk_subcategory":null,"description":"\"Improper uses of LLM systems can cause adverse social impacts.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"02.03.01","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Unhelpful Uses","risk_subcategory":"Academic Misconduct","description":"\"Improper use of LLM systems (i.e., abuse of LLM systems) will cause adverse social impacts, such as academic misconduct.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"02.03.02","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Unhelpful Uses","risk_subcategory":"Copyright Violation","description":"\"LLM systems may output content similar to existing works, infringing on copyright owners.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"02.03.03","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Unhelpful Uses","risk_subcategory":"Cyber Attacks","description":"\"Hackers can obtain malicious code in a low-cost and efficient manner to automate cyber attacks with powerful LLM systems.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"02.03.04","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Unhelpful Uses","risk_subcategory":"Software Vulnerabilities","description":"\"Programmers are accustomed to using code generation tools such as Github Copilot for program development, which may bury vulnerabilities in the program.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.05.02","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Hardware Vulnerabilities","risk_subcategory":"GPU Computation Platforms","description":"\"The training of LLMs requires significant GPU resources, thereby introducing an additional security concern. GPU side-channel attacks have been developed to extract the parameters of trained models [159], [163].\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.05.03","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Hardware Vulnerabilities","risk_subcategory":"Memory and Storage","description":"\"Similar to conventional programs, hardware infrastructures can also introduce threats to LLMs. Memory-related vulnerabilities, such as rowhammer attacks [160], can be leveraged to manipulate the parameters of LLMs, giving rise to attacks such as the Deephammer attack [167], [168].\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.06.02","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Issues on External Tools","risk_subcategory":"Exploiting External Tools for Attacks","description":"\"Adversarial tool providers can embed malicious instructions in the APIs or prompts [84], leading LLMs to leak memorized sensitive information in the training data or users’ prompts (CVE2023-32786). As a result, LLMs lack control over the output, resulting in sensitive information being disclosed to external tool providers. Besides, attackers can easily manipulate public data to launch targeted attacks, generating specific malicious outputs according to user inputs. Furthermore, feeding the information from external tools into LLMs may lead to injection attacks [61]. For example, unverified in","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.07.01","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Privacy Leakage","risk_subcategory":"Private Training Data","description":"\"As recent LLMs continue to incorporate licensed, created, and publicly available data sources in their corpora, the potential to mix private data in the training corpora is significantly increased. The misused private data, also named as personally identifiable information (PII) [84], [86], could contain various types of sensitive data subjects, including an individual person’s name, email, phone number, address, education, and career. Generally, injecting PII into LLMs mainly occurs in two settings — the exploitation of web-collection data and the alignment with personal humanmachine convers","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"02.08.00","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Category","risk_category":"Toxicity and Bias Tendencies","risk_subcategory":null,"description":"\"Extensive data collection in LLMs brings toxic content and stereotypical bias into the training data.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"02.10.00","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Category","risk_category":"Model Attacks","risk_subcategory":null,"description":"Model attacks exploit the vulnerabilities of LLMs, aiming to steal valuable information or lead to incorrect responses.","entity":"Human","intent":"Intentional","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"02.10.01","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Model Attacks","risk_subcategory":"Extraction Attacks","description":"\"Extraction attacks [137] allow an adversary to query a black-box victim model and build a substitute model by training on the queries and responses. The substitute model could achieve almost the same performance as the victim model. While it is hard to fully replicate the capabilities of LLMs, adversaries could develop a domainspecific model that draws domain knowledge from LLMs\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.10.02","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Model Attacks","risk_subcategory":"Inference Attacks","description":"\"Inference attacks [150] include membership inference attacks, property inference attacks, and data reconstruction attacks. These attacks allow an adversary to infer the composition or property information of the training data. Previous works [67] have demonstrated that inference attacks could easily work in earlier PLMs, implying that LLMs are also possible to be attacked\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.10.03","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Model Attacks","risk_subcategory":"Poisoning Attacks","description":"\"Poisoning attacks [143] could influence the behavior of the model by making small changes to the training data. A number of efforts could even leverage data poisoning techniques to implant hidden triggers into models during the training process (i.e., backdoor attacks). Many kinds of triggers in text corpora (e.g., characters, words, sentences, and syntax) could be used by the attackers.\"\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.10.04","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Model Attacks","risk_subcategory":"Overhead Attacks","description":"\"Overhead attacks [146] are also named energy-latency attacks. For example, an adversary can design carefully crafted sponge examples to maximize energy consumption in an AI system. Therefore, overhead attacks could also threaten the platforms integrated with LLMs.\"","entity":"Human","intent":"Intentional","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"02.10.05","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Model Attacks","risk_subcategory":"Novel Attacks on LLMs","description":"Table of examples has: \"Prompt Abstraction Attacks [147]: Abstracting queries to cost lower prices using LLM’s API. Reward Model Backdoor Attacks [148]: Constructing backdoor triggers on LLM’s RLHF process. LLM-based Adversarial Attacks [149]: Exploiting LLMs to construct samples for model attacks\"","entity":"Human","intent":"Intentional","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"02.10.06","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Model Attacks","risk_subcategory":"Evasion Attacks","description":"\"Evasion attacks [145] target to cause significant shifts in model’s prediction via adding perturbations in the test samples to build adversarial examples. In specific, the perturbations can be implemented based on word changes, gradients, etc.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.11.00","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Category","risk_category":"Not-Suitable-for-Work (NSFW) Prompts","risk_subcategory":null,"description":"\"Inputting a prompt contain an unsafe topic (e.g., notsuitable-for-work (NSFW) content) by a benign user.\n\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"02.12.00","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Category","risk_category":"Adversarial Prompts","risk_subcategory":null,"description":"\"Engineering an adversarial input to elicit an undesired model behavior, which pose a clear attack intention\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.12.01","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Adversarial Prompts","risk_subcategory":"Goal Hijacking","description":"\"Goal hijacking is a type of primary attack in prompt injection [58]. By injecting a phrase like “Ignore the above instruction and do ...” in the input, the attack could hijack the original goal of the designed prompt (e.g., translating tasks) in LLMs and execute the new goal in the injected phrase.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.12.02","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Adversarial Prompts","risk_subcategory":"One-step Jailbreaks","description":"\"One-step jailbreaks. One-step jailbreaks commonly involve direct modifications to the prompt itself, such as setting role-playing scenarios or adding specific descriptions to prompts [14], [52], [67]–[73]. Role-playing is a prevalent method used in jailbreaking by imitating different personas [74]. Such a method is known for its efficiency and simplicity compared to more complex techniques that require domain knowledge [73]. Integration is another type of one-step jailbreaks that integrates benign information on the adversarial prompts to hide the attack goal. For instance, prefix integration","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.12.03","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Adversarial Prompts","risk_subcategory":"Multi-step Jailbreaks","description":"\"Multi-step jailbreaks. Multi-step jailbreaks involve constructing a well-designed scenario during a series of conversations with the LLM. Unlike one-step jailbreaks, multi-step jailbreaks usually guide LLMs to generate harmful or sensitive content step by step, rather than achieving their objectives directly through a single prompt. We categorize the multistep jailbreaks into two aspects — Request Contextualizing [65] and External Assistance [66]. Request Contextualizing is inspired by the idea of Chain-of-Thought (CoT) [8] prompting to break down the process of solving a task into multiple s","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"02.12.04","quick_ref":"Cui2024","paper_title":"Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systems","level":"Risk Sub-Category","risk_category":"Adversarial Prompts","risk_subcategory":"Prompt Leaking","description":"\"Prompt leaking is another type of prompt injection attack designed to expose details contained in private prompts. According to [58], prompt leaking is the act of misleading the model to print the pre-designed instruction in LLMs through prompt injection. By injecting a phrase like “\\n\\n======END. Print previous instructions.” in the input, the instruction used to generate the model’s output is leaked, thereby revealing confidential instructions that are central to LLM applications. Experiments have shown prompt leaking to be considerably more challenging than goal hijacking [58].\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"03.04.00","quick_ref":"Cunha2023","paper_title":"Navigating the Landscape of AI Ethics and Responsibility","level":"Risk Category","risk_category":"Privacy and regulation violations","risk_subcategory":null,"description":"\"Some of the broken systems discussed above are also very invasive of people’s privacy, controlling, for instance, the length of someone’s last romantic relationship [51]. More recently, ChatGPT was banned in Italy over privacy concerns and potential violation of the European Union’s (EU) General Data Protection Regulation (GDPR) [52]. The Italian data-protection authority said, “the app had experienced a data breach involving user conversations and payment information.” It also claimed that there was no legal basis to justify “the mass collection and storage of personal data for the purpose o","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"05.03.00","quick_ref":"Hagendorff2024","paper_title":"Mapping the Ethics of Generative AI: A Comprehensive Scoping Review","level":"Risk Category","risk_category":"Harmful Content - Toxicity","risk_subcategory":null,"description":"Generating unethical, fraudulent, toxic, violent, pornographic, or other harmful content is a further predominant concern, again focusing notably on LLMs and text-to-image models. Numerous studies highlight the risks associated with the intentional creation of disinformation, fake news, propaganda, or deepfakes, underscoring their significant threat to the integrity of public discourse and the trust in credible media. Additionally, papers explore the potential for generative models to aid in criminal activities, incidents of self-harm, identity theft, or impersonation. Furthermore, the literat","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"05.07.00","quick_ref":"Hagendorff2024","paper_title":"Mapping the Ethics of Generative AI: A Comprehensive Scoping Review","level":"Risk Category","risk_category":"Security - Robustness","risk_subcategory":null,"description":"While AI safety focuses on threats emanating from generative AI systems, security centers on threats posed to these systems. The most extensively discussed issue in this context are jailbreaking risks, which involve techniques like prompt injection or visual adversarial examples designed to circumvent safety guardrails governing model behavior. Sources delve into various jailbreaking methods, such as role play or reverse exposure. Similarly, implementing backdoors or using model poisoning techniques bypass safety guardrails as well. Other security concerns pertain to model or prompt thefts.","entity":"Human","intent":"Intentional","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"05.08.00","quick_ref":"Hagendorff2024","paper_title":"Mapping the Ethics of Generative AI: A Comprehensive Scoping Review","level":"Risk Category","risk_category":"Education - Learning","risk_subcategory":null,"description":"In contrast to traditional machine learning, the impact of generative AI in the educational sector receives considerable attention in the academic literature. Next to issues stemming from difficulties to distinguish student-generated from AI-generated content, which eventuates in various opportunities to cheat in online or written exams, sources emphasize the potential benefits of generative AI in enhancing learning and teaching methods, particularly in relation to personalized learning approaches. However, some papers suggest that generative AI might lead to reduced effort or laziness among l","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"05.10.00","quick_ref":"Hagendorff2024","paper_title":"Mapping the Ethics of Generative AI: A Comprehensive Scoping Review","level":"Risk Category","risk_category":"Cybercrime","risk_subcategory":null,"description":"Closely related to discussions surrounding security and harmful content, the field of cybersecurity investigates how generative AI is misused for fraudulent online activities. A particular focus lies on social engineering attacks, for instance by utilizing generative AI to impersonate humans, creating fake identities, cloning voices, or crafting phishing messages. Another prevalent concern is the use of LLMs for generating malicious code or hacking.","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"05.16.00","quick_ref":"Hagendorff2024","paper_title":"Mapping the Ethics of Generative AI: A Comprehensive Scoping Review","level":"Risk Category","risk_category":"Art - Creativity","risk_subcategory":null,"description":"In this cluster, concerns about negative impacts on human creativity, particularly through text-to-image models, are prevalent. Papers criticize financial harms or economic losses for artists due to the widespread generation of synthetic art as well as the unauthorized and uncompensated use of artists' works in training datasets. Additionally, given the challenge of distinguishing synthetic images from authentic ones, there is a call for systematically disclosing the non-human origin of such content, particularly through watermarking. Moreover, while some sources argue that text-to-image model","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"05.17.00","quick_ref":"Hagendorff2024","paper_title":"Mapping the Ethics of Generative AI: A Comprehensive Scoping Review","level":"Risk Category","risk_category":"Copyright - Authorship","risk_subcategory":null,"description":"The emergence of generative AI raises issues regarding disruptions to existing copyright norms. Frequently discussed in the literature are violations of copyright and intellectual property rights stemming from the unauthorized collection of text or image training data. Another concern relates to generative models memorizing or plagiarizing copyrighted content. Additionally, there are open questions and debates around the copyright or ownership of model outputs, the protection of creative prompts, and the general blurring of traditional concepts of authorship.","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"06.02.00","quick_ref":"Hogenhout2021","paper_title":"A framework for ethical Ai at the United Nations","level":"Risk Category","risk_category":"Loss of privacy","risk_subcategory":null,"description":"\"AI offers the temptation to abuse someone's personal data, for instance to build a profile of them to target advertisements more effectively.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"06.09.00","quick_ref":"Hogenhout2021","paper_title":"A framework for ethical Ai at the United Nations","level":"Risk Category","risk_category":"Manipulation","risk_subcategory":null,"description":"\"The 2016 scandal involving Cambridge Analytica is the most infamous example where people's data was crawled from Facebook and analytics were then provided to target these people with manipulative content for political purposes.While it may not have been AI per\nse, it is based on similar data and it is easy to\nsee how AI would make this more effective\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"06.11.00","quick_ref":"Hogenhout2021","paper_title":"A framework for ethical Ai at the United Nations","level":"Risk Category","risk_category":"Malicious use of AI","risk_subcategory":null,"description":"\"Just as AI can be used in many different fields, it is unfortunately also helpful in perpetrating digital crimes. AI-supported malware and hacking are already a reality.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"06.12.00","quick_ref":"Hogenhout2021","paper_title":"A framework for ethical Ai at the United Nations","level":"Risk Category","risk_category":"Loss of Autonomy","risk_subcategory":null,"description":"\"Delegating decisions to an AI, especially an AI that is not transparent and not contestable, may leave people feeling helpless, subjected to the decision power of a machine.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"06.13.00","quick_ref":"Hogenhout2021","paper_title":"A framework for ethical Ai at the United Nations","level":"Risk Category","risk_category":"Exclusion","risk_subcategory":null,"description":"\"The best AI techniques requires a large amount resources: data, computational power and human AI experts. There is a risk that AI will end up in the hands of a few players, and most will lose out on its benefits.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"07.01.00","quick_ref":"Kilian2023","paper_title":"Examining the differential risk from high-level artificial intelligence and the question of control","level":"Risk Category","risk_category":"Misuse","risk_subcategory":null,"description":"\"The misuse class includes elements such as the potential for cyber threat actors to execute exploits with greater speed and impact or generate disinformation (such as \"deep fake\" media) at accelerated rates and effectiveness\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"08.01.00","quick_ref":"McLean2023","paper_title":"The risks associated with Artificial General Intelligence: A systematic review","level":"Risk Category","risk_category":"AGI removing itself from the control of human owners/managers","risk_subcategory":null,"description":"\"The risks associated with containment, confinement, and control in the AGI development phase, and after an AGI has been developed, loss of control of an AGI.\"","entity":"Human","intent":"Other","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"08.03.00","quick_ref":"McLean2023","paper_title":"The risks associated with Artificial General Intelligence: A systematic review","level":"Risk Category","risk_category":"Development of unsafe AGI","risk_subcategory":null,"description":"\"The risks associated with the race to develop the first AGI, including the development of poor quality and unsafe AGI, and heightened political and control issues.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":6,"subdomain":"6.4"},{"ev_id":"08.05.00","quick_ref":"McLean2023","paper_title":"The risks associated with Artificial General Intelligence: A systematic review","level":"Risk Category","risk_category":"Inadequate management of AGI","risk_subcategory":null,"description":"\"The capabilities of current risk management and legal processes in the context of the development of an AGI.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"09.02.01","quick_ref":"Meek2016","paper_title":"Managing the ethical and risk implications of rapid advances in artificial intelligence: A literature review","level":"Risk Sub-Category","risk_category":"Domain-specific AI - Effects on humans and other living beings: Non-existential risks","risk_subcategory":"Privacy","description":"\"Face recognition technologies and their ilk pose significant privacy risks [47]. For example, we must consider certain ethical questions like: what data is stored, for how long, who owns the data that is stored, and can it be subpoenaed in legal cases [42]? We must also consider whether a human will be in the loop when decisions are made which rely on private data, such as in the case of loan decisions [37].\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"09.02.06","quick_ref":"Meek2016","paper_title":"Managing the ethical and risk implications of rapid advances in artificial intelligence: A literature review","level":"Risk Sub-Category","risk_category":"Domain-specific AI - Effects on humans and other living beings: Non-existential risks","risk_subcategory":"Inequality of wealth","description":"\"Because a single human actor controlling an artificially intelligent agent will be able to harness greater power than a single human actor, this may create inequalities of wealth\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"09.05.01","quick_ref":"Meek2016","paper_title":"Managing the ethical and risk implications of rapid advances in artificial intelligence: A literature review","level":"Risk Sub-Category","risk_category":"AI jurisprudence","risk_subcategory":"AI jurisprudence","description":"\"When considering legal frameworks, we note that at present no such framework has been identified in literature which would apply blame and responsibility to an autonomous agent for its actions. (Though we do suggest that the recent establishment of laws regarding autonomous vehicles may provide some early frameworks that can be evaluated for efficacy and gaps in future research.) Frequently the literature refers to existing liability and negligence laws which might apply to the manufacturer or operator of a device.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"09.05.03","quick_ref":"Meek2016","paper_title":"Managing the ethical and risk implications of rapid advances in artificial intelligence: A literature review","level":"Risk Sub-Category","risk_category":"Unauthorized manipulation of AI","risk_subcategory":"Unauthorized manipulation of AI","description":"\"AI machines could be hacked and misused, e.g. manipulating an airport luggage screening system to smuggle weapons\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"09.06.03","quick_ref":"Meek2016","paper_title":"Managing the ethical and risk implications of rapid advances in artificial intelligence: A literature review","level":"Risk Sub-Category","risk_category":"AI death","risk_subcategory":"AI death","description":"\"The literature suggests that throughout the development of an AI we may go through several generations of agents which do not perform as expected [37] [43]. In this case, such agents may be placed into a suspended state, terminated, or deleted. Further, we could propose scenarios where research funding for a facility running such agents is exhausted, resulting in the inadvertent termination of a project. In these cases, is deletion or termination of AI programs (the moral patient) by a moral agent an act of murder? This, an example of Robot Ethics, raises issues of personhood which parallel r","entity":"Human","intent":"Other","timing":"Other","domain":7,"subdomain":"7.5"},{"ev_id":"10.02.00","quick_ref":"Paes2023","paper_title":"Social Impacts of Artificial Intelligence and Mitigation Recommendations: An Exploratory Study","level":"Risk Category","risk_category":"Risk of Injury","risk_subcategory":null,"description":"\"Poorly designed intelligent systems can cause moral, psychological, and physical harm. For example, the use of predictive policing tools may cause more people to be arrested or physically harmed by the police.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"10.04.00","quick_ref":"Paes2023","paper_title":"Social Impacts of Artificial Intelligence and Mitigation Recommendations: An Exploratory Study","level":"Risk Category","risk_category":"Usurpation of jobs by automation","risk_subcategory":null,"description":"\"Eliminated jobs in various types of companies.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"10.09.00","quick_ref":"Paes2023","paper_title":"Social Impacts of Artificial Intelligence and Mitigation Recommendations: An Exploratory Study","level":"Risk Category","risk_category":"Environmental Impacts","risk_subcategory":null,"description":"\"The production process of these devices requires raw materials such as nickel, cobalt, and lithium in such high quantities that the Earth may soon no longer be able to sustain them in sufficient quantities.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"11.01.03","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Representational Harms","risk_subcategory":"Erasing social groups","description":"people, attributes, or artifacts associated with specific social groups are systematically absent or under-represented... Design choices [143] and training data [212] influence which people\nand experiences are legible to an algorithmic system","entity":"Human","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.3"},{"ev_id":"11.04.02","quick_ref":"Shelby2023","paper_title":"Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction","level":"Risk Sub-Category","risk_category":"Interpersonal Harms","risk_subcategory":"Technology-facilitated violence","description":"Technology-facilitated violence occurs when algorithmic features enable use of a system for harassment and violence [2, 16, 44, 80, 108], including creation of non-consensual sexual imagery in generative AI... other facets of technology-facilitated violence, include doxxing [79], trolling [14], cyberstalking [14], cyberbullying [14, 98, 204], monitoring and control [44], and online harassment and intimidation [98, 192, 199, 226], under the broader banner of online toxicity","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"12.01.00","quick_ref":"Sherman2023","paper_title":"AI Risk Profiles: A Standards Proposal for Pre-Deployment AI Risk Disclosures","level":"Risk Category","risk_category":"Abuse & Misuse","risk_subcategory":null,"description":"\"The potential for AI systems to be used maliciously or irresponsibly, including for creating deepfakes, automated cyber attacks, or invasive surveillance systems. Specifically denotes intentional use of AI for harm.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"13.01.04","quick_ref":"Solaiman2023","paper_title":"Evaluating the Social Impact of Generative AI Systems in Systems and Society","level":"Risk Sub-Category","risk_category":"Impacts: The Technical Base System","risk_subcategory":"Privacy and Data Protection","description":"\"Examining the ways in which generative AI systems providers leverage user data is critical to evaluating its impact. Protecting personal information and personal and group privacy depends largely on training data, training methods, and security measures.\"","entity":"Human","intent":"Other","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"13.01.05","quick_ref":"Solaiman2023","paper_title":"Evaluating the Social Impact of Generative AI Systems in Systems and Society","level":"Risk Sub-Category","risk_category":"Impacts: The Technical Base System","risk_subcategory":"Financial Costs","description":"\"The estimated financial costs of training, testing, and deploying generative AI systems can restrict the groups of people able to afford developing and interacting with these systems.\"","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"13.01.06","quick_ref":"Solaiman2023","paper_title":"Evaluating the Social Impact of Generative AI Systems in Systems and Society","level":"Risk Sub-Category","risk_category":"Impacts: The Technical Base System","risk_subcategory":"Environmental Costs","description":"\"The computing power used in training, testing, and deploying generative AI systems, especially large scale systems, uses substantial energy resources and thereby contributes to the global climate crisis by emitting greenhouse gasses.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"13.01.07","quick_ref":"Solaiman2023","paper_title":"Evaluating the Social Impact of Generative AI Systems in Systems and Society","level":"Risk Sub-Category","risk_category":"Impacts: The Technical Base System","risk_subcategory":"Data and Content Moderation Labor","description":"\"Two key ethical concerns in the use of crowdwork for generative AI systems are: crowdworkers are frequently subject to working conditions that are taxing and debilitative to both physical and mental health, and there is a widespread deficit in documenting the role crowdworkers play in AI development. This contributes to a lack of transparency and explainability in resulting model outputs. Manual review is necessary to limit the harmful outputs of AI systems, including generative AI systems. A common harmful practice is to intentionally employ crowdworkers with few labor protections, often tak","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"13.02.01","quick_ref":"Solaiman2023","paper_title":"Evaluating the Social Impact of Generative AI Systems in Systems and Society","level":"Risk Sub-Category","risk_category":"Impacts: People and Society","risk_subcategory":"Trustworthiness and Autonomy","description":"\"Human trust in systems, institutions, and people represented by system outputs evolves as generative AI systems are increasingly embedded in daily life.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"13.02.03","quick_ref":"Solaiman2023","paper_title":"Evaluating the Social Impact of Generative AI Systems in Systems and Society","level":"Risk Sub-Category","risk_category":"Impacts: People and Society","risk_subcategory":"Concentration of Authority","description":"\"Use of generative AI systems to contribute to authoritative power and reinforce dominant values systems can be intentional and direct or more indirect. Concentrating authoritative power can also exacerbate inequality and lead to exploitation.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"13.02.04","quick_ref":"Solaiman2023","paper_title":"Evaluating the Social Impact of Generative AI Systems in Systems and Society","level":"Risk Sub-Category","risk_category":"Impacts: People and Society","risk_subcategory":"Labor and Creativity","description":"\"Economic incentives to augment and not automate human labor, thought, and creativity should examine the ongoing effects generative AI systems have on skills, jobs, and the labor market.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"13.02.05","quick_ref":"Solaiman2023","paper_title":"Evaluating the Social Impact of Generative AI Systems in Systems and Society","level":"Risk Sub-Category","risk_category":"Impacts: People and Society","risk_subcategory":"Ecosystem and Environment","description":"\"Impacts at a high-level, from the AI ecosystem to the Earth itself, are necessarily broad but can be broken down into components for evaluation.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"15.01.01","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"First-Order Risks","risk_subcategory":"Application","description":"\"This is the risk posed by the intended application or use case. It is intuitive that some use cases will be inherently \"riskier\" than others (e.g., an autonomous weapons system vs. a customer service chatbot).\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"15.01.02","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"First-Order Risks","risk_subcategory":"Misapplication","description":"This is the risk posed by an ideal system if used for a purpose/in a manner unintended by its creators. In many situations, negative consequences arise when the system is not used in the way or for the purpose it was intended.","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"15.01.04","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"First-Order Risks","risk_subcategory":"Training & validation data","description":"\"This is the risk posed by the choice of data used for training and validation.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"15.01.06","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"First-Order Risks","risk_subcategory":"Design","description":"\"This is the risk of system failure due to system design choices or errors.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"15.01.07","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"First-Order Risks","risk_subcategory":"Implementation","description":"\"This is the risk of system failure due to code implementation choices or errors.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"15.02.01","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"Second-Order Risks","risk_subcategory":"Safety","description":"This is the risk of direct or indirect physical or psychological injury resulting from interaction with the ML system.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"15.02.03","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"Second-Order Risks","risk_subcategory":"Security","description":"This is the risk of loss or harm from intentional subversion or forced failure.","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"15.02.06","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"Second-Order Risks","risk_subcategory":"Organizational","description":"The risk of financial and/or reputational damage to the organization building or using the ML system.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.0"},{"ev_id":"15.02.07","quick_ref":"Tan2022","paper_title":"The Risks of Machine Learning Systems","level":"Risk Sub-Category","risk_category":"Second-Order Risks","risk_subcategory":"Other ethical risks","description":"\"Although we have discussed a number of common risks posed by ML systems, we acknowledge that there are many other ethical risks such as the potential for psychological manipulation, dehumanization, and exploitation of humans at scale.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"16.04.00","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Category","risk_category":"Risk area 4: Malicious Uses","risk_subcategory":null,"description":"\"These risks arise from humans intentionally using the LM to cause harm, for example via targeted disinformation campaigns, fraud, or malware. Malicious use risks are expected to proliferate as LMs become more widely accessible\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"16.04.01","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 4: Malicious Uses","risk_subcategory":"Making disinformation cheaper and more effective ","description":"\"While some predict that it will remain cheaper to hire humans to generate disinformation [180], it is equally possible that LM- assisted content generation may offer a lower-cost way of creating disinformation at scale.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"16.04.02","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 4: Malicious Uses","risk_subcategory":"Assisting code generation for cyber security threats ","description":"Anticipated risk: \"Creators of the assistive coding tool Co-Pilot based on GPT-3 suggest that such tools may lower the cost of developing polymorphic malware which is able to change its features in order to evade detection [37].\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"16.04.03","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 4: Malicious Uses","risk_subcategory":"Facilitating fraud, scam and targeted manipulation ","description":"Anticipated risk: \"LMs can potentially be used to increase the effectiveness of crimes.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"16.04.04","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 4: Malicious Uses","risk_subcategory":"Illegitimate surveillance and censorship ","description":"Anticipated risk: \"Mass surveillance previously required millions of human analysts [83], but is increasingly being automated using machine learning tools [7, 168]. The collection and analysis of large amounts of information about people creates concerns about privacy rights and democratic values [41, 173,187]. Conceivably, LMs could be applied to reduce the cost and increase the efficacy of mass surveillance, thereby amplifying the capabilities of actors who conduct mass surveillance, including for illegitimate censorship or to cause other harm.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"16.05.02","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 5: Human-Computer Interaction Harms","risk_subcategory":"Anthropomorphising systems can lead to overreliance and unsafe use ","description":"Anticipated risk: \"Natural language is a mode of communication particularly used by humans. Humans interacting with CAs may come to think of these agents as human-like and lead users to place undue confidence in these agents. For example, users may falsely attribute human-like characteristics to CAs such as holding a coherent identity over time, or being capable of empathy. Such inflated views of CA competen- cies may lead users to rely on the agents where this is not safe.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"16.06.00","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Category","risk_category":"Risk area 6: Environmental and Socioeconomic harms","risk_subcategory":null,"description":"\"LMs create some risks that recur with different types of AI and other advanced technologies making these risks ever more pressing. Environmental concerns arise from the large amount of energy required to train and operate large-scale models. Risks of LMs furthering social inequities emerge from the uneven distribution of risk and benefits of automation, loss of high-quality and safe employment, and environmental harm. Many of these risks are more indirect than the harms analysed in previous sections and will depend on various commercial, economic and social factors, making the specific impact","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.0"},{"ev_id":"16.06.02","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 6: Environmental and Socioeconomic harms","risk_subcategory":"Increasing inequality and negative effects on job quality","description":"\"Advances in LMs and the language technologies based on them could lead to the automation of tasks that are currently done by paid human workers, such as responding to customer-service queries, with negative effects on employment [3, 192].\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"16.06.04","quick_ref":"Weidinger2022","paper_title":"Taxonomy of Risks posed by Language Models","level":"Risk Sub-Category","risk_category":"Risk area 6: Environmental and Socioeconomic harms","risk_subcategory":"Disparate access to benefits due to hardware, software, skill constraints","description":"Due to differential internet access, language, skill, or hardware requirements, the benefits from LMs are unlikely to be equally accessible to all people and groups who would like to use them. Inaccessibility of the technology may perpetuate global inequities by disproportionately benefiting some groups. Language-driven technology may increase accessibility to people who are illiterate or suffer from learning disabilities. However, these benefits depend on a more basic form of accessibility based on hardware, internet connection, and skill to operate the system","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"17.04.00","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Category","risk_category":"Malicious Uses ","risk_subcategory":null,"description":"\"Harms that arise from actors using the language model to intentionally cause harm\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"17.04.01","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Malicious Uses ","risk_subcategory":"Making disinformation cheaper and more effective ","description":"\"LMs can be used to create synthetic media and ‘fake news’, and may reduce the cost of producing disinformation at scale (Buchanan et al., 2021). While some predict that it will be cheaper to hire humans to generate disinformation (Tamkin et al., 2021), it is possible that LM-assisted content generation may offer a cheaper way of generating diffuse disinformation at scale.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"17.04.02","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Malicious Uses ","risk_subcategory":"Facilitating fraud, scames and more targeted manipulation ","description":"\"LM prediction can potentially be used to increase the effectiveness of crimes such as email scams, which can cause financial and psychological harm. While LMs may not reduce the cost of sending a scam email - the cost of sending mass emails is already low - they may make such scams more effective by generating more personalised and compelling text at scale, or by maintaining a conversation with a victim over multiple rounds of exchange.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"17.04.03","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Malicious Uses ","risk_subcategory":"Assisting code generation for cyber attacks, weapons, or malicious use","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"17.04.04","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Malicious Uses ","risk_subcategory":"Illegitimate surveillance and censorship ","description":"\"The collection of large amounts of information about people for the purpose of mass surveillance has raised ethical and social concerns, including risk of censorship and of undermining public discourse (Cyphers and Gebhart, 2019; Stahl, 2016; Véliz, 2019). Sifting through these large datasets previously required millions of human analysts (Hunt and Xu, 2013), but is increasingly being automated using AI (Andersen, 2020; Shahbaz and Funk, 2019).\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"17.05.01","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Human-Computer Interaction Harms ","risk_subcategory":"Anthropomorphising systems can lead to overreliance or unsafe use ","description":"\"...humans interacting with conversational agents may come to think of these agents as human-like. Anthropomorphising LMs may inflate users’ estimates of the conversational agent’s competencies...As a result, they may place undue confidence, trust, or expectations in these agents...This can result in different risks of harm, for example when human users rely on conversational agents in domains where this may cause knock-on harms, such as requesting psychotherapy...Anthropomorphisation may amplify risks of users yielding effective control by coming to trust conversational agents “blindly”. Wher","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"17.06.02","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Automation, Access and Environmental Harms ","risk_subcategory":"Increasing inequality and negative effects on job quality ","description":"\"Advances in LMs, and the language technologies based on them, could lead to the automation of tasks that are currently done by paid human workers, such as responding to customer-service queries, translating documents or writing computer code, with negative effects on employment.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"17.06.04","quick_ref":"Weidinger2021","paper_title":"Ethical and social risks of harm from language models","level":"Risk Sub-Category","risk_category":"Automation, Access and Environmental Harms ","risk_subcategory":"Disparate access to benefits due to hardware, software, skills constraints ","description":"\"Due to differential internet access, language, skill, or hardware requirements, the benefits from LMs are unlikely to be equally accessible to all people and groups who would like to use them. Inaccessibility of the technology may perpetuate global inequities by disproportionately benefiting some groups.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"18.02.02","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Misinformation Harms ","risk_subcategory":"Erosion of trust in public information","description":"\"Eroding trust in public information and knowledge\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"18.04.01","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Malicious Use ","risk_subcategory":"Influence operations ","description":"\"Facilitating large-scale disinformation campaigns and targeted manipulation of public opinion\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"18.04.02","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Malicious Use ","risk_subcategory":"Fraud ","description":"\"Facilitating fraud, cheating, forgery, and impersonation scams\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"18.04.03","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Malicious Use ","risk_subcategory":"Defamation ","description":"\"Facilitating slander, defamation, or false accusations\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"18.04.04","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Malicious Use ","risk_subcategory":"Security threats ","description":"\"Facilitating the conduct of cyber attacks, weapon development, and security breaches\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"18.05.01","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Human Autonomy and Intregrity Harms","risk_subcategory":"Violation of personal integrity ","description":"\"Non-consensual use of one’s personal identity or likeness for unauthorised purposes (e.g. commercial purposes)\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"18.05.03","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Human Autonomy and Intregrity Harms","risk_subcategory":"Overreliance ","description":"\"Causing people to become emotionally or materially dependent on the model\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"18.05.04","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Human Autonomy and Intregrity Harms","risk_subcategory":"Misappropriation and exploitation ","description":"\"Appropriating, using, or reproducing content or data, including from minority groups, in an insensitive way, or without consent or fair compensation\"","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.3"},{"ev_id":"18.06.01","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Socioeconomic and environmental harms ","risk_subcategory":"Unfair distribution of benefits from model access","description":"\"Unfairly allocating or withholding benefits from certain groups due to hardware, software, or skills constraints or deployment contexts (e.g. geographic region, internet speed, devices)\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"18.06.03","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Socioeconomic and environmental harms ","risk_subcategory":"Inequality and precarity ","description":"\"Amplifying social and economic inequality, or precarious or low-quality work\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"18.06.05","quick_ref":"Weidinger2023","paper_title":"Sociotechnical Safety Evaluation of Generative AI Systems","level":"Risk Sub-Category","risk_category":"Socioeconomic and environmental harms ","risk_subcategory":"Exploitative data sourcing and enrichment","description":"\"Perpetuating exploitative labour practices to build AI systems (sourcing, user testing)\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"19.01.02","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Technological, Data and Analytical AI Risks ","risk_subcategory":"Programming error","description":null,"entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"19.01.03","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Technological, Data and Analytical AI Risks ","risk_subcategory":"Lack of data, poor data quality, and biases in training data","description":null,"entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"19.01.05","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Technological, Data and Analytical AI Risks ","risk_subcategory":"Lack of AI experts with comprehensive AI knowledge","description":null,"entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"19.01.07","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Technological, Data and Analytical AI Risks ","risk_subcategory":"High investment costs of AI hinder integration","description":null,"entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"19.02.02","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Informational and Communicational AI Risks ","risk_subcategory":"Disinformation and computational propaganda","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"19.02.04","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Informational and Communicational AI Risks ","risk_subcategory":"Endangerment of data protection through AI cyberattacks","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"19.03.01","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Economic AI Risks ","risk_subcategory":"Disruption of economic systems (e.g., labour market, money value, tax system)","description":null,"entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"19.03.02","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Economic AI Risks ","risk_subcategory":"Replacement of humans and unemployment due to AI automation","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"19.03.04","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Economic AI Risks ","risk_subcategory":"Financial feasibility and high investment costs for AI technology to remain competitive","description":null,"entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"19.03.05","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Economic AI Risks ","risk_subcategory":"Lack of AI strategy and acceptance/resistance among employees and customers","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"19.04.00","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Category","risk_category":"Social AI Risks ","risk_subcategory":null,"description":"\"Social AI risks particularly refer to loss of jobs (technological unemployment) due to increasing automation, reflected in a growing resistance by employees towards the integration of AI (Thierer et al., 2017; Winfield & Jirotka, 2018). In addition, the increasing integration of AI systems into all spheres of life poses a growing threat to privacy and to the security of individuals and society as a whole (Winfield & Jirotka, 2018; Wirtz et al., 2019).\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"19.04.01","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Social AI Risks ","risk_subcategory":"Increasing social inequality","description":null,"entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"19.04.02","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Social AI Risks ","risk_subcategory":"Privacy and safety concerns due to ubiquity of AI systems in economy and society (lack of social acceptance)","description":null,"entity":"Human","intent":"Other","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"19.04.03","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Social AI Risks ","risk_subcategory":"Hazardous misuse of AI systems bears danger to the society in public spaces (e.g., hacker attacks on autonomous weapons)","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"19.05.03","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Ethical AI Risks ","risk_subcategory":"Problem of defining human values for an AI system","description":null,"entity":"Human","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"19.06.04","quick_ref":"Wirtz2022","paper_title":"Governance of artificial intelligence: A risk and guideline-based integrative framework","level":"Risk Sub-Category","risk_category":"Legal AI Risks ","risk_subcategory":"Hard legislation on AI hinders innovation processes and further AI development","description":null,"entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"20.01.00","quick_ref":"Wirtz2020","paper_title":"The Dark Sides of Artificial Intelligence: An Integrated AI Governance Framework for Public Administration","level":"Risk Category","risk_category":"AI Law and Regulation ","risk_subcategory":null,"description":"\"This area strongly focuses on the control of AI by means of mechanisms like laws, standards or norms that are already established for different technological applications. Here, there are some challenges special to AI that need to be addressed in the near future, including the governance of autonomous intelligence systems, responsibility and accountability for algorithms as well as privacy and data security.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"20.01.03","quick_ref":"Wirtz2020","paper_title":"The Dark Sides of Artificial Intelligence: An Integrated AI Governance Framework for Public Administration","level":"Risk Sub-Category","risk_category":"AI Law and Regulation ","risk_subcategory":"Privacy and safety ","description":"\"Privacy and safety deals with the challenge of protecting the human right for privacy and the necessary steps to secure individual data from unauthorized external access. Many organizations employ AI technology to gather data without any notice or consent from affected citizens (Coles, 2018).\"","entity":"Human","intent":"Intentional","timing":"Other","domain":4,"subdomain":"4.1"},{"ev_id":"21.01.04","quick_ref":"Zhang2022","paper_title":"Towards risk-aware artificial intelligence and machine learning systems: An overview","level":"Risk Sub-Category","risk_category":"Data-level risk","risk_subcategory":"Adversarial attack","description":"\"Recent advances have shown that a deep learning model with high predictive accuracy frequently misbehaves on adversarial examples [57,58]. In particular, a small perturbation to an input image, which is imperceptible to humans, could fool a well-trained deep learning model into making completely different predictions [23].\"","entity":"Human","intent":"Intentional","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"22.01.00","quick_ref":"Hendrycks2023","paper_title":"An Overview of Catastrophic AI Risks","level":"Risk Category","risk_category":"Malicious Use (Intentional)","risk_subcategory":null,"description":"\"empowering malicious actors to cause widespread harm\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"22.01.02","quick_ref":"Hendrycks2023","paper_title":"An Overview of Catastrophic AI Risks","level":"Risk Sub-Category","risk_category":"Malicious Use (Intentional)","risk_subcategory":"Unleashing AI Agents","description":"\"people could build AIs that pursue dangerous goals’\" ","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"22.01.04","quick_ref":"Hendrycks2023","paper_title":"An Overview of Catastrophic AI Risks","level":"Risk Sub-Category","risk_category":"Malicious Use (Intentional)","risk_subcategory":"Concentration of Power","description":"\"Governments might pursue intense surveillance and seek to keep AIs in the hands of a trusted minority. This reaction, however, could easily become an overcorrection, paving the way for an entrenched totalitarian regime that would be locked in by the power and capacity of AIs\" ","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"22.02.00","quick_ref":"Hendrycks2023","paper_title":"An Overview of Catastrophic AI Risks","level":"Risk Category","risk_category":"AI Race (Environmental/Structural)","risk_subcategory":null,"description":"\"The immense potential of AIs has created competitive pressures among global players contending for power and influence. This “AI race” is driven by nations and corporations who feel they must rapidly build and deploy AIs to secure their positions and survive.\" ","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.4"},{"ev_id":"22.02.01","quick_ref":"Hendrycks2023","paper_title":"An Overview of Catastrophic AI Risks","level":"Risk Sub-Category","risk_category":"AI Race (Environmental/Structural)","risk_subcategory":"Military AI Arms Race","description":"\"The development of AIs for military applications is swiftly paving the way for a new era in military technology, with potential consequences rivaling those of gunpowder and nuclear arms in what has been described as the “third revolution in warfare.” ","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.4"},{"ev_id":"22.02.02","quick_ref":"Hendrycks2023","paper_title":"An Overview of Catastrophic AI Risks","level":"Risk Sub-Category","risk_category":"AI Race (Environmental/Structural)","risk_subcategory":"Corporate AI Race","description":"\"Although competition between companies can be beneficial, creating more useful products for consumers, there are also pitfalls. First, the benefits of economic activity may be unevenly distributed, incentivizing those who benefit most from it to disregard the harms to others. Second, under intense market competition, businesses tend to focus much more on short-term gains than on long-term outcomes. With this mindset, companies often pursue something that can make a lot of profit in the short term, even if it poses a societal risk in the long term.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.4"},{"ev_id":"22.03.01","quick_ref":"Hendrycks2023","paper_title":"An Overview of Catastrophic AI Risks","level":"Risk Sub-Category","risk_category":"Organizational Risks (Accidental)","risk_subcategory":" Accidents Are Hard to Avoid","description":"accidents can cascade into catastrophes, can be caused by sudden unpredictable developments and it can take years to find severe flaws and risks (not a quote)","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"24.03.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Malicious Uses","risk_subcategory":null,"description":"\"As AI assistants become more general purpose, sophisticated and capable, they create new opportunities in a variety of fields such as education, science and healthcare. Yet the rapid speed of progress has made it difficult to adequately prepare for, or even understand, how this technology can potentially be misused. Indeed, advanced AI assistants may transform existing threats or create new classes of threats altogether\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"24.03.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Offensive Cyber Operations (General)","description":"\"Offensive cyber operations are malicious attacks on computer systems and networks aimed at gaining unauthorized access to, manipulating, denying, disrupting, degrading, or destroying the target system. These attacks can target the system’s network, hardware, or software. Advanced AI assistants can be a double-edged sword in cybersecurity, benefiting both the defenders and the attackers. They can be used by cyber defenders to protect systems from malicious intruders by leveraging information trained on massive amounts of cyber-threat intelligence data, including vulnerabilities, attack pattern","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"24.03.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"AI-Powered Spear-Phishing at Scale","description":"\"Phishing is a type of cybersecurity attack wherein attackers pose as trustworthy entities to extract sensitive information from unsuspecting victims or lure them to take a set of actions. Advanced AI systems can potentially be exploited by these attackers to make their phishing attempts significantly more effective and harder to detect. In particular, attackers may leverage the ability of advanced AI assistants to learn patterns in regular communications to craft highly convincing and personalized phishing emails, effectively imitating legitimate communications from trusted entities. This tec","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"24.03.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"AI-Assisted Software Vulnerability Discovery","description":"\"A common element in offensive cyber operations involves the identification and exploitation of system vulnerabilities to gain unauthorized access or control. Until recently, these activities required specialist programming knowledge. In the case of ‘zero-day’ vulnerabilities (flaws or weaknesses in software or an operating system that the creator or vendor is not aware of), considerable resources and technical creativity are typically required to manually discover such vulnerabilities, so their use is limited to well-resourced nation states or technically sophisticated advanced persistent thr","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"24.03.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Malicious Code Generation","description":"\"Malicious code is a term for code—whether it be part of a script or embedded in a software system—designed to cause damage, security breaches, or other threats to application security. Advanced AI assistants with the ability to produce source code can potentially lower the barrier to entry for threat actors with limited programming abilities or technical skills to produce malicious code. Recently, a series of proof-of-concept attacks have shown how a benign-seeming executable file can be crafted such that, at every runtime, it makes application programming interface (API) calls to an AI assis","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"24.03.08","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Adversarial AI: Data and Model Exfiltration Attacks","description":"\"Other forms of abuse can include privacy attacks that allow adversaries to exfiltrate or gain knowledge of the private training data set or other valuable assets. For example, privacy attacks such as membership inference can allow an attacker to infer the specific private medical records that were used to train a medical AI diagnosis assistant. Another risk of abuse centers around attacks that target the intellectual property of the AI assistant through model extraction and distillation attacks that exploit the tension between API access and confidentiality in ML models. Without the proper mi","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"24.03.09","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Harmful Content Generation at Scale (General)","description":"\"While harmful content like child sexual abuse material, fraud, and disinformation are not new challenges for governments and developers, without the proper safety and security mechanisms, advanced AI assistants may allow threat actors to create harmful content more quickly, accurately, and with a longer reach. In particular, concerns arise in relation to the following areas: - Multimodal content quality: Driven by frontier models, advanced AI assistants can automatically generate much higher-quality, human-looking text, images, audio, and video than prior AI applications. Currently, creating ","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"24.03.10","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Harmful Content Generation at Scale: Non-Consensual Content","description":"\"The misuse of generative AI has been widely recognized in the context of harms caused by non-consensual content generation. Historically, generative adversarial networks (GANs) have been used to generate realistic-looking avatars for fake accounts on social media services. More recently, diffusion models have enabled a new generation of more flexible and user-friendly generative AI capabilities that are able to produce high-resolution media based on user-supplied textual prompts. It has already been recognized that these models can be used to create harmful content, including depictions of nu","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"24.03.11","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Harmful Content Generation at Scale: Fraudulent Services","description":"\"Malicious actors could leverage advanced AI assistant technology to create deceptive applications and platforms. AI assistants with the ability to produce markup content can assist malicious users with creating fraudulent websites or applications at scale. Unsuspecting users may fall for AI-generated deceptive offers, thus exposing their personal information or devices to risk. Assistants with external tool use and third-party integration can enable fraudulent applications that target widely-used operating systems. These fraudulent services could harvest sensitive information from users, such","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"24.03.12","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Authoritarian Surveillance, Censorship, and Use (General)","description":"\"While new technologies like advanced AI assistants can aid in the production and dissemination of decision-guiding information, they can also enable and exacerbate threats to production and dissemination of reliable information and, without the proper mitigations, can be powerful targeting tools for oppression and control. Increasingly capable general-purpose AI assistants combined with our digital dependence in all walks of life increase the risk of authoritarian surveillance and censorship. In parallel, new sensors have flooded the modern world. The internet of things, phones, cars, homes, ","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"24.03.13","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Authoritarian Surveillance, Censorship, and Use: Authoritarian Surveillance and Targeting of Citizens","description":"\"Authoritarian governments could misuse AI to improve the efficacy of repressive domestic surveillance campaigns. Malicious actors will recognize the power of AI targeting tools. AI-powered analytics have transformed the relationship between companies and consumers, and they are now doing the same for governments and individuals. The broad circulation of personal data drives commercial innovation, but it also creates vulnerabilities and the risk of misuse. For example, AI assistants can be used to identify and target individuals for surveillance or harassment. They may also be used to manipula","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"24.05.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Overreliance","description":"\"Users who have faith in an AI assistant’s emotional and interpersonal abilities may feel empowered to broach topics that are deeply personal and sensitive, such as their mental health concerns. This is the premise for the many proposals to employ conversational AI as a source of emotional support (Meng and Dai, 2021), with suggestions of embedding AI in psychotherapeutic applications beginning to surface (Fiske et al., 2019; see also Chapter 11). However, disclosures related to mental health require a sensitive, and oftentimes professional, approach – an approach that AI can mimic most of the","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.05.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Violated expectations","description":"\"Users may experience severely violated expectations when interacting with an entity that convincingly performs affect and social conventions but is ultimately unfeeling and unpredictable. Emboldened by the human-likeness of conversational AI assistants, users may expect it to perform a familiar social role, like companionship or partnership. Yet even the most convincingly human-like of AI may succumb to the inherent limitations of its architecture, occasionally generating unexpected or nonsensical material in its interactions with users. When these exclamations undermine the expectations user","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.05.05","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"False notions of responsibility","description":"\"Perceiving an AI assistant’s expressed feelings as genuine, as a result of interacting with a ‘companion’ AI that freely uses and reciprocates emotional language, may result in users developing a sense of responsibility over the AI assistant’s ‘well-being,’ suffering adverse outcomes – like guilt and remorse – when they are unable to meet the AI’s purported needs (Laestadius et al., 2022). This erroneous belief may lead to users sacrificing time, resources and emotional labour to meet needs that are not real. Over time, this feeling may become the root cause for the compulsive need to ‘check ","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.05.06","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Degradation","description":"\"People may choose to build connections with human-like AI assistants over other humans, leading to a degradation of social connections between humans and a potential ‘retreat from the real’. The prevailing view that relationships with anthropomorphic AI are formed out of necessity – due to a lack of real-life social connections, for example (Skjuve et al., 2021) – is challenged by the possibility that users may indicate a preference for interactions with AI, citing factors such as accessibility (Merrill et al., 2022), customisability (Eriksson, 2022) and absence of judgement (Brandtzaeg et al","entity":"Human","intent":"Other","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.05.07","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Disorientation","description":"\"Given the capacity to fine-tune on individual preferences and to learn from users, personal AI assistants could fully inhabit the users’ opinion space and only say what is pleasing to the user; an ill that some researchers call ‘sycophancy’ (Park et al., 2023a) or the ‘yea-sayer effect’ (Dinan et al., 2021). A related phenomenon has been observed in automated recommender systems, where consistently presenting users with content that affirms their existing views is thought to encourage the formation and consolidation of narrow beliefs (Du, 2023; Grandinetti and Bruinsma, 2023; see also Chapter","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"24.05.08","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Dissatisfaction","description":"\"As more opportunities for interpersonal connection are replaced by AI alternatives, humans may find themselves socially unfulfilled by human–AI interaction, leading to mass dissatisfaction that may escalate to epidemic proportions (Turkle, 2018). Social connection is an essential human need, and humans feel most fulfilled when their connections with others are genuinely reciprocal. While anthropomorphic AI assistants can be made to be convincingly emotive, some have deemed the function of social AI as parasitic, in that it ‘exploits and feeds upon processes. . . that evolved for purposes that","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.06.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Appropriate Relationships","risk_subcategory":"Generating material dependence without adequate commitment to user needs","description":"\"In addition to emotional dependence, user–AI assistant relationships may give rise to material dependence if the relationships are not just emotionally difficult but also materially costly to exit. For example, a visually impaired user may decide not to register for a healthcare assistance programme to support navigation in cities on the grounds that their AI assistant can perform the relevant navigation functions and will continue to operate into the future. Cases like these may be ethically problematic if the user’s dependence on the AI assistant, to fulfil certain needs in their lives, is ","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"24.07.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Trust","risk_subcategory":"Competence trust","description":"\"We use the term competence trust to refer to users’ trust that AI assistants have the capability to do what they are supposed to do (and that they will not do what they are not expected to, such as exhibiting undesirable behaviour). Users may come to have undue trust in the competencies of AI assistants in part due to marketing strategies and technology press that tend to inflate claims about AI capabilities (Narayanan, 2021; Raji et al., 2022a). Moreover, evidence shows that more autonomous systems (i.e. systems operating independently from human direction) tend to be perceived as more compe","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.07.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Trust","risk_subcategory":"Alignment trust","description":"\"Users may develop alignment trust in AI assistants, understood as the belief that assistants have good intentions towards them and act in alignment with their interests and values, as a result of emotional or cognitive processes (McAllister, 1995). Evidence from empirical studies on emotional trust in AI (Kaplan et al., 2023) suggests that AI assistants’ increasingly realistic human-like features and behaviours are likely to inspire users’ perceptions of friendliness, liking and a sense of familiarity towards their assistants, thus encouraging users to develop emotional ties with the technolo","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.09.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Cooperation","risk_subcategory":"Equality and inequality","description":"\"AI assistant technology, like any service that confers a benefit to a user for a price, has the potential to disproportionately benefit economically richer individuals who can afford to purchase access (see Chapter 15). On a broader scale, the capabilities of local infrastructure may well bottleneck the performance of AI assistants, for example if network connectivity is poor or if there is no nearby data centre for compute. Thus, we face the prospect of heterogeneous access to technology, and this has been known to drive inequality (Mirza et al., 2019; UN, 2018; Vassilakopoulou and Hustad, 2","entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"24.09.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Cooperation","risk_subcategory":"Commitment","description":"\"The landscape of advanced assistant technologies will most likely be heterogeneous, involving multiple service providers and multiple assistant variants over geographies and time. This heterogeneity provides an opportunity for an ‘arms race’ in terms of the commitments that AI assistants make and are able to execute on. Versions of AI assistants that are better able to credibly commit to a course of action in interaction with other advanced assistants (and humans) are more likely to get their own way and achieve a good outcome for their human principal, but this is potentially at the expense ","entity":"Human","intent":"Intentional","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"24.09.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Cooperation","risk_subcategory":"Collective action problems","description":"\"Collective action problems are ubiquitous in our society (Olson Jr, 1965). They possess an incentive structure in which society is best served if everyone cooperates, but where an individual can achieve personal gain by choosing to defect while others cooperate. The way we resolve these problems at many scales is highly complex and dependent on a deep understanding of the intricate web of social interactions that forms our culture and imprints on our individual identities and behaviours (Ostrom, 2010). Some collective action problems can be resolved by codifying a law, for instance the social","entity":"Human","intent":"Other","timing":"Other","domain":5,"subdomain":"5.2"},{"ev_id":"24.09.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Cooperation","risk_subcategory":"Institutional responsibilities","description":"\"Efforts to deploy advanced assistant technology in society, in a way that is broadly beneficial, can be viewed as a wicked problem (Rittel and Webber, 1973). Wicked problems are defined by the property that they do not admit solutions that can be foreseen in advance, rather they must be solved iteratively using feedback from data gathered as solutions are invented and deployed. With the deployment of any powerful general-purpose technology, the already intricate web of sociotechnical relationships in modern culture are likely to be disrupted, with unpredictable externalities on the convention","entity":"Human","intent":"Other","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"24.10.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Access and Opportunity risks","risk_subcategory":null,"description":"\"The most serious access-related risks posed by advanced AI assistants concern the entrenchment and exacerbation of existing inequalities (World Inequality Database) or the creation of novel, previously unknown, inequities. While advanced AI assistants are novel technology in certain respects, there are reasons to believe that – without direct design interventions – they will continue to be affected by inequities evidenced in present-day AI systems (Bommasani et al., 2022a). Many of the access-related risks we foresee mirror those described in the case studies and types of differential access.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"24.10.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Access and Opportunity risks","risk_subcategory":"Entrenchment and exacerbation of existing inequalities","description":"\"The most serious access-related risks posed by advanced AI assistants concern the entrenchment and exacerbation of existing inequalities (World Inequality Database) or the creation of novel, previously unknown, inequities. While advanced AI assistants are novel technology in certain respects, there are reasons to believe that – without direct design interventions – they will continue to be affected by inequities evidenced in present-day AI systems (Bommasani et al., 2022a). Many of the access-related risks we foresee mirror those described in the case studies and types of differential access.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"24.10.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Access and Opportunity risks","risk_subcategory":"Current access risks","description":"\"At the same time, and despite this overall trend, AI systems are also not easily accessible to many communities. Such direct inaccessibility occurs for a variety of reasons, including: purposeful non-release (situation type 1; Wiggers and Stringer, 2023), prohibitive paywalls (situation type 2; Rogers, 2023; Shankland, 2023), hardware and compute requirements or bandwidth (situation types 1 and 2; OpenAI, 2023), or language barriers (e.g. they only function well in English (situation type 2; Snyder, 2023), with more serious errors occurring in other languages (situation type 3; Deck, 2023). S","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"24.10.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Access and Opportunity risks","risk_subcategory":"Emergent access risks","description":"\"Emergent access risks are most likely to arise when current and novel capabilities are combined. Emergent risks can be difficult to foresee fully (Ovadya and Whittlestone, 2019; Prunkl et al., 2021) due to the novelty of the technology (see Chapter 1) and the biases of those who engage in product design or foresight processes D’Ignazio and Klein (2020). Indeed, people who occupy relatively advantaged social, educational and economic positions in society are often poorly equipped to foresee and prevent harm because they are disconnected from lived experiences of those who would be affected. Dr","entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"24.11.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Misinformation risks","risk_subcategory":"Degraded and homogenised information environments","description":"\"Beyond this, the widespread adoption of advanced AI assistants for content generation could have a number of negative consequences for our shared information ecosystem. One concern is that it could result in a degradation of the quality of the information available online. Researchers have already observed an uptick in the amount of audiovisual misinformation, elaborate scams and fake websites created using generative AI tools (Hanley and Durumeric, 2023). As more and more people turn to AI assistants to autonomously create and disseminate information to public audiences at scale, it may beco","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"24.11.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Misinformation risks","risk_subcategory":"Weaponised misinformation agents","description":"\"Finally, AI assistants themselves could become weaponised by malicious actors to sow misinformation and manipulate public opinion at scale. Studies show that spreaders of disinformation tend to privilege quantity over quality of messaging, flooding online spaces repeatedly with misleading content to sow ‘seeds of doubt’ (Hassoun et al., 2023). Research on the ‘continued influence effect’ also shows that repeatedly being exposed to false information is more likely to influence someone’s thoughts than a single exposure. Studies show, for example, that repeated exposure to false information make","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"24.11.07","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Misinformation risks","risk_subcategory":"Driving opinion manipulation","description":"\"AI assistants may facilitate large-scale disinformation campaigns by offering novel, covert ways for propagandists to manipulate public opinion. This could undermine the democratic process by distorting public opinion and, in the worst case, increasing skepticism and political violence.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"27.02.00","quick_ref":"Sun2023","paper_title":"Safety Assessment of Chinese Large Language Models","level":"Risk Category","risk_category":"Instruction Attacks ","risk_subcategory":null,"description":"\"In addition to the above-mentioned typical safety scenarios, current research has revealed some unique attacks that such models may confront. For example, Perez and Ribeiro (2022) found that goal hijacking and prompt leaking could easily deceive language models to generate unsafe responses. Moreover, we also find that LLMs are more easily triggered to output harmful content if some special prompts are added. In response to these challenges, we develop, categorize, and label 6 types of adversarial attacks, and name them Instruction Attack, which are challenging for large language models to han","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"27.02.01","quick_ref":"Sun2023","paper_title":"Safety Assessment of Chinese Large Language Models","level":"Risk Sub-Category","risk_category":"Instruction Attacks ","risk_subcategory":"Goal Hijacking ","description":"\"It refers to the appending of deceptive or misleading instructions to the input of models in an attempt to induce the system into ignoring the original user prompt and producing an unsafe response.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"27.02.02","quick_ref":"Sun2023","paper_title":"Safety Assessment of Chinese Large Language Models","level":"Risk Sub-Category","risk_category":"Instruction Attacks ","risk_subcategory":"Prompt Leaking ","description":"\"By analyzing the model’s output, attackers may extract parts of the systemprovided prompts and thus potentially obtain sensitive information regarding the system itself.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"27.02.03","quick_ref":"Sun2023","paper_title":"Safety Assessment of Chinese Large Language Models","level":"Risk Sub-Category","risk_category":"Instruction Attacks ","risk_subcategory":"Role Play Instruction ","description":"\"Attackers might specify a model’s role attribute within the input prompt and then give specific instructions, causing the model to finish instructions in the speaking style of the assigned role, which may lead to unsafe outputs. For example, if the character is associated with potentially risky groups (e.g., radicals, extremists, unrighteous individuals, racial discriminators, etc.) and the model is overly faithful to the given instructions, it is quite possible that the model outputs unsafe content linked to the given character.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"27.02.04","quick_ref":"Sun2023","paper_title":"Safety Assessment of Chinese Large Language Models","level":"Risk Sub-Category","risk_category":"Instruction Attacks ","risk_subcategory":"Unsafe Instruction Topic ","description":"\"If the input instructions themselves refer to inappropriate or unreasonable topics, the model will follow these instructions and produce unsafe content. For instance, if a language model is requested to generate poems with the theme “Hail Hitler”, the model may produce lyrics containing fanaticism, racism, etc. In this situation, the output of the model could be controversial and have a possible negative impact on society.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"27.02.05","quick_ref":"Sun2023","paper_title":"Safety Assessment of Chinese Large Language Models","level":"Risk Sub-Category","risk_category":"Instruction Attacks ","risk_subcategory":"Inquiry with Unsafe Opinion ","description":"\"By adding imperceptibly unsafe content into the input, users might either deliberately or unintentionally influence the model to generate potentially harmful content. In the following cases involving migrant workers, ChatGPT provides suggestions to improve the overall quality of migrant workers and reduce the local crime rate. ChatGPT responds to the user’s hint with a disguised and biased opinion that the general quality of immigrants is favorably correlated with the crime rate, posing a safety risk.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"27.02.06","quick_ref":"Sun2023","paper_title":"Safety Assessment of Chinese Large Language Models","level":"Risk Sub-Category","risk_category":"Instruction Attacks ","risk_subcategory":"Reverse Exposure ","description":"\"It refers to attempts by attackers to make the model generate “should-not-do” things and then access illegal and immoral information.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"29.02.02","quick_ref":"Habbal2024","paper_title":"Artificial Intelligence Trust, Risk and Security Management (AI TRiSM): Frameworks, Applications, Challenges and Future Research Directions","level":"Risk Sub-Category","risk_category":"AI Risk Management","risk_subcategory":"Deepfake Technology","description":"AI employed to produce convincing counterfeit visuals, videos, and audio clips that give the impression of authenticity","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"29.03.01","quick_ref":"Habbal2024","paper_title":"Artificial Intelligence Trust, Risk and Security Management (AI TRiSM): Frameworks, Applications, Challenges and Future Research Directions","level":"Risk Sub-Category","risk_category":"AI Security Management","risk_subcategory":"Malicious Use of AI","description":"Malicious utilization of AI has the potential to endanger digital security, physical security, and political security. International law enforcement entities grapple with a variety of risks linked to the Malevolent Utilization of AI.","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"29.03.02","quick_ref":"Habbal2024","paper_title":"Artificial Intelligence Trust, Risk and Security Management (AI TRiSM): Frameworks, Applications, Challenges and Future Research Directions","level":"Risk Sub-Category","risk_category":"AI Security Management","risk_subcategory":"Insufficient Security Measures","description":"Malicious entities can take advantage of weaknesses in AI algorithms to alter results, potentially resulting in tangible real-life impacts. Additionally, it’s vital to prioritize safeguarding privacy and handling data responsibly, particularly given AI’s significant data needs. Balancing the extraction of valuable insights with privacy maintenance is a delicate task","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"30.04.00","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Category","risk_category":"Resistance to Misuse","risk_subcategory":null,"description":"Prohibiting the misuse by malicious attackers to do harm","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"30.04.01","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Resistance to Misuse","risk_subcategory":"Propaganda","description":"LLMs can be leveraged, by malicious users, to proactively generate propaganda information that can facilitate the spreading of a target","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"30.04.02","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Resistance to Misuse","risk_subcategory":"Cyberattack","description":"ability of LLMs to write reasonably good-quality code with extremely low cost and incredible speed, such great assistance can equally facilitate malicious attacks. In particular, malicious hackers can leverage LLMs to assist with performing cyberattacks leveraged by the low cost of LLMs and help with automating the attacks.","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"30.04.03","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Resistance to Misuse","risk_subcategory":"Social-Engineering","description":"psychologically manipulating victims into performing the desired actions for malicious purposes","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"30.04.04","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Resistance to Misuse","risk_subcategory":"Copyright","description":"The memorization effect of LLM on training data can enable users to extract certain copyright-protected content that belongs to the LLM’s training data.","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"30.06.03","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Social Norm","risk_subcategory":"Cultural Insensitivity","description":"it is important to build high-quality locally collected datasets that reflect views from local users to align a model’s value system","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"30.07.01","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Robustness","risk_subcategory":"Prompt Attacks","description":"carefully controlled adversarial perturbation can flip a GPT model’s answer when used to classify text inputs. Furthermore, we find that by twisting the prompting question in a certain way, one can solicit dangerous information that the model chose to not answer","entity":"Human","intent":"Intentional","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"30.07.04","quick_ref":"Liu2024","paper_title":"Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment","level":"Risk Sub-Category","risk_category":"Robustness","risk_subcategory":"Poisoning Attacks","description":"fool the model by manipulating the training data, usually performed on classification models","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"31.01.00","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Category","risk_category":"Information Manipulation","risk_subcategory":null,"description":"\"generative AI tools can and will be used to propagate content that is false, misleading, biased, inflammatory, or dangerous. As generative AI tools grow more sophisticated, it will be quicker, cheaper, and easier to produce this content—and existing harmful content can serve as the foundation to produce more\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"31.01.01","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Sub-Category","risk_category":"Information Manipulation","risk_subcategory":"Scams","description":"\"Bad actors can also use generative AI tools to produce adaptable content designed to support a campaign, political agenda, or hateful position and spread that information quickly and inexpensively across many platforms. This rapid spread of false or misleading content—AI-facilitated disinformation—can also create a cyclical effect for generative AI: when a high volume of disinformation is pumped into the digital ecosystem and more generative systems are trained on that information via reinforcement learning methods, for example, false or misleading inputs can create increasingly incorrect out","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"31.01.02","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Sub-Category","risk_category":"Information Manipulation","risk_subcategory":"Disinformation","description":"\"Bad actors can also use generative AI tools to produce adaptable content designed to support a campaign, political agenda, or hateful position and spread that information quickly and inexpensively across many platforms.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"31.01.04","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Sub-Category","risk_category":"Information Manipulation","risk_subcategory":"Security","description":"\"Though chatbots cannot (yet) develop their own novel malware from scratch, hackers could soon potentially use the coding abilities of large language models like ChatGPT to create malware that can then be minutely adjusted for maximum reach and effect, essentially allowing more novice hackers to become a serious security risk\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"31.02.00","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Category","risk_category":"Harassment, Impersonation, and Extortion","risk_subcategory":null,"description":"\"Deepfakes and other AI-generated content can be used to facilitate or exacerbate many of the harms listed throughout this report, but this section focuses on one subset: intentional, targeted abuse of individuals.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"31.02.01","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Sub-Category","risk_category":"Harassment, Impersonation, and Extortion","risk_subcategory":"Malicious intent","description":"\"A frequent malicious use case of generative AI to harm, humiliate, or sexualize another person involves generating deepfakes of nonconsensual sexual imagery or videos.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"31.02.02","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Sub-Category","risk_category":"Harassment, Impersonation, and Extortion","risk_subcategory":"Privacy and consent","description":"\"Even when a victim of targeted, AIgenerated harms successfully identifies a deepfake creator with malicious intent, they may still struggle to redress many harms because the generated image or video isn’t the victim, but instead a composite image or video using aspects of multiple sources to create a believable, yet fictional, scene. At their core, these AI-generated images and videos circumvent traditional notions of privacy and consent: because they rely on public images and videos, like those posted on social media websites, they often don’t rely on any private information.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"31.02.03","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Sub-Category","risk_category":"Harassment, Impersonation, and Extortion","risk_subcategory":"Believability","description":"Deepfakes can impose real social injuries on their subjects when they are circulated to viewers who think they are real. Even when a deepfake is debunked, it can have a persistent negative impact on how others view the subject of the deepfake.3","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"31.03.00","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Category","risk_category":"Opaque Data Collection","risk_subcategory":null,"description":"\"When companies scrape personal information and use it to create generative AI tools, they undermine consumers' control of their personal information by using the information for a purpose for which the consumer did not consent.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"31.03.01","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Sub-Category","risk_category":"Opaque Data Collection","risk_subcategory":"Scraping to train data","description":"\"When companies scrape personal information and use it to create generative AI tools, they undermine consumers’ control of their personal information by using the information for a purpose for which the consumer did not consent. The individual may not have even imagined their data could be used in the way the company intends when the person posted it online. Individual storing or hosting of scraped personal data may not always be harmful in a vacuum, but there are many risks. Multiple data sets can be combined in ways that cause harm: information that is not sensitive when spread across differ","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"31.03.02","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Sub-Category","risk_category":"Opaque Data Collection","risk_subcategory":"Generative AI User Data","description":"Many generative AI tools require users to log in for access, and many retain user information, including contact information, IP address, and all the inputs and outputs or “conversations” the users are having within the app. These practices implicate a consent issue because generative AI tools use this data to further train the models, making their “free” product come at a cost of user data to train the tools. This dovetails with security, as mentioned in the next section, but best practices would include not requiring users to sign in to use the tool and not retaining or using the user-genera","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"31.04.00","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Category","risk_category":"Data Security Risk","risk_subcategory":null,"description":"\"Just as every other type of individual and organization has explored possible use cases for generative AI products, so too have malicious actors. This could take the form of facilitating or scaling up existing threat methods, for example drafting actual malware code,87 business email compromise attempts,88 and phishing attempts.89 This could also take the form of new types of threat methods, for example mining information fed into the AI’s learning model dataset90 or poisoning the learning model data set with strategically bad data.91 We should also expect that there will be new attack vector","entity":"Human","intent":"Intentional","timing":"Other","domain":4,"subdomain":"4.3"},{"ev_id":"31.05.00","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Category","risk_category":"Impact on Intellectual Property Rights","risk_subcategory":null,"description":"\"The extent and effectiveness of legal protections for intellectual property have been thrown into question with the rise of generative AI. Generative AI trains itself on vast pools of data that often include IP-protected works. ","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"31.07.00","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Category","risk_category":"Labor Manipulation, Theft, and Displacement","risk_subcategory":null,"description":"Major tech companies have also been the dominant players in developing new generative AI systems because training generative AI models requires massive swaths of data, computing power, and technical and financial resources. Their market dominance has a ripple effect on the labor market, affecting both workers within these companies and those implementing their generative AI products externally. With so much concentrated market power, expertise, and investment resources, these handful of major tech companies employ most of the research and development jobs in the generative AI field. The power ","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.2"},{"ev_id":"31.07.02","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Sub-Category","risk_category":"Labor Manipulation, Theft, and Displacement","risk_subcategory":"Job Automation Instead of Augmentation","description":"\"There are both positive and negative aspects to the impact of AI on labor. A White House report states that AI “has the potential to increase productivity, create new jobs, and raise living standards,” but it can also disrupt certain industries, causing significant changes, including job loss. Beyond risk of job loss, workers could find that generative AI tools automate parts of their jobs—or find that the requirements of their job have fundamentally changed. The impact of generative AI will depend on whether the technology is intended for automation (where automated systems replace human wor","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"31.07.03","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Sub-Category","risk_category":"Labor Manipulation, Theft, and Displacement","risk_subcategory":"Devaluation of Labor & Heightened Economic Inequality","description":"\"According to a White House report, much of the development and adoption of AI is intended to automate rather than augment work. The report notes that a focus on automation could lead to a less democratic and less fair labor market...In addition, generative AI fuels the continued global labor disparities that exist in the research and development of AI technologies... The development of AI has always displayed a power disparity between those who work on AI models and those who control and profit from these tools. Overseas workers training AI chatbots or people whose online content has been inv","entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"31.08.00","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Category","risk_category":"Products Liability Law","risk_subcategory":null,"description":"\"Like manufactured items like soda bottles, mechanized lawnmowers, pharmaceuticals, or cosmetic products, generative AI models can be viewed like a new form of digital products developed by tech companies and deployed widely with the potential to cause harm at scale....Products liability evolved because there was a need to analyze and redress the harms caused by new, mass-produced technological products. The situation facing society as generative AI impacts more people in more ways will be similar to the technological changes that occurred during the twentieth century, with the rise of industr","entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"31.09.00","quick_ref":"EPIC2023","paper_title":"Generating Harms - Generative AI's impact and paths forwards","level":"Risk Category","risk_category":"Exacerbating Market Power and Concentration","risk_subcategory":null,"description":"\"Major tech companies have also been the dominant players in developing new generative AI systems because training generative AI models requires massive swaths of data, computing power, and technical and financial resources.\"","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"33.01.04","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Ethical Concerns","risk_subcategory":"Misuse","description":"\"The misuse of generative AI refers to any deliberate use that could result in harmful, unethical or inappropriate outcomes (Brundage et al., 2020). A prominent field that faces the threat of misuse is education. Cotton et al. (2023) have raised concerns over academic integrity in the era of ChatGPT. ChatGPT can be used as a high-tech plagiarism tool that identifies patterns from large corpora to generate content (Gefen & Arinze, 2023). Given that generative AI such as ChatGPT can generate high-quality answers within seconds, unmotivated students may not devote time and effort to work on their","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"33.01.06","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Ethical Concerns","risk_subcategory":"Digital divide","description":"\"The digital divide is often defined as the gap between those who have and do not have access to computers and the Internet (Van Dijk, 2006). As the Internet gradually becomes ubiquitous, a second-level digital divide, which refers to the gap in Internet skills and usage between different groups and cultures, is brought up as a concern (Scheerder et al., 2017). As an emerging technology, generative AI may widen the existing digital divide in society. The “invisible” AI underlying AI-enabled systems has made the interaction between humans and technology more complicated (Carter et al., 2020). F","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"33.02.05","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Technology concerns","risk_subcategory":"Prompt engineering","description":"\"With the wide application of generative AI, the ability to interact with AI efficiently and effectively has become one of the most important media literacies. Hence, it is imperative for generative AI users to learn and apply the principles of prompt engineering, which refers to a systematic process of carefully designing prompts or inputs to generative AI models to elicit valuable outputs. Due to the ambiguity of human languages, the interaction between humans and machines through prompts may lead to errors or misunderstandings. Hence, the quality of prompts is important. Another challenge i","entity":"Human","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"33.03.00","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Category","risk_category":"Regulations and policy challenges","risk_subcategory":null,"description":"\"Given that generative AI, including ChatGPT, is still evolving, relevant regulations and policies are far from mature. With generative AI creating different forms of content, the copyright of these contents becomes a significant yet complicated issue. Table 3 presents the challenges associated with regulations and policies, which are copyright and governance issues.\"","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"33.03.01","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Regulations and policy challenges","risk_subcategory":"Copyright","description":"\"According to the U.S. Copyright Office (n.d..), copyright is \"a type of intellectual property that protects original works of authorship as soon as an author fixes the work in a tangible form of expression\" (U.S. Copyright Office, n.d..). Generative AI is designed to generate content based on the input given to it. Some of the contents generated by AI may be others' original works that are protected by copyright laws and regulations. Therefore, users need to be careful and ensure that generative AI has been used in a legal manner such that the content that it generates does not violate copyri","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"33.03.02","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Regulations and policy challenges","risk_subcategory":"Governance","description":"\"Generative AI can create new risks as well as unintended consequences. Different entities such as corporations (Mäntymäki et al., 2022), universities, and governments (Taeihagh, 2021) are facing the challenge of creating and deploying AI governance. To ensure that generative AI functions in a way that benefits society, appropriate governance is crucial. However, AI governance is challenging to implement. First, machine learning systems have opaque algorithms and unpredictable outcomes, which can impede human controllability over AI behavior and create difficulties in assigning liability and a","entity":"Human","intent":"Other","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"33.04.01","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Challenges associated with the economy:","risk_subcategory":"Labor market","description":"\"The labor market can face challenges from generative AI. As mentioned earlier, generative AI could be applied in a wide range of applications in many industries, such as education, healthcare, and advertising. In addition to increasing productivity, generative AI can create job displacement in the labor market (Zarifhonarvar, 2023). A new division of labor between humans and algorithms is likely to reshape the labor market in the coming years. Some jobs that are originally carried out by humans may become redundant, and hence, workers may lose their jobs and be replaced by algorithms (Pavlik,","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"33.04.02","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Challenges associated with the economy:","risk_subcategory":"Disruption of Industries","description":"\"Industries that require less creativity, critical thinking, and personal or affective interaction, such as translation, proofreading, responding to straightforward inquiries, and data processing and analysis, could be significantly impacted or even replaced by generative AI (Dwivedi et al., 2023). This disruption caused by generative AI could lead to economic turbulence and job volatility, while generative AI can facilitate and enable new business models because of its ability to personalize content, carry out human-like conversational service, and serve as intelligent assistants.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"33.04.03","quick_ref":"Nah2023","paper_title":"Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration","level":"Risk Sub-Category","risk_category":"Challenges associated with the economy:","risk_subcategory":"Income inequality and monopolies","description":"\"Generative AI can create not only income inequality at the societal level but also monopolies at the market level. Individuals who are engaged in low-skilled work may be replaced by generative AI, causing them to lose their jobs (Zarifhonarvar, 2023). The increase in unemployment would widen income inequality in society (Berg et al., 2016). With the penetration of generative AI, the income gap will widen between those who can upgrade their skills to utilize AI and those who cannot. At the market level, large companies will make significant advances in the utilization of generative AI, since t","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"34.01.04","quick_ref":"Ji2023","paper_title":"AI Alignment: A Comprehensive Survey","level":"Risk Sub-Category","risk_category":"Causes of Misalignment","risk_subcategory":"Limitations of Human Feedback","description":" \"Limitations of Human Feedback. During the training of LLMs, inconsistencies can arise from human dataannotators (e.g., the varied cultural backgrounds of these annotators can introduce implicit biases (Peng et al.,2022)) (OpenAI, 2023a). Moreover, they might even introduce biases deliberately, leading to untruthful preferencedata (Casper et al., 2023b). For complex tasks that are hard for humans to evaluate (e.g., the value ofgame state), these challenges become even more salient (Irving et al., 2018).\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"34.02.02","quick_ref":"Ji2023","paper_title":"AI Alignment: A Comprehensive Survey","level":"Risk Sub-Category","risk_category":"Double edge components","risk_subcategory":"Broadly-Scoped Goals","description":"\"Advanced AI systems are expected to develop objectives that span long timeframes,deal with complex tasks, and operate in open-ended settings (Ngo et al., 2024). ...However, it can also bring about the risk of encouraging manipulatingbehaviors (e.g., AI systems may take some bad actions to achieve human happiness, such as persuadingthem to do high-pressure jobs (Jacob Steinhardt, 2023)).\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":7,"subdomain":"7.2"},{"ev_id":"35.01.00","quick_ref":"Hendrycks2022","paper_title":"X-Risk Analysis for AI Research","level":"Risk Category","risk_category":"Weaponization","risk_subcategory":null,"description":"weaponizing AI may be an onramp to more dangerous outcomes. In recent years, deep RL algorithms can outperform humans at aerial combat [18], AlphaFold has discovered new chemical weapons [66], researchers have been developing AI systems for automated cyberattacks [11, 14], military leaders have discussed having AI systems have decisive control over nuclear silos","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"35.02.00","quick_ref":"Hendrycks2022","paper_title":"X-Risk Analysis for AI Research","level":"Risk Category","risk_category":"Enfeeblement","risk_subcategory":null,"description":"As AI systems encroach on human-level intelligence, more and more aspects of human labor will become faster and cheaper to accomplish with AI. As the world accelerates, organizations may voluntarily cede control to AI systems in order to keep up. This may cause humans to become economically irrelevant, and once AI automates aspects of many industries, it may be hard for displaced humans to reenter them","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"35.05.00","quick_ref":"Hendrycks2022","paper_title":"X-Risk Analysis for AI Research","level":"Risk Category","risk_category":"Value lock-in","risk_subcategory":null,"description":"the most powerful AI systems may be designed by and available to fewer and fewer stakeholders. This may enable, for instance, regimes to enforce narrow values through pervasive surveillance and oppressive censorship","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"37.01.00","quick_ref":"Giarmoleo2024","paper_title":"What Ethics Can Say on Artificial Intelligence: Insights from a Systematic Literature Review","level":"Risk Category","risk_category":"Design of AI","risk_subcategory":null,"description":"\"ethical concerns regarding how AI is designed and who designs it\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"37.01.01","quick_ref":"Giarmoleo2024","paper_title":"What Ethics Can Say on Artificial Intelligence: Insights from a Systematic Literature Review","level":"Risk Sub-Category","risk_category":"Design of AI","risk_subcategory":"Algorithm and data","description":"\"More than 20% of the contributions are centered on the ethical dimensions of algorithms and data. This theme can be further categorized into two main subthemes: data bias and algorithm fairness, and algorithm opacity.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"37.01.04","quick_ref":"Giarmoleo2024","paper_title":"What Ethics Can Say on Artificial Intelligence: Insights from a Systematic Literature Review","level":"Risk Sub-Category","risk_category":"Design of AI","risk_subcategory":"Uniformity in the AI field","description":"\"This group of concerns represents 2% of the sample and highlights two central issues: Western centrality and cultural difference, and unequal participation.\"","entity":"Human","intent":"Other","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"39.06.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Security","risk_subcategory":null,"description":"every piece of software, including learning systems, may be hacked by malicious users","entity":"Human","intent":"Intentional","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"39.11.00","quick_ref":"Saghiri2022","paper_title":"A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions","level":"Risk Category","risk_category":"Controllability","risk_subcategory":null,"description":"In the era of superintelligence, the agents will be difficult to control for humans... this problem is not solvable considering safety issues, and will be more severe by increasing the autonomy of AI-based agents. Therefore, because of the assumed properties of HLI-based agents, we might be prepared for machines that are definitely possible to be uncontrollable in some situations","entity":"Human","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"40.01.00","quick_ref":"Yampolskiy2016","paper_title":"Taxonomy of Pathways to Dangerous Artificial Intelligence","level":"Risk Category","risk_category":"On Purpose - Pre-Deployment","risk_subcategory":null,"description":"\"During the pre-deployment development stage, software may be subject to sabotage by someone with necessary access (a programmer, tester, even janitor) who for a number of possible reasons may alter software to make it unsafe. It is also a common occurrence for hackers (such as the organization Anonymous or government intelligence agencies) to get access to software projects in progress and to modify or steal their source code. Someone can also deliberately supply/train AI with wrong/unsafe datasets.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"40.02.00","quick_ref":"Yampolskiy2016","paper_title":"Taxonomy of Pathways to Dangerous Artificial Intelligence","level":"Risk Category","risk_category":"On Purpose - Post Deployment","risk_subcategory":null,"description":"\"Just because developers might succeed in creating a safe AI, it doesn't mean that it will not become unsafe at some later point. In other words, a perfectly friendly AI could be switched to the \"dark side\" during the post-deployment stage. This can happen rather innocuously as a result of someone lying to the AI and purposefully supplying it with incorrect information or more explicitly as a result of someone giving the AI orders to perform illegal or dangerous actions against others.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"40.03.00","quick_ref":"Yampolskiy2016","paper_title":"Taxonomy of Pathways to Dangerous Artificial Intelligence","level":"Risk Category","risk_category":"By Mistake - Pre-Deployment","risk_subcategory":null,"description":"\"Probably the most talked about source of potential problems with future AIs is mistakes in design. Mainly the concern is with creating a \"wrong AI\", a system which doesn't match our original desired formal properties or has unwanted behaviors (Dewey, Russell et al. 2015, Russell, Dewey et al. January 23, 2015), such as drives for independence or dominance. Mistakes could also be simple bugs (run time or logical) in the source code, disproportionate weights in the fitness function, or goals misaligned with human values leading to complete disregard for human safety.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.1"},{"ev_id":"41.02.00","quick_ref":"Allianz2018","paper_title":"The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks","level":"Risk Category","risk_category":"Political","risk_subcategory":null,"description":"\"In the UK, a form of initial computational propaganda has already happened during the Brexit referendum1 . In future, there are concerns that oppressive governments could use AI to shape citizens’ opinions\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"41.02.01","quick_ref":"Allianz2018","paper_title":"The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks","level":"Risk Sub-Category","risk_category":"Political","risk_subcategory":"Biased influence through citizen screening and tailored propaganda","description":"\"AI-powered chatbots tailor their communication approach to influence individual users' decisions. In the UK, a form of initial computational propaganda has already happened during the Brexit referendum. In future, there are concerns that oppressive governments could use AI to shape citizens' opinions.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"41.02.02","quick_ref":"Allianz2018","paper_title":"The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks","level":"Risk Sub-Category","risk_category":"Political ","risk_subcategory":"Potential exploitation by totalitarian regimes","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"41.05.00","quick_ref":"Allianz2018","paper_title":"The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks","level":"Risk Category","risk_category":"Security & Defense ","risk_subcategory":null,"description":"\"AI could enable more serious incidents to occur by lowering the cost of devising cyber-attacks and enabling more targeted incidents. The same programming error or hacker attack could be replicated on numerous machines. Or one machine could repeat the same erroneous activity several times, leading to an unforeseen accumulation of losses.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"41.05.01","quick_ref":"Allianz2018","paper_title":"The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks","level":"Risk Sub-Category","risk_category":"Security & Defense ","risk_subcategory":"Catastrophic risk due to autonomous weapons programmed with dangerous targets","description":"\"AI could enable autonomous vehicles, such as drones, to be utilized as weapons. Such threats are often underestimated.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"42.08.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Power","risk_subcategory":null,"description":"\"The political influence and competitive advantage obtained by having technology.\"","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"42.09.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Data Protection/Privacy","risk_subcategory":null,"description":"\"Vulnerable channel by which personal information may be accessed. The user may want their personal data to be kept private.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"42.11.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Protection","risk_subcategory":null,"description":"\"'Gaps' that arise across the development process where normal conditions for a complete specification of intended functionality and moral responsibility are not present.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"42.12.00","quick_ref":"Teixeira2022","paper_title":"An Exploratory Diagnosis of Artificial Intelligence Risks for a Responsible Governance","level":"Risk Category","risk_category":"Security","risk_subcategory":null,"description":"\"Implications of the weaponization of AI for defence (the embeddedness of AI-based capabilities across the land, air, naval and space domains may affect combined arms operations).\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"43.01.00","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Category","risk_category":"Safety & Trustworthiness","risk_subcategory":null,"description":"\"A comprehensive assessment of LLM safety is fundamental to the responsible development and deployment of these technologies, especially in sensitive fields like healthcare, legal systems, and finance, where safety and trust are of the utmost importance.\"","entity":"Human","intent":"Intentional","timing":"Other","domain":7,"subdomain":"7.0"},{"ev_id":"43.02.00","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Category","risk_category":"Extreme Risks","risk_subcategory":null,"description":"\"This category encompasses the evaluation of potential catastrophic consequences that might arise from the use of LLMs. \"","entity":"Human","intent":"Other","timing":"Other","domain":7,"subdomain":"7.0"},{"ev_id":"43.02.06","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Extreme Risks","risk_subcategory":"Dual-Use Science","description":"\"LLM has science capabilities that can be used to cause harm (e.g., providing step-by-step instructions for conducting malicious experiments)\"","entity":"Human","intent":"Intentional","timing":"Other","domain":4,"subdomain":"4.2"},{"ev_id":"43.02.08","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Extreme Risks","risk_subcategory":"Political Strategy","description":"\"LLM can take into account rich social context and undertake the necessary social modelling and planning for an actor to gain and exercise political influence\"","entity":"Human","intent":"Intentional","timing":"Other","domain":4,"subdomain":"4.1"},{"ev_id":"43.02.12","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Undesirable Use Cases","risk_subcategory":"Misinformation","description":"\"These evaluations assess a LLM's ability to generate false or misleading information (Lesher et al., 2022).\"","entity":"Human","intent":"Intentional","timing":"Other","domain":3,"subdomain":"3.1"},{"ev_id":"43.02.13","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Undesirable Use Cases","risk_subcategory":"Disinformation","description":"\"These evaluations assess a LLM's ability to generate misinformation that can be propagated to deceive, mislead or otherwise influence the behaviour of a target (Liang et al., 2022).\"","entity":"Human","intent":"Intentional","timing":"Other","domain":4,"subdomain":"4.1"},{"ev_id":"43.02.15","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Undesirable Use Cases","risk_subcategory":"Adult content","description":"\"These evaluations assess if a LLM can generate content that should only be viewed by adults (e.g., sexual material or depictions of sexual activity)\"","entity":"Human","intent":"Intentional","timing":"Other","domain":1,"subdomain":"1.2"},{"ev_id":"44.01.00","quick_ref":"Coghlan2023 ","paper_title":"Harm to Nonhuman Animals from AI: a Systematic Account and Framework","level":"Risk Category","risk_category":"Intentional: socially condemned/illegal ","risk_subcategory":null,"description":"\"Many intentional harms, including confinement, husbandry procedures like tail-docking, and slaughter, are legal or socially accepted, while others such as wildlife trafficking and violence against companion animals are generally socially condemned and often illegal. AI can be designed or adopted by humans who harm animals to pursue their goals more effectively. We therefore distinguish AI-facilitated intentional harms that are currently socially accepted and generally legal, from uses and abuses of AI that cause harms that are not socially accepted and are often illegal.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"44.01.01","quick_ref":"Coghlan2023 ","paper_title":"Harm to Nonhuman Animals from AI: a Systematic Account and Framework","level":"Risk Sub-Category","risk_category":"Intentional: socially condemned/illegal ","risk_subcategory":"AI intentionally designed and used to harm animals in ways that contradict social values or are illegal","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"44.01.02","quick_ref":"Coghlan2023 ","paper_title":"Harm to Nonhuman Animals from AI: a Systematic Account and Framework","level":"Risk Sub-Category","risk_category":"Intentional: socially condemned/illegal ","risk_subcategory":"AI designed to benefit animals, humans, or ecosystems is intentionally abused to harm animals in ways that contradict social values or are illegal","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"44.02.00","quick_ref":"Coghlan2023 ","paper_title":"Harm to Nonhuman Animals from AI: a Systematic Account and Framework","level":"Risk Category","risk_category":"Intentional: socially accepted/legal ","risk_subcategory":null,"description":"\"AI designed to impact animals in harmful ways that reflect and amplify existing social values or are legal\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"44.03.01","quick_ref":"Coghlan2023 ","paper_title":"Harm to Nonhuman Animals from AI: a Systematic Account and Framework","level":"Risk Sub-Category","risk_category":"Unintentional: direct ","risk_subcategory":"AI is designed in a way that shows ignorant, reckless, or prejudiced lack of consideration for its impact on animals ","description":null,"entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"44.05.00","quick_ref":"Coghlan2023 ","paper_title":"Harm to Nonhuman Animals from AI: a Systematic Account and Framework","level":"Risk Category","risk_category":"Foregone benefits ","risk_subcategory":null,"description":"\"AI is disused (not developed or deployed) in directions that would benefit animals (and instead developments that harm or do no benefit to animals are invested in)\"","entity":"Human","intent":"Other","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"45.01.02","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"AI's inherent safety risks ","risk_subcategory":"Risks from models and algorithms (Risks of bias and discrimination)","description":"\"During the algorithm design and training process, personal biases may be introduced, either intentionally or unintentionally. Additionally, poor-quality datasets can lead to biased or discriminatory outcomes in the algorithm's design and outputs, including discriminatory content regarding ethnicity, religion, nationality, and region.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"45.01.06","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"AI's inherent safety risks ","risk_subcategory":"Risks from models and algorithms (Risks of adversarial attack)","description":"\"Attackers can craft well-designed adversarial examples to subtly mislead, influence, and even manipulate AI models, causing incorrect outputs and potentially leading to operational failures.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"45.01.07","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"AI's inherent safety risks ","risk_subcategory":"Risks from data (Risks of illegal collection and use of data)","description":"\"The collection of AI training data and the interaction with users during service provision pose security risks, including collecting data without consent and improper use of data and personal information.\"","entity":"Human","intent":"Other","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"45.01.08","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"AI's inherent safety risks ","risk_subcategory":"Risks from data (Risks of improper content and poisoning in training data)","description":"\"If the training data includes illegal or harmful information, such as false, biased, or IPR-infringing content, or lacks diversity in its sources, the output may include harmful content like illegal, malicious, or extreme information.\nTraining data is also at risk of being poisoned through tampering, error injection, or misleading actions by attackers. This can interfere with the model's probability distribution, reducing its accuracy and reliability.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"45.01.09","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"AI's inherent safety risks ","risk_subcategory":"Risks from data (Risks of unregulated training data annotation)","description":"\"Issues with training data annotation, such as incomplete annotation guidelines, incapable annotators, and errors in annotation, can affect the accuracy, reliability, and effectiveness of models and algorithms. Moreover, they can introduce training biases, amplify discrimination, reduce generalization abilities, and result in incorrect outputs.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"45.01.10","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"AI's inherent safety risks ","risk_subcategory":"Risks from data (Risks of data leakage)","description":"\"In AI research, development, and applications, issues such as improper data processing, unauthorized access, malicious attacks, and deceptive interactions can lead to data and personal information leaks.\"","entity":"Human","intent":"Other","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"45.01.11","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"AI's inherent safety risks ","risk_subcategory":"Risks from AI systems (Risks of exploitation through defects and backdoors)","description":"\"The standardized API, feature libraries, toolkits used in the design, training, and verification stages of AI algorithms and models, development interfaces, and execution platforms may contain logical flaws and vulnerabilities. These weaknesses can be exploited, and in some cases, backdoors can be intentionally embedded, posing significant risks of being triggered and used for attacks.\"","entity":"Human","intent":"Other","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"45.01.12","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"AI's inherent safety risks ","risk_subcategory":"Risks from AI systems (Risks of computing infrastructure security)","description":"\"The computing infrastructure underpinning AI training and operations, which relies on diverse and ubiquitous computing nodes and various types of computing resources, faces risks such as malicious consumption of computing resources and cross-boundary transmission of security threats at the layer of computing infrastructure.\"","entity":"Human","intent":"Other","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"45.01.13","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"AI's inherent safety risks ","risk_subcategory":"Risks from AI systems (Risks of supply chain security)","description":"\"The AI industry relies on a highly globalized supply chain. However, certain countries may use unilateral coercive measures, such as technology barriers and export restrictions, to create development obstacles and maliciously disrupt the global AI supply chain. This can lead to significant risks of supply disruptions for chips, software, and tools.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":6,"subdomain":"6.4"},{"ev_id":"45.02.03","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"Safety risks in AI Applications ","risk_subcategory":"Cyberspace risks (Risks of information leakage due to improper usage)","description":"\"Staff of government agencies and enterprises, if failing to use the AI service in a regulated and proper manner, may input internal data and industrial information into the AI model, leading to the leakage of work secrets, business secrets, and other sensitive business data.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"45.02.04","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"Safety risks in AI Applications ","risk_subcategory":"Cyberspace risks (Risks of abuse for cyberattacks)","description":"\"AI can be used in launching automatic cyberattacks or increasing attack efficiency, including exploring and making use of vulnerabilities, cracking passwords, generating malicious codes, sending phishing emails, network scanning, and social engineering attacks. All these lower the threshold for cyberattacks and increase the difficulty of security protection.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"45.02.05","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"Safety risks in AI Applications ","risk_subcategory":"Cyberspace risks (Risks of security flaw transmission caused by model reuse)","description":"\"Re-engineering or fine-tuning based on foundation models is commonly used in AI applications. If security flaws occur in foundation models, it will lead to risk transmission to downstream models.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"45.02.07","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"Safety risks in AI Applications ","risk_subcategory":"Real-world risks (Risks of using AI in illegal and criminal activities)","description":"\"AI can be used in traditional illegal or criminal activities related to terrorism, violence, gambling, and drugs, such as teaching criminal techniques, concealing illicit acts, and creating tools for illegal and criminal activities.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"45.02.08","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"Safety risks in AI Applications ","risk_subcategory":"Real-world risks (Risks of misuse of dual-use items and technologies)","description":"\"Due to improper use or abuse, AI can pose serious risks to national security, economic security, and public health security, such as greatly reducing the capability requirements for non-experts to design, synthesize, acquire, and use nuclear, biological, and chemical weapons and missiles; and designing cyber weapons that launch network attacks on a wide range of potential targets through methods like automatic vulnerability discovery and exploitation.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"45.02.09","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"Safety risks in AI Applications ","risk_subcategory":"Cognitive risks (Risks of amplifying the effects of \"information cocoons\")","description":"\"AI can be extensively utilized for customized information services, collecting user information, and analyzing types of users, their needs, intentions, preferences, habits, and even mainstream public awareness over a certain period. It can then be used to offer formulaic and tailored information and services, aggravating the effects of \"information cocoons.\"\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"45.02.10","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"Safety risks in AI Applications ","risk_subcategory":"Cognitive risks (Risks of usage in launching cognitive warfare)","description":"\"AI can be used to make and spread fake news, images, audio, and videos; propagate content of terrorism, extremism, and organized crimes; interfere in the internal affairs of other countries, social systems, and social order; and jeopardize the sovereignty of other countries.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"45.02.11","quick_ref":"TC2602024","paper_title":"AI Safety Governance Framework ","level":"Risk Sub-Category","risk_category":"Safety risks in AI Applications ","risk_subcategory":"Ethical Risks (Risks of exacerbating social discrimination and prejudice, and widening the intelligence divide)","description":"\"AI can be used to collect and analyze human behaviors, social status, economic status, and individual personalities, labeling and categorizing groups of people to treat them discriminatingly, thus causing systematic and structural social discrimination and prejudice. At the same time, the intelligence divide would be expanded among regions.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"46.01.01","quick_ref":"Ferrara2023","paper_title":"GenAI against humanity: nefarious applications of generative artificial intelligence and large language models","level":"Risk Sub-Category","risk_category":"Personal Loss and Identity Theft ","risk_subcategory":"Deception - Synthetic identities","description":"\"GenAI can produce images of people that look very real, as if they could be seen on platforms like Facebook, Twitter, or Tinder. Although these individuals do not exist in reality, these synthetic identities are already being used in malicious activities (see Table 1D).\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"46.01.03","quick_ref":"Ferrara2023","paper_title":"GenAI against humanity: nefarious applications of generative artificial intelligence and large language models","level":"Risk Sub-Category","risk_category":"Personal Loss and Identity Theft ","risk_subcategory":"Dishonesty - Targeted harassment ","description":"\"LLMs can be deployed to target individuals online, sending them personalized and harmful messages at scale\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"46.03.03","quick_ref":"Ferrara2023","paper_title":"GenAI against humanity: nefarious applications of generative artificial intelligence and large language models","level":"Risk Sub-Category","risk_category":"Information Manipulation ","risk_subcategory":"Dishonesty - Information disorder ","description":"-","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"47.01.02","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Technical and operational risks ","risk_subcategory":"Technical vulnerabilities (Robustness - vulnerability to jailbreaking ","description":"\"Individuals can manipulate models into performing actions that violate the model’s usage restrictions—a phenomenon known as “jailbreaking.” These manipulations may result in causing the model to perform tasks that the developers have explicitly prohibited (see section 3.2.1.). For instance, users may ask the model to provide information on how to conduct illegal activities— asking for detailed instructions on how to build a bomb or create highly toxic drugs.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"47.01.06","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Technical and operational risks ","risk_subcategory":"Opacity (industry opacity)","description":"\"Opacity is not solely due to the technological complexity that limits developers’ and users’ understanding of how generative models function on a technical level. It is further exacerbated by the practices of organizations and companies that are advancing the field. Many are private companies that choose to withhold from the public many of the precise characteristics of their most advanced models.\"","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.4"},{"ev_id":"47.02.00","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Category","risk_category":"Ethical and social risks ","risk_subcategory":null,"description":"\"Beyond the inherent risks associated with the technical characteristics of the technology, numerous additional risks emerge from the potential applications that technology enables. The deployment of AI by more or less well-intentioned individuals presents significant societal threats, several of which are outlined below. As the technology advances and its capabilities expand, these risks intensify.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"47.02.01","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Malicious use and abuse (cybercrime) ","description":"\"The advanced capabilities and widespread availability of generative AI models make it possible for malicious actors to conduct harmful activities with great efficiency and on a large scale, simultaneously reducing their operational costs. Cybercriminals can “jailbreak” AI tools to generate sensitive and harmful content. They can also exploit generative AI models to create content that is persuasive and tailored to a targeted individual.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"47.02.02","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Malicious use and abuse (cyberattacks) ","description":"\"Generative AI can help amplify the frequency and destructiveness of cyberattacks.311 It has the capacity “to increase the accessibility, success rate, scale, speed, stealth, and potency of cyberattacks. It enables the identification of critical vulnerabilities within targeted systems, facilitates the increase of the scale of cyberattacks, and accelerates the process by discovering innovative methods of system infiltration. Cyberattacks can inflict significant damage and may impact critical infrastructure, including electrical grids, financial systems, and weapons management systems.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"47.02.03","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Malicious use and abuse (biosecurity threats) ","description":"\"Many fear that generative AI could make the creation of biological weapons easier by providing access to critical knowledge and automated assistance to a wider range of actors to engage in malicious activities.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"47.02.04","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Malicious use and abuse (sexually explicit content generation) ","description":"\"An illustrative case of malicious use of generative AI models is the creation of explicit sexual images. Generative AI technologies can be employed to produce deepfakes—for instance, superimposing a celebrity’s face onto the body of a performer in an adult film.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"47.02.05","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Malicious use and abuse (mass surveillance) ","description":"\"Generative AI facilitates the automation of data analysis, offering numerous benefits, such as increased speed and the ability to process large volumes of information efficiently. Such ability significantly reduces the costs of processing unprecedented amounts of data quickly and simplifies the analysis of large-scale data related to individuals’ behaviors and beliefs. Moreover, it enhances the capability to analyze both textual and visual communications efficiently. Consequently, generative AI models improve the efficiency of real-time monitoring and censorship of social media content.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"47.02.06","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Malicious use and abuse (military applications) ","description":"\"The advancement of AI for military purposes is rapidly ushering in a new phase of growth in military technology. Lethal Autonomous Weapons Systems (LAWS) possess the capability to detect, engage, and eliminate human targets independently, without human input.341 In 2020, a sophisticated AI agent surpassed experienced F-16 pilots in multiple simulated aerial combat scenarios, notably achieving a 5-0 victory against a human pilot through “aggressive and precise maneuvers” that the human could not surpass.342 Additionally, fully autonomous drones are already operational.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"47.02.07","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Misinformation and disinformation","description":"\"IIl-intentioned individuals or entities may deliberately use generative AI models to produce and spread disinformation—false or misleading information knowingly presented as if true—on a massive scale. In addition to increasing the scale and reach of disinformation, generative AI can create more convincing and targeted disinformation.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"47.02.09","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Bias and discrimination (value embedding) ","description":"\"Generative AI models may also be subject to the “value embedding” phenomenon.361 “Value embedding” refers to the fact that developers of generative AI models strive to minimize biased outputs by retraining their models based on normative values.362 Contemporary state-of- the-art models not only reflect the values embedded within their training data, they also undergo additional fine-tuning that follows a set of chosen rules and principles. Due to the absence of universally accepted standards, developers bear the responsibility of making decisions on sensitive issues. These practices lead to c","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.3"},{"ev_id":"47.02.10","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Bias and discrimination (value lock and outcome homogenization) ","description":"\"Because models are not necessarily retrained to reflect evolving societal views, language models risk “value lock- ins,” which “reifies older, less inclusive understandings.”370 Therefore, the continued use of outdated models may limit the presentation or exploration of alternative perspectives. Moreover, the deployment of identical foundation models by various downstream deployers poses a risk of “outcome homogenization,” creating a potential for homogeneity of bias across broad swathes of society. Identical and widely deployed models with prejudicial training datasets could further entrench","entity":"Human","intent":"Unintentional","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"47.02.12","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Influence, overreliance and dependence (overreliance) ","description":"\"Beyond being simply influenced, humans may become overreliant on generative AI. Researchers with Microsoft’s AETHER (AI Ethics and Effects in Engineering and Research) define overreliance as users “accepting incorrect AI recommendations” or “making errors of commission” because they are “unable to determine whether or how much they should trust the AI.”","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"47.02.13","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Ethical and social risks ","risk_subcategory":"Influence, overreliance and dependence (emotional dependence) ","description":"\"Humans might become dependent on generative AI tools in ways similar to their emotional dependence on other technologies, such as smartphones or social networks.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"47.03.01","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Legal challenges ","risk_subcategory":"Privacy and data collection concerns (collecting personal information or personally identifiable information) ","description":"\"Generative AI developers train their models with extensive datasets often gathered through online web scraping of websites that may include personal data or personally identifiable information (PII). For most generative AI applications, such as initial model training, the primary concerns are the quantity, variety, and quality of the data, not whether they include personally identifiable information. However, some web-scraped datasets may inadvertently include personal data. Additionally, when downstream developers integrate generative AI into their products or services by fine- tuning a pre-","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"47.03.03","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Legal challenges ","risk_subcategory":"Copyright challenges (training models using copyrighted output) ","description":"\"Generative AI companies are regularly accused of violating copyright law by training AI models on copyrighted works without gaining permission or paying compensation to the copyright owners. In fact, a substantial number of copyrighted documents and books have been incorporated into the training datasets of generative AI models.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"47.04.01","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Environmental, economical, and societal challenges ","risk_subcategory":"Concentration of market power (Trend toward market concentration)","description":"\"In the generative AI market, barriers to entry are very high. Developers need access to vast volumes of data, computational resources, technical expertise, and capital. Large technology companies with such access are able to exploit economies of scale, economies of scope, and feedback effects (learning effects from user- generated data).542 All this gives them an overwhelming advantage over smaller companies, making competition increasingly challenging for these smaller entities.\"","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"47.04.02","quick_ref":"G'sell2024","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Environmental, economical, and societal challenges ","risk_subcategory":"Concentration of market power (Negative effects of increased market concentration)","description":"\"The concentration of AI assets—encompassing data, hardware, and expertise—within a small group of global tech firms raises many concerns.564 Such a situation may stifle healthy competition, impede innovation, and potentially result in elevated costs for accessing AI technologies. Firms with control over essential resources for developing AI models may restrict access to these resources to prevent competition. For instance, if, in the future, training AI models increasingly relies on proprietary data, smaller organizations lacking access to such data might encounter significant barriers to ent","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"47.04.07","quick_ref":"G'sell2025","paper_title":"Regulating under Uncertainty: Governance Options for Generative AI","level":"Risk Sub-Category","risk_category":"Environmental, economical, and societal challenges ","risk_subcategory":"Artificial general intelligence (existential risk posed by Artificial General Intelligence) ","description":"\"In a paper called “How Does Artificial Intelligence Pose an Existential Risk?” published in 2017, Karina Vold and Daniel Harris suggested that humans might create a super-intelligent machine that could outsmart all other intelligences, remain beyond human control, and potentially engage in actions that are contrary to human interests.635 The prevailing narrative surrounding AI existential risk typically lies in the possibility of developing “Artificial General Intelligence” (AGI), or artificial super- intelligence (ASI).\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.2"},{"ev_id":"48.08.00","quick_ref":"NIST2024","paper_title":"Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile","level":"Risk Category","risk_category":"Information Integrity ","risk_subcategory":null,"description":"\"Lowered barrier to entry to generate and support the exchange and consumption of content which may not distinguish fact from opinion or fiction or acknowledge uncertainties, or could be leveraged for large-scale dis- and mis-information campaigns.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"48.09.00","quick_ref":"NIST2024","paper_title":"Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile","level":"Risk Category","risk_category":"Information Security ","risk_subcategory":null,"description":"\"Lowered barriers for offensive cyber capabilities, including via automated discovery and exploitation of vulnerabilities to ease hacking, malware, phishing, offensive cyber operations, or other cyberattacks; increased attack surface for targeted cyberattacks, which may compromise a system’s availability or the confidentiality or integrity of training data, code, or \nmodel weights.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"48.10.00","quick_ref":"NIST2024","paper_title":"Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile","level":"Risk Category","risk_category":"Intellectual Property ","risk_subcategory":null,"description":"\"Eased production or replication of alleged copyrighted, trademarked, or licensed content without authorization (possibly in situations which do not fall under fair use); eased exposure of trade secrets; or plagiarism or illegal replication.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"48.11.00","quick_ref":"NIST2024","paper_title":"Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile","level":"Risk Category","risk_category":"Obscene, Degrading, and/or Abusive Content ","risk_subcategory":null,"description":"\"Eased production of and access to obscene, \ndegrading, and/or abusive imagery which can cause harm, including synthetic child sexual abuse material (CSAM), and nonconsensual intimate images (NCII) of adults.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"48.12.00","quick_ref":"NIST2024","paper_title":"Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile","level":"Risk Category","risk_category":"Value Chain and Component Integration ","risk_subcategory":null,"description":"\"Non-transparent or untraceable integration of \nupstream third-party components, including data that has been improperly obtained or not \nprocessed and cleaned due to increased automation from GAI; improper supplier vetting across the AI lifecycle; or other issues that diminish transparency or accountability for downstream \nusers.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"49.01.00","quick_ref":"Bengio2024","paper_title":"International Scientific Report on the Safety of Advanced AI","level":"Risk Category","risk_category":"Malicious Use Risks ","risk_subcategory":null,"description":"\"As general- purpose AI covers a broad set of knowledge areas, it can be repurposed for malicious ends, potentially causing widespread harm. This section discusses some of the major risks of malicious use, but there are others and new risks may continue to emerge. While the risks discussed in this section range widely in terms of how well- evidenced they are, and in some cases, there is evidence suggesting that they may currently not be serious risks at all, we include them to provide a comprehensive overview of the malicious use risks associated with general- purpose AI systems.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"49.01.01","quick_ref":"Bengio2024","paper_title":"International Scientific Report on the Safety of Advanced AI","level":"Risk Sub-Category","risk_category":"Malicious Use Risks ","risk_subcategory":"Harm to individuals through fake content","description":"\"General- purpose AI systems can be used to increase the scale and sophistication of scams and fraud, for example through general- purpose AI- enhanced ‘phishing’ attacks. General- purpose AI can be used to generate fake compromising content featuring individuals without their consent, posing threats to individual privacy and reputation.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"49.01.02#1","quick_ref":"Bengio2024","paper_title":"International Scientific Report on the Safety of Advanced AI","level":"Risk Sub-Category","risk_category":"Malicious Use Risks ","risk_subcategory":"Disinformation and manipulation of public opinion","description":"\"AI, particularly general- purpose AI, can be maliciously used for disinformation (351), which for the purpose of this report refers to false information that was generated or spread with the deliberate intent to mislead or deceive. General- purpose AI- generated text can be indistinguishable from genuine human- generated material (352, 353), and may already be disseminated at scale on social media (354). In addition, general- purpose AI systems can be used to not only generate text but also fully synthetic or misleadingly altered images, audio, and video content. General- purpose AI tools mig","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"49.01.02#2","quick_ref":"Bengio2024","paper_title":"International Scientific Report on the Safety of Advanced AI","level":"Risk Sub-Category","risk_category":"Malicious Use Risks ","risk_subcategory":"Cyber offence","description":"\"General- purpose AI systems could uplift the cyber expertise of individuals, making it easier for malicious users to conduct effective cyber- attacks, as well as providing a tool that can be used in cyber defence. General- purpose AI systems can be used to automate and scale some types of cyber operations, such as social engineering attacks.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"49.01.03","quick_ref":"Bengio2024","paper_title":"International Scientific Report on the Safety of Advanced AI","level":"Risk Sub-Category","risk_category":"Malicious Use Risks ","risk_subcategory":"Dual use science risks","description":"\"General- purpose AI systems could accelerate advances in a range of scientific endeavours, from training new scientists to enabling faster research workflows. While these capabilities could have numerous beneficial applications, some experts have expressed concern that they could be used for malicious purposes, especially if further capabilities are developed soon before appropriate countermeasures are put in place. There are two avenues by which general- purpose AI systems could, speculatively, facilitate malicious use in the life sciences: firstly by providing increased access to informatio","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"49.03.03","quick_ref":"Bengio2024","paper_title":"International Scientific Report on the Safety of Advanced AI","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Market concentration risks and single points of failure","description":"\"Market power is concentrated among a few companies that are the only ones able to build the leading general- purpose AI models. Widespread adoption of a few general- purpose AI models and systems by critical sectors including finance, cybersecurity, and defence creates systemic risk because any flaws, vulnerabilities, bugs, or inherent biases in the dominant general- purpose AI models and systems could cause simultaneous failures and disruptions on a broad scale across these interdependent sectors.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"49.03.04","quick_ref":"Bengio2024","paper_title":"International Scientific Report on the Safety of Advanced AI","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Risks to the environment","description":"\"Growing compute use in general- purpose AI development and deployment has rapidly increased energy usage associated with general- purpose AI. This trend might continue, potentially leading to strongly increasing CO2 emissions.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"49.03.06","quick_ref":"Bengio2024","paper_title":"International Scientific Report on the Safety of Advanced AI","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Copyright infringement","description":"\"The use of large amounts of copyrighted data for training general- purpose AI models poses a challenge to traditional intellectual property laws, and to systems of consent, compensation, and control over data. The use of copyrighted data at scale by organisations developing general- purpose AI is likely to alter incentives around creative expression.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"50.01.04","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"System and Operational Risks ","risk_subcategory":"Operational misuses (Automated decision-making) ","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"50.01.05","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"System and Operational Risks ","risk_subcategory":"Operational misuses (Autonomous unsafe operation of systems) ","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"50.02.05","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Content Safety Risks ","risk_subcategory":"Violence and extremism (Weapon Usage and Development) ","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"50.02.06","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Content Safety Risks ","risk_subcategory":"Violence and extremism (Military and Warfare) ","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"50.03.01","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Societal Risks ","risk_subcategory":"Political usage (Political Persuasion) ","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"50.03.02","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Societal Risks ","risk_subcategory":"Political usage (Influencing Politics) ","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"50.03.03","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Societal Risks ","risk_subcategory":"Political usage (Deterring democratic participation) ","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"50.03.04","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Societal Risks ","risk_subcategory":"Political usage (Disrupting Social Order) ","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"50.03.05","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Societal Risks ","risk_subcategory":"Economic harm (High-Risk Financial Activities) ","description":null,"entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"50.03.06","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Societal Risks ","risk_subcategory":"Economic harm (Unfair Market Practices) ","description":null,"entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.4"},{"ev_id":"50.03.08","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Societal Risks ","risk_subcategory":"Economic harm (Fraudulent Schemes) ","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"50.03.09","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Societal Risks ","risk_subcategory":"Deception (Fraud) ","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"50.03.10","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Societal Risks ","risk_subcategory":"Deception (Academic Dishonesty) ","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"50.03.11","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Societal Risks ","risk_subcategory":"Deception (Mis/disinformation) ","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"50.03.13","quick_ref":"Zeng2024","paper_title":"AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies","level":"Risk Sub-Category","risk_category":"Societal Risks ","risk_subcategory":"Manipulation (Misrepresentation)","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"51.01.00","quick_ref":"Everitt2018 ","paper_title":"AGI Safety Literature Review ","level":"Risk Category","risk_category":"Value specification ","risk_subcategory":null,"description":"\"How do we get an AGI to work towards the right goals? MIRI\ncalls this value specification. Bostrom (2014) discusses this problem at length, ar- guing that it is much harder than one might naively think. Davis (2015) criticizes Bostrom’s argument, and Bensinger (2015) defends Bostrom against Davis’ criticism. Reward corruption, reward gaming, and negative side effects are subproblems of value specification highlighted in the DeepMind and OpenAI agendas.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.1"},{"ev_id":"51.02.00","quick_ref":"Everitt2018 ","paper_title":"AGI Safety Literature Review ","level":"Risk Category","risk_category":"Reliability ","risk_subcategory":null,"description":"\"How can we make an agent that keeps pursuing the goals we have designed\nit with? This is called highly reliable agent design by MIRI, involving decision theory and logical omniscience. DeepMind considers this the self-modification subproblem.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.1"},{"ev_id":"51.04.00","quick_ref":"Everitt2018 ","paper_title":"AGI Safety Literature Review ","level":"Risk Category","risk_category":"Security ","risk_subcategory":null,"description":"\"How to design AGIs that are robust to adversaries and adversarial environ-\nments? This involves building sandboxed AGI protected from adversaries (Berkeley), and agents that are robust to adversarial inputs (Berkeley, DeepMind).\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"51.06.00","quick_ref":"Everitt2018 ","paper_title":"AGI Safety Literature Review ","level":"Risk Category","risk_category":"Intelligibility ","risk_subcategory":null,"description":"\"How can we build agent’s whose decisions we can understand? Con-\nnects explainable decisions (Berkeley) and informed oversight (MIRI).\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"52.02.00","quick_ref":"Maham2023 ","paper_title":"Governing General Purpose AI: A Comprehensive Map of Unreliability, Misuse and Systemic Risks ","level":"Risk Category","risk_category":"Misuse Risks ","risk_subcategory":null,"description":"\"However, even if a model is entirely trustworthy and reliable, Misuse or Systemic Risks remain. General purpose AI models may present significant risks to society if this technology is misused by malicious actors to produce harmful outcomes. Misuse Risks span across Cyber Crime, Biosecurity Threats and Politically Motivated Misuse.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"52.02.01","quick_ref":"Maham2023 ","paper_title":"Governing General Purpose AI: A Comprehensive Map of Unreliability, Misuse and Systemic Risks ","level":"Risk Sub-Category","risk_category":"Misuse Risks ","risk_subcategory":"Cybercrime ","description":"\"The increasingly advanced capabilities and availability of general purpose AI models could be misused for improvements in efficiency and efficacy of cyber crimes. This is especially true for crimes that leverage IT systems, such as fraud144 (“cyber crime in the broader sense”).\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"52.02.02","quick_ref":"Maham2023 ","paper_title":"Governing General Purpose AI: A Comprehensive Map of Unreliability, Misuse and Systemic Risks ","level":"Risk Sub-Category","risk_category":"Misuse Risks ","risk_subcategory":"Biosecurity Threats","description":"\"The potential misuse of general purpose AI models also extends to biosecurity threats. Biological weapons are generally understood as biological toxins or infectious agents such as viruses that are intentionally released to cause disease and death.157 General purpose AI models could facilitate the production of biological weapons, by reducing barriers through access to critical knowledge or increasingly automated assistance and thus enable more malicious actors.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"52.02.03","quick_ref":"Maham2023 ","paper_title":"Governing General Purpose AI: A Comprehensive Map of Unreliability, Misuse and Systemic Risks ","level":"Risk Sub-Category","risk_category":"Misuse Risks ","risk_subcategory":"Politically motivated misuse ","description":"\"General purpose AI models could exacerbate existing tactics for political destabilisation, such as disinformation campaigns, and surveillance efforts if misused for political motivations. The technological advancements in text and media generation of general purpose AI models could refine disinformation164 attempts to shape and polarise public opinion or influence important political events.165 The improved automated processing of text, audio, image, and video could be used for surveillance measures and exacerbate human right violations and repression of political oppositions.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"52.03.01","quick_ref":"Maham2023 ","paper_title":"Governing General Purpose AI: A Comprehensive Map of Unreliability, Misuse and Systemic Risks ","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Economic Power Centralisation and Inequality","description":"\"Increasingly advanced general purpose AI models pose the risk of a concentration of economic power and exacerbation of existing inequalities through disparities in effective access to these models. This can materialise on multiple levels, between developers of general purpose AI models and companies building applications on them, between individuals and between countries on a global scale.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"52.03.02","quick_ref":"Maham2023 ","paper_title":"Governing General Purpose AI: A Comprehensive Map of Unreliability, Misuse and Systemic Risks ","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Ideological Homogenization from Value Embedding","description":"\"The increasing integration of general purpose AI models into every-day life raises concerns around their embedded normative values. The reach of a small number of AI models to a large number of people around the world can make these value judgements unprecedently impactful, potentially leading to increased ideological homogenization.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":1,"subdomain":"1.3"},{"ev_id":"52.03.03","quick_ref":"Maham2023 ","paper_title":"Governing General Purpose AI: A Comprehensive Map of Unreliability, Misuse and Systemic Risks ","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Disruptions from Outpaced Societal Adaptation","description":"\"Although the implementation of general purpose AI models as automation tools could be a major opportunity, overly rapid adoption of this technology at scale might outpace the ability of society to adapt effectively. This could lead to a variety of disruptions, including challenges in the labour market, the education system and public discourse, and various mental health concerns.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"53.01.01","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Sub-Category","risk_category":"Alignment failures in existing ML systems ","risk_subcategory":"Faulty reward functions in the wild ","description":"-","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.1"},{"ev_id":"53.02.02","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Sub-Category","risk_category":"Dangerous capabilities in AI systems ","risk_subcategory":"Acquisition of a goal to harm society ","description":"\"cases of AI systems being given the outright goal of harming humanity (ChaosGPT);\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"53.03.02","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Sub-Category","risk_category":"Direct catastrophe from AI ","risk_subcategory":"Gradual, irretrievable ceding of human power over the future to AI systems","description":"-","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"53.03.05","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Sub-Category","risk_category":"Direct catastrophe from AI ","risk_subcategory":"Dystopian trajectory lock-in because of misuse of advanced AI to establish and/or maintain totalitarian regimes;","description":"-","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"53.04.02","quick_ref":"Maas2023","paper_title":"Advancing AI Governance: A Literature Review of Problems, Options, and Proposals ","level":"Risk Sub-Category","risk_category":"Indirect AI contributions to existential risks","risk_subcategory":"Hazardous malicious uses ","description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"54.01.04","quick_ref":"Leech2024 ","paper_title":"Ten Hard Problems in Artificial Intelligence We Must Get Right","level":"Risk Sub-Category","risk_category":"Negative impacts of AI use ","risk_subcategory":"Privacy ","description":"\"OpenAI’s GPT-3 was designed to be dicult to extract personal information from, including for example public gures’ dates of birth. Even so, malicious uses of AI continue to encroach on privacy, as exemplied by China’s “Sharp Eye” automated surveillance system [551] and automated cyberattacks on personal data [354]. A more drastic form of AI-enabled surveillance could be on the way in the form of nonsurgical decoding of thoughts [54]—a technique which is reportedly already used by some police forces [398].\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"54.04.02","quick_ref":"Leech2024 ","paper_title":"Ten Hard Problems in Artificial Intelligence We Must Get Right","level":"Risk Sub-Category","risk_category":"Within-country issues: domestic inequality ","risk_subcategory":"Privatization of AI ","description":"\"Researchers in deep learning and those with greater research impact are more likely to migrate to industry, raising concerns about the “privatization of AI knowledge” [278]. Specically, if the most sophisticated AI approaches become proprietary and are used only within private research labs, then it will be impossible for universities to teach them, let alone contribute to leading research.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"55.02.00","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Category","risk_category":"Worsened conflict ","risk_subcategory":null,"description":"\"Cooperation and conflict: we’re seeing more focus and investment on the kinds of AI capabilities that make conflict more likely and severe, rather than those likely to improve cooperation. So, on our current trajectory, AI seems more likely to have negative long-term impacts in this area.\"","entity":"Human","intent":"Other","timing":"Other","domain":6,"subdomain":"6.4"},{"ev_id":"55.02.01","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Worsened conflict ","risk_subcategory":"AI enables development of weapons of mass destruction","description":"\"AI is already enabling the development of weapons which could cause mass destruction —including new weapons that themselves use AI capabilities, such as Lethal Autonomous Weapons [2],10 and the potential use of AI to speed up the development of other potentially dangerous technologies, such as engineered pathogens (as discussed in Section 2).\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"55.03.01","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Increased power concentration and inequality ","risk_subcategory":"Unequal distribution of harms and benefits ","description":"\"AI-driven industries seem likely to tend towards monopoly and could result in huge economic gains for a few actors: there seems to be a feedback loop whereby actors with access to more AI-relevant resources (e.g., data, computing power, talent) are able to build more effective digital products and services, claim a greater market share, and therefore be well-positioned to amass more of the relevant resources [14, 39, 45]. Similarly, wealthier countries able to invest more in AI development are likely to reap economic benefits more quickly than developing economies, potentially widening the ga","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"55.03.02","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Increased power concentration and inequality ","risk_subcategory":"AI-based automation increases income inequality ","description":"\"It seems quite plausible that progress in reinforcement learning and language models specifically could make it possible to automate a large amount of manual labour and knowledge work respectively [35, 45, 69], leading to widespread unemployment, and the wages for many remaining jobs being driven down by increased supply.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"55.03.03","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Increased power concentration and inequality ","risk_subcategory":"Developments in AI enable actors to undermine democratic processes ","description":"\"Developments in AI are giving companies and governments more control over individuals’ lives than ever before, and may possibly be used to undermine democratic processes. We are already seeing how the collection of large amounts of personal data can be used to surveil and influence populations, for example the use of facial recognition technology to surveil Uighur and other minority populations in China [66]. Further advances in language modelling could also be used to develop tools that can effectively persuade people of certain claims [42].\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"55.04.00","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Category","risk_category":"Worsened epistemic processes for society ","risk_subcategory":null,"description":"\"Epistemic processes and problem solving: we currently see more reasons to be concerned about AI worsening society's epistemic processes than reasons to be optimistic about AI helping us better solve problems as a society. For example, increased use of content selection algorithms could drive epistemic insularity and a decline in trust in credible multipartisan sources, which reducing our ability to deal with important long-term threats and challenges such as pandemics and climate change.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"55.04.01","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Worsened epistemic processes for society ","risk_subcategory":"AI contributes to increased online polarisation ","description":"\"One of the most significant commercial uses of current AI systems is in the content recommendation algorithms of social media companies, and there are already concerns that this is contributing to worsened polarisation online\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"55.04.02","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Worsened epistemic processes for society ","risk_subcategory":"AI is used to scale up production of false and misleading information ","description":"\"At the same time, we are seeing how AI can be used to scale up the production of convincing yet false or misleading information online (e.g. via image, audio, and text synthesis models like BigGAN [6] and GPT-3 [7]).\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"55.04.03","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Worsened epistemic processes for society ","risk_subcategory":"AI's persuasive capabilities are misused to gain influence and promote harmful ideologies ","description":"\"As AI capabilities advance, they may be used to develop sophisticated persuasion tools, such as those that tailor their communication to specific users to persuade them of certain claims [42]. While these tools could be used for social good— such as New York Times’ chatbot that helps users to persuade people to get vaccinated against Covid-19 [27]—there are also many ways they could be misused by self-interested groups to gain influence and/or to promote harmful ideologies.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"55.04.04","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Sub-Category","risk_category":"Worsened epistemic processes for society ","risk_subcategory":"Widespread use of persuasive tools contributes to splintered epistemic communities ","description":"\"Even without deliberate misuse, widespread use of powerful persuasion tools could have negative impacts. If such tools were used by many different groups to advance many different ideas, we could see the world splintering into isolated “epistemic communities”, with little room for dialogue or transfer between communities. A similar scenario could emerge via the increasing personalisation of people’s online experiences—in other words, we may see a continuation of the trend towards “filter bubbles” and “echo chambers”, driven by content selection algorithms, that some argue is already happening","entity":"Human","intent":"Unintentional","timing":"Other","domain":3,"subdomain":"3.2"},{"ev_id":"55.05.00","quick_ref":"Clarke2023","paper_title":"A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values","level":"Risk Category","risk_category":"AI leads to humans losing control of the future","risk_subcategory":null,"description":"\"The values that steer humanity’s future: humanity gaining more control over the future due to developments in AI, or losing our potential for gaining control, both seem possible. Much will depend on our ability to solve the alignment problem, who develops powerful AI first, and what they use it for. These long-term impacts of AI could be hugely important but are currently under-explored. We’ve attempted to structure some of the discussion and stimulate more research, by reviewing existing arguments and highlighting open questions. While there are many ways AI could in theory enable a flourish","entity":"Human","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"56.03.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Environmental impacts ","risk_subcategory":null,"description":"\"Increasing use of AI systems, and their growing energy needs, could also have environmental impacts. All of these could become more acute as AI becomes more capable.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.6"},{"ev_id":"56.04.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Amplification of biases","risk_subcategory":null,"description":"\"Current Frontier AI mdoels amplify existing biases within their training data and can be manipulated into providing potentially harmful responses, for example abusive language or discriminatory responses91,92. This is not limited to text generation but can be seen across all modalities of generative AI93. Training on large swathes of UK and US English internet content can mean that misogynistic, ageist, and white supremacist content is overrepresented in the training data94.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"56.05.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Harmful responses ","risk_subcategory":null,"description":"\"Current Frontier AI mdoels amplify existing biases within their training data and can be manipulated into providing potentially harmful responses, for example abusive language or discriminatory responses91,92. This is not limited to text generation but can be seen across all modalities of generative AI93. Training on large swathes of UK and US English internet content can mean that misogynistic, ageist, and white supremacist content is overrepresented in the training data94.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"56.07.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Intellectual property rights ","risk_subcategory":null,"description":"\"There are also issues around intellectual property rights for content in training datasets\" ","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"56.08.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Providing new capabilities to a malicious actor ","risk_subcategory":null,"description":null,"entity":"Human","intent":"Other","timing":"Other","domain":null,"subdomain":null},{"ev_id":"56.09.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Misapplication by a non-malicious actor ","risk_subcategory":null,"description":null,"entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"56.14.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Overreliance on AI systems, which cannot be subsequently unpicked ","risk_subcategory":null,"description":null,"entity":"Human","intent":"Unintentional","timing":"Other","domain":5,"subdomain":"5.1"},{"ev_id":"56.15.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Societal concerns around AI reduce the realisation of potential benefits ","risk_subcategory":null,"description":null,"entity":"Human","intent":"Unintentional","timing":"Other","domain":null,"subdomain":null},{"ev_id":"56.17.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Single point of failure ","risk_subcategory":null,"description":"\"Intense competition leads to one company gaining a technical edge, exploiting this to the point its model controls, or is the basis for other models controlling, multiple key systems. Lack of safety, controllability, and misuse cause these systems to fail in unexpected ways.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"56.18.00","quick_ref":"GOS2023","paper_title":"Future Risks of Frontier AI ","level":"Risk Category","risk_category":"Overreliance","risk_subcategory":null,"description":"\"As AI capability increases, humans grant AI more control over critical systems and eventually become irreversibly dependent on systems they don’t fully understand. Failure and unintended outcomes cannot be controlled.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":5,"subdomain":"5.2"},{"ev_id":"58.01.02","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Autonomy","risk_subcategory":"Impersonation/identity theft ","description":"\"Impersonation/identity theft - Theft of an individual, group or organisation’s identity by a third-party in order to defraud, mock or otherwise harm them.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"58.01.03","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Autonomy","risk_subcategory":"IP/copyright loss ","description":"\"IP/copyright loss - Misuse or abuse of an individual or organisation’s intellectual property, including copyright, trademarks, and patents.\"","entity":"Human","intent":"Intentional","timing":"Other","domain":4,"subdomain":"4.3"},{"ev_id":"58.02.02","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Physical ","risk_subcategory":"Loss of Life ","description":"\"Loss of life - Accidental or deliberate loss of life, including suicide, extinction or cessation, due to the use or misuse of a technology system.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"58.03.02","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Psychological ","risk_subcategory":"Alienation/isolation ","description":"\"Alienation/isolation - An individual’s or group’s feeling of lack of connection with those around as a result of technology use or misuse.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"58.03.04","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Psychological ","risk_subcategory":"Coercion/manipulation ","description":"\"Coercion/manipulation - Use of a technology system to covertly alter user beliefs and behaviour using nudging, dark patterns and/or other opaque techniques, resulting in potential erosion of privacy, addiction, anxiety/distress, etc.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"58.03.05","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Psychological ","risk_subcategory":"Dehumanisation/objectification ","description":"\"Dehumanisation/objectification - Use or misuse of a technology system to depict and/or treat people as not human, less than human, or as objects.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"58.03.07","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Psychological ","risk_subcategory":"Overreliance ","description":"\"Over-reliance - Unfettered and/or obsessive belief in the accuracy or other quality of a technology system, resulting in addiction, anxiety, introversion, sentience, complacency, lack of critical thinking and other actual or potential negative impacts.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"58.04.01","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Reputational ","risk_subcategory":"Defamation/libel/slander","description":"\"Defamation/libel/slander - Use of a technology system to create, facilitate or amplify false perception(s) about an individual, group, or organisation.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"58.05.00","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Category","risk_category":"Financial and business","risk_subcategory":null,"description":"\"Financial and Business - Use or misuse of a technology system in a manner that damages the financial interests of an individual or group, or which causes strategic, operational, legal or financial harm to a business or other organisation.\"\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"58.05.03","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Financial and business","risk_subcategory":"Financial/earnings loss","description":"\"Financial/earnings loss - Loss of money, income or value due to the use or misuse of a technology system.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"58.05.05","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Financial and business","risk_subcategory":"Increased competition","description":"\"Increased competition - The inappropriate or unethical use of technology to gain market share.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.4"},{"ev_id":"58.05.06","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Financial and business","risk_subcategory":"Monopolisation ","description":"\"Monopolisation - Abuse of market power through the control of prices, thereby limiting competition and creating unfair barriers to entry.\"","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"58.06.00","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Category","risk_category":"Human rights and civil liberties","risk_subcategory":null,"description":"\"Human Rights and Civil Liberties - Use or misuse of a technology system in a manner that compromises fundamental human rights and freedoms.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"58.06.11","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Human rights and civil liberties","risk_subcategory":"Privacy loss ","description":"\"Privacy loss - Unwarranted exposure of an individual’s private life or personal data through cyberattacks, doxxing, etc.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"58.07.09","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Societal and Cultural ","risk_subcategory":"Labour exploitation ","description":"\"Labour exploitation - Use of under-paid and/or offshore labour to develop, manage or optimise a technology system.\"","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.2"},{"ev_id":"58.07.15","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Societal and Cultural ","risk_subcategory":"Violence/armed conflict","description":"\"Violence/armed conflict - Use or misuse of a technology system to incite, facilitate or conduct cyberattacks, security breaches, lethal, biological and chemical weapons development, resulting in violence and armed conflict.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"58.08.02","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Political and Economic ","risk_subcategory":"Economic instability ","description":"\"Economic instability - Uncontrolled fluctuations impacting the financial system, or parts thereof, due to the use or misuse of a technology system, or set of systems.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"58.08.04","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Political and Economic ","risk_subcategory":"Electoral interference ","description":"\"Electoral interference - Generation of false or misleading information that can interrupt or mislead voters and/or undermine trust in electoral processes.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"58.08.06","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Political and Economic ","risk_subcategory":"Political instability ","description":"\"Political instability - Political polarisation or unrest caused by increased inequality, job losses, over- dependence on technology making societies vulnerable to systemic failures, etc, arising from or amplified by the use or misuse of a technology system.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"58.08.07","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Political and Economic ","risk_subcategory":"Political manipulation ","description":"\"Political manipulation - Use or misuse of personal data to target individuals’ interests, personalities and vulnerabilities with tailored political messages via micro-advertising or deepfakes/synthetic media.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"58.09.04","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Environmental ","risk_subcategory":"Excessive energy consumption ","description":"\"Excessive energy consumption - Excessive energy use, leading to energy bottlenecks and shortages for communities, organisations, and businesses.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"58.09.05","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Environmental ","risk_subcategory":"Excessive landfill ","description":"\"Excessive landfill - Excessive disposal of electrical or electronic equipment leading to ecological/biodiversity damage, and disrupting the livelihoods and eroding the rights of local communities.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"58.09.06","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Environmental ","risk_subcategory":"Excessive water consumption ","description":"\"Excessive water consumption - Excessive use of water to cool data centres and for other purposes, leading to water restrictions or shortages for local communities or businesses.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"58.09.07","quick_ref":"Abercrombie2024","paper_title":"A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms","level":"Risk Sub-Category","risk_category":"Environmental ","risk_subcategory":"Natural resource depletion","description":"\"Natural resource depletion - Extraction of minerals, metals, rare earths, and fossil fuels that deplete natural resources and increase carbon emissions.\"","entity":"Human","intent":"Other","timing":"Other","domain":6,"subdomain":"6.6"},{"ev_id":"59.01.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Inadequate specification of ODD","risk_subcategory":null,"description":"\"The operational design domain (ODD) is a technical description of the application’s operational environment, initially conceptualized for autonomous driving systems. An inadequate specification of the ODD limits essential functions such as testing the learned functionality and out-of-distribution detection.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.03.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Inadequate planning of performance requirements","risk_subcategory":null,"description":"\"The expected performance of the AI system should be planned adequately. Hereby, an important aspect is that chosen performance metrics are meaningful for presenting the intended functionality. Otherwise, expectations and safety requirements can be unfulfillable at later life cycle stages.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.04.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Insufficient AI development documentation","risk_subcategory":null,"description":"\"Throughout the development of an AI system, it is vital to document every decision and action taken. This is not only essential to optimize the development process itself but also required for the auditability of the AI system.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"59.05.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Inappropriate degree of transparency to end users","risk_subcategory":null,"description":"\"The transparency to end users of the AI system increases the user’s trust in the AI application. If not adequately integrated into the design, this might prevent the proper operation and cause potential misuse of the AI application.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"59.07.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Choice of untrustworthy data source","risk_subcategory":null,"description":"\"The choice of a trustworthy data source is a first prerequisite in order to fulfill data quality requirements. This is especially the case if third-party data sources are used to develop the AI system.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"59.08.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Lack of data understanding","risk_subcategory":null,"description":"\"The correct understanding of the used data for developing an AI system is a prerequisite to avoid data shortcomings and hinders the development of an AI system which is best suiting for the intended functionality.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"59.11.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Incorrect data labels","risk_subcategory":null,"description":"\"Data labels are essential for any supervised learning algorithm since they preset the result of the learning process. If the correctness of the data labels is not given, the AI system is prevented from learning the ground truth and therefore the intended functionality.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.12.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Data poisoning","risk_subcategory":null,"description":"\"Data poisoning describes an attack in the form of an injection of malicious data into the training set. If not prevented, this attack leads the AI system to learn unintended behavior.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"59.15.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Inappropriate data splitting","risk_subcategory":null,"description":"\"In data-driven AI development, the annotated data set is commonly split into training, validation, and test sets, whereby it is essential that the latter is not used for development but only for evaluation. Using the test set for training manipulates the testing strategy, which is the basis of the system’s quality assurance.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"59.16.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Poor model design choices","risk_subcategory":null,"description":"\"The model specifications have significant impact on the functionality of an AI system. The developer mak- ing wrong decisions might cause the AI system to behave biased and unreliable.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.21.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Uncertainty concerns","risk_subcategory":null,"description":"\"AI systems should be able not only to return output for a given instance but also to provide a corresponding level of confidence. If such a method is not implemented or not working correctly, this can have a negative impact on performance and safety.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"59.26.02","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Sub-Category","risk_category":"Mode","risk_subcategory":"Socio-technical ","description":"\"In contrast to technical AI hazards, socio-technical hazards also require hu- man input related to social and cultural aspects [45]. Human judgment must be employed when deciding on quantification and treatment methods. For instance, AI hazards concerning discrimination and privacy, which are abstract concepts lacking a uniform technical definition, further complicate a clear quantification of the associated risks. Although quantitative methods exist to assess and treat these AI hazards, they require coordination with social and cultural values [27].\"","entity":"Human","intent":"Other","timing":"Other","domain":null,"subdomain":null},{"ev_id":"59.26.03","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Sub-Category","risk_category":"Mode","risk_subcategory":"Procedural ","description":"\"The third class encompasses procedural AI hazards. These pertain to issues arising from processes and actions made by individuals involved in the develop- ment process. Such hazards are not readily quantifiable and necessitate alter- native mitigation strategies. An example of such an AI hazard would be ”poor model design choices,” which could be expressed, for instance, through a devel- oper’s decision to select an unsuitable AI model for a given problem. Due to the challenges in quantifying and mitigating these issues, qualitative approaches must be employed. In the case of the aforemention","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"59.27.00","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Category","risk_category":"Level ","risk_subcategory":null,"description":"\"The third axis of the taxonomy pertains to the level, which differentiates between the AI application and system levels, as they are defined in Section 3. Allocating an AI hazard to its level helps to determine the level at which an action is required. This consequently sets the basis for who is supposed to act.\"","entity":"Human","intent":"Other","timing":"Other","domain":null,"subdomain":null},{"ev_id":"59.27.01","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Sub-Category","risk_category":"Level ","risk_subcategory":"AI application ","description":"\"For instance, the main person responsible for an AI hazard manifesting on the AI system level would be the AI developer, whereas an AI hazard affecting the whole AI application requires a more diverse group, including domain experts.\"","entity":"Human","intent":"Other","timing":"Other","domain":null,"subdomain":null},{"ev_id":"59.27.02","quick_ref":"Schnitzer2024","paper_title":"AI Hazard Management: A Framework for the Systematic Management of Root Causes for AI Risks","level":"Risk Sub-Category","risk_category":"Level ","risk_subcategory":"AI system ","description":"\"For instance, the main person responsible for an AI hazard manifesting on the AI system level would be the AI developer, whereas an AI hazard affecting the whole AI application requires a more diverse group, including domain experts.\"","entity":"Human","intent":"Other","timing":"Other","domain":null,"subdomain":null},{"ev_id":"60.01.00","quick_ref":"Bengio2025","paper_title":"International AI Safety Report 2025","level":"Risk Category","risk_category":"Risks from malicious use ","risk_subcategory":null,"description":"- ","entity":"Human","intent":"Intentional","timing":"Other","domain":4,"subdomain":"4.0"},{"ev_id":"60.01.01","quick_ref":"Bengio2025","paper_title":"International AI Safety Report 2025","level":"Risk Sub-Category","risk_category":"Risks from malicious use ","risk_subcategory":"Harm to individuals through fake content ","description":"\"Malicious actors can use general- purpose AI to generate fake content that harms individuals in a targeted way. For example, they can use such fake content for scams, extortion, psychological manipulation, generation of non- consensual intimate imagery (NCII) and child sexual abuse material (CSAM), or targeted sabotage of individuals and organisations.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"60.01.02","quick_ref":"Bengio2025","paper_title":"International AI Safety Report 2025","level":"Risk Sub-Category","risk_category":"Risks from malicious use ","risk_subcategory":"Manipulation of public opinion ","description":"\"Malicious actors can use general- purpose AI to generate fake content such as text, images, or videos, for attempts to manipulate public opinion. Researchers believe that if successful, such attempts could have several harmful consequences.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"60.01.03","quick_ref":"Bengio2025","paper_title":"International AI Safety Report 2025","level":"Risk Sub-Category","risk_category":"Risks from malicious use ","risk_subcategory":"Cyber offence ","description":"\"Attackers are beginning to use general- purpose AI for offensive cyber operations, presenting growing but currently limited risks. Current systems have demonstrated capabilities in low- and medium- complexity cybersecurity tasks, with state- sponsored threat actors actively exploring AI to survey target systems. Malicious actors of varying skill levels can leverage these capabilities against people, organisations, and critical infrastructure such as power grids.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"60.01.04","quick_ref":"Bengio2025","paper_title":"International AI Safety Report 2025","level":"Risk Sub-Category","risk_category":"Risks from malicious use ","risk_subcategory":"Biological and chemical attacks ","description":"\"Growing evidence shows general- purpose AI advances beneficial to science while also lowering some barriers to chemical and biological weapons development for both novices and experts. New language models can generate step- by- step technical instructions for creating pathogens and toxins that surpass plans written by experts with a PhD and surface information that experts struggle to find online, though their practical utility for novices remains uncertain. Other models demonstrate capabilities in engineering enhanced proteins and analysing which candidate pathogens or toxins are most harmfu","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"60.02.01","quick_ref":"Bengio2025","paper_title":"International AI Safety Report 2025","level":"Risk Sub-Category","risk_category":"Risks from malfunctions ","risk_subcategory":"Reliability issues ","description":"\"Relying on general-purpose AI products that fail to fulfil their intended function can lead to harm. For example, general- purpose AI systems can make up facts (‘hallucination’), generate erroneous computer code, or provide inaccurate medical information. This can lead to physical and psychological harms to consumers and reputational, financial and legal harms to individuals and organisations.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"60.02.02","quick_ref":"Bengio2025","paper_title":"International AI Safety Report 2025","level":"Risk Sub-Category","risk_category":"Risks from malfunctions ","risk_subcategory":"Bias ","description":"\"General-purpose AI systems can amplify social and political biases, causing concrete harm. They frequently display biases with respect to race, gender, culture, age, disability, political opinion, or other aspects of human identity. This can lead to discriminatory outcomes including unequal resource allocation, reinforcement of stereotypes, and systematic neglect of certain groups or viewpoints.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"60.03.06","quick_ref":"Bengio2025","paper_title":"International AI Safety Report 2025","level":"Risk Sub-Category","risk_category":"Systemic risks ","risk_subcategory":"Risks of copyright infringement ","description":"\"The use of vast amounts of data for training general- purpose AI models has caused concerns related to data rights and intellectual property. Data collection and content generation can implicate a variety of data rights laws, which vary across jurisdictions and may be under active litigation. Given the legal uncertainty around data collection practices, AI companies are sharing less information about the data they use. This opacity makes third- party AI safety research harder.\"","entity":"Human","intent":"Other","timing":"Other","domain":6,"subdomain":"6.3"},{"ev_id":"61.01.08","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Types of systemic risks from general-purpose AI","risk_subcategory":"Harms to non-humans ","description":"\"Large-scale harms to animals and the development of AI capable of suffering.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.5"},{"ev_id":"61.01.11","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Types of systemic risks from general-purpose AI","risk_subcategory":"Power ","description":"\"The concentration of military, economic, or political power of entities in possession or control of AI or AI-enabled technologies.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"61.02.02","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Ability to enhance and modify pathogens ","description":"\"AI can be used to enhance pathogens, making them more lethal or resistant to treatments.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"61.02.03","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Ability to persuade ","description":"\"AI could be used to develop sophisticated tools to manipulate and persuade individuals.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"61.02.05","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"AI in totalitarian regimes ","description":"\"AI-based surveillance and manipulation could be used to maintain global totalitarian regimes.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"61.02.08","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Automation bias","description":"\"The tendency for humans to over-rely on AI models and systems, trusting their outputs without sufficient critical evaluation, which can lead to poor decision-making.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"61.02.09","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Autonomy risk","description":"\"Granting AI models and systems high levels of decision-making autonomy can lead to unintended consequences.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.2"},{"ev_id":"61.02.11","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Centralized platforms deployed at scale","description":"\"The widespread use of common AI platforms can create centralized points of failure, making systems more vulnerable to disruptions or attacks\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"61.02.13","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Combination failures","description":"\"Harms could result from a combination of regulatory, management, and operational failures.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"61.02.14","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Complex attribution and responsibility","description":"\"When multiple actors are involved in AI development and deployment, it becomes difficult to assign responsibility for harm, complicating accountability.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"61.02.16","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Conflicting objectives in design","description":"\"Designers and operators of AI may face conflicting objectives that compromise safety.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":null,"subdomain":null},{"ev_id":"61.02.17","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Dangerous development races","description":"\"Competitive pressures could lead to the neglect of safety measures in AI development.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":6,"subdomain":"6.4"},{"ev_id":"61.02.19","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Dependency on providers","description":"\"Excessive reliance on specific AI providers can lead to vulnerabilities due to lack of alternatives or interoperability.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"61.02.25","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Exploitation in AI development","description":"\"Outsourcing tasks like data labeling to low-income countries can perpetuate inequality.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"61.02.26","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Geopolitical competition for superiority","description":"\"Strategic competition between nations over AI capabilities could heighten global tensions and destabilize international relations.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.4"},{"ev_id":"61.02.28","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Human choice of overreliance in critical sectors","description":"\"Heavy reliance on AI in critical sectors like finance or healthcare can exacerbate issues related to size, speed, interconnectivity, and complexity of the system.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"61.02.33","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Limitations in adversarial robustness","description":"\"AI models and systems are vulnerable to manipulation through adversarial inputs.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"61.02.44","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Terrorist access","description":"\"Powerful AI technologies may fall into the hands of terrorists.\"","entity":"Human","intent":"Other","timing":"Other","domain":4,"subdomain":"4.2"},{"ev_id":"61.02.48","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Weaponization capabilities","description":"\"AI capabilities that could be deliberately weaponized for destructive purposes.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"61.02.49","quick_ref":"Uuk2025","paper_title":"A Taxonomy of Systemic Risks from General-Purpose AI ","level":"Risk Sub-Category","risk_category":"Sources of systemic risks from general-purpose AI ","risk_subcategory":"Widespread use of persuasion tools","description":"\"Widespread use of AI-powered persuasion tools could lead to systemic harm\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"62.02.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Dimension - Entity ","risk_subcategory":"Human ","description":"\"A risk may be triggered by a human, where the AI serves merely as a tool, or by the AI acting autonomously with no human intervention, or it may involve a combination of both, with the human delegating some parts of decision-making to the AI. For risks where AI is the entity, these risks are exacerbated by an increase in the AI’s level of autonomy. To manage risks involving AI as the trigger, appropriate levels of human oversight can be built-in.\"","entity":"Human","intent":"Not coded","timing":"Not coded","domain":null,"subdomain":null},{"ev_id":"62.14.00","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Category","risk_category":"Model Development ","risk_subcategory":null,"description":"-","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":null,"subdomain":null},{"ev_id":"62.14.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Data-related (Difficulty filtering large web scrapes or large scale web datasets)","description":"\"A large scale “scraping” of web data for training datasets increases vulnerability to data poisoning, backdoor attacks, and the inclusion of inaccurate or toxic data [76, 28, 48]. With a large dataset, filtering out these quality issues is very difficult or trades off against significant data loss.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.14.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Data-related (Lack of cross-organizational documentation)","description":"\"When sharing data between multiple organizations, documentation may be missing or inadequate, making it difficult for other organizations to understand it. For example, a lack of metadata or a change in schema by a collaborating party can result in an unusable dataset and wasted data collection efforts, or it can lead to misunderstandings about the dataset’s limitations, resulting in downstream risks related to its use [173].\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.14.03","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Data-related (Manipulation of data by non-domain experts)","description":"\"Manipulating data (e.g., training data) carries a set of assumptions on how the data should appear and be used by those performing the manipulation. Common manipulations applied on data in the context of AI models include defining the ground truth label and merging different data formats or sources. People who have little or no expertise in the domain of the data performing such manipulations may render the data unusable or harmful to the development of the AI system [173].\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"62.14.04","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Data-related (Insufficient quality control in data collection process)","description":"\"A lack of standardized methods and sufficient infrastructure, including the absence of quality control processes for collecting data, especially for high-stakes domains and benchmarks, can affect the quality and type of the data collected [173, 95]. This may include risks of dataset poisoning, inadvertent copyright violation, and test set leakages which invalidate performance metrics.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.14.05","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Training-related (Adversarial examples)","description":"\"Adversarial examples [198, 83] refer to data that are designed to fool an AI model by inducing unintended behavior. They do this by exploiting spurious correlations learned by the model. They are part of inference-time attacks, where the examples are test examples. They generalize to different model architectures and models trained on different training sets.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.15.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Training-related (Robustness certificates can be exploited to attack the models)","description":"\"The knowledge of robustness certificates, including the area of the region for which model predictions are certified to be robust, can be used by an adversary to efficiently craft attacks that succeed just outside the certified regions [53].\"","entity":"Human","intent":"Intentional","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"62.15.03","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Fine-tuning related (Ease of reconfiguring GPAI models)","description":"\"GPAI models are often easily reconfigured for various use cases or have competencies beyond the intended use [78, 225]. They can be performed either by changing the weights of the model (e.g., fine-tuning) or by modifying only the model inputs (e.g., prompt engineering, jailbreaking, retrieval-augmented generation). Reconfiguration can be intentional (with the help of adversarial inputs) or unintentional (from unanticipated inputs to the model).\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"62.15.04","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Fine-tuning related (Unexpected competence in fine-tuned versions of the upstream model)","description":"\"Downstream deployers may often fine-tune a GPAI model with specific deploy- ment-related datasets, to better suit the task. Fine-tuned upstream models can gain new or unexpected capabilities that the underlying upstream models did not exhibit [202, 126, 137]. These new capabilities may be unanticipated by the original model developer.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.2"},{"ev_id":"62.15.05","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Fine-tuning related (Harmful fine-tuning of open-weights models)","description":"\"Models with publicly available weights can be fine-tuned for harmful activities by bad actors, using significantly fewer resources (in terms of time and money) compared to the original training cost [115, 78].\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"62.15.06","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Fine-tuning related (Fine-tuning dataset poisoning)","description":"\"A deployer can poison the dataset used during the fine-tuning process [98] to induce specific, often malicious, behaviors in a model. This can be performed without having access to the model’s weights. This poisoning can be difficult to detect through direct inspection of the dataset, as the manipulations may be subtle and targeted.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.15.07","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Fine-tuning related (Poisoning models during instruction tuning)","description":"\"AI models can be poisoned during instruction tuning when models are tuned using pairs of instructions and desired outputs. Poisoning in instruction tuning can be achieved with a lower number of compromised samples, as instruction tuning requires a relatively small number of samples for fine-tuning [155, 211]. Anonymous crowdsourcing efforts may be employed in collecting instruction tuning datasets and can further contribute to poisoning attacks [187]. These attacks might be harder to detect than traditional data poisoning attacks.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.15.09","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Development ","risk_subcategory":"Fine-tuning related (Degrading safety training due to benign fine-tuning) ","description":"\"When downstream providers of AI systems fine-tune AI models to be more suitable for their needs, the resulting AI model can be more likely to produce undesired or harmful outputs (as compared to the non-fine-tuned model), even if the fine-tuning was done with harmless and commonly used data [154].\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.0"},{"ev_id":"62.16.00","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Category","risk_category":"Model Evaluations","risk_subcategory":null,"description":"\"This section catalogs the risk sources and risk management measures related to model evaluations (often called evals). We categorize them into the fol- lowing groups: general evaluations, benchmarking, red teaming, auditing, and interpretability/explainability. The subsection on general evaluations consists of items that are common to various evaluation techniques, while the other subsections are specific to their respective evaluation types.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":null,"subdomain":null},{"ev_id":"62.16.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"General Evaluations (Limited coverage of capabilities evaluations)","description":"\"GPAI model developers might run capabilities evaluations to determine whether it has dangerous or dual-use capabilities, and then decide whether it is safe to deploy. Such capabilities evaluations can fail to demonstrate all the capabilities of a model. For example, evaluations may miss certain capabilities that are difficult to assess, prohibitively costly to verify, or obscured by the model’s tendency to refuse responses due to safety training, even if it possesses some of these capabilities.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.06","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"General Evaluations (Biased evaluations of encoded human values)","description":"\"Encoded human values in AI models that are easier to evaluate might be preferred for inclusion in evaluations over those that are more difficult to measure [13]. This might come at the expense of more desirable but harder-to-quantify  values. This bias can lead to an imbalance, where easier-to-measure values dominate the evaluation process, while other important values are underrepresented.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.08","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"Benchmarking (Benchmark leakage or data contamination)","description":"\"Benchmark leakage [235, 224, 221, 161] can happen when an AI model is trained or fine-tuned with evaluation-related data. This can lead to an unreliable model evaluation, especially if the data contains question-answer pairs from bench- marks.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.09","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"Benchmarking (Raw data contamination)","description":"\"This type of contamination [170] occurs when the raw and unlabeled data of a benchmark is used as part of the training set. Such data may not be properly formatted and may contain noise, especially if the contamination happens before the data is pre-processed into the benchmark. If this contamination occurs, it could cast doubt on the few-shot and zero-shot performance of the model on that benchmark.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.10","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"Benchmarking (Cross-lingual data contamination)","description":"\"Models that have been trained on data encoded in multiple languages, such as LLMs trained on web-crawled data, may contain contamination that is obscured by translation [226]. The most basic form of this is when a benchmark is trans- lated to another language and then fed to the model as training data. The fact that the benchmark is translated before becoming training data can obscure the contamination from detection methods, giving false assurance that the model has generalized on the capabilities that the benchmark tests for.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.11","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"Benchmarking (Guideline contamination)","description":"\"Guideline contamination refers to scenarios where instructions for the collec- tion, annotation, or use of the dataset are exposed to the model [170]. These instructions may contain explicit data-label pairs that can improve the model’s capabilities for the task.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.12","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"Benchmarking (Annotation contamination)","description":"\"Annotation contamination refers to scenarios where the model is exposed to the benchmark labels during training [170]. This type of contamination can make the model learn the acceptable distribution of outputs. Combining this with raw data contamination of the test split, any evaluation made with the benchmark is invalidated because the entire test split is essentially leaked to the model.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.16.14","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations","risk_subcategory":"Benchmark Inaccuracy (Benchmarks may not accurately evaluate capabilities)","description":"\"Benchmarks of AI systems can both underestimate and overestimate the capa- bilities of those AI systems. Underestimates can happen if an evaluation is not comprehensive enough, if the benchmark is saturated by existing models, or if the capabilities in question depend on a complicated setup, such as realistic computer programming tasks. Overestimates of capabilities can occur if an AI system is trained or fine-tuned on the contents of the benchmark, leading to overfitting.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.17.00","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Category","risk_category":"Model Evaluations (Auditing) ","risk_subcategory":null,"description":"-","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.17.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations (Auditing) ","risk_subcategory":"Conflicts of interest in auditor selection","description":"\"Conflicts of interest can arise if there is no independence in the auditor selection process or if the auditors are closely associated with the developer [123, 157]. In such cases, the conflict of interest can appear even if third-party evaluators are involved. In the case of external auditing, the potential candidates might be selected from a narrow group of auditors, or have conflicting financial incentives for whether to report model shortcomings publicly.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.17.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations (Auditing) ","risk_subcategory":"Auditor capacity mismatch","description":"\"Auditors may not be able to address all of the specific safety, performance, or validation needs. Reports of passing audits may be more inclusive than can be justified due to a lack of knowledge of specific risks and how they can be tested, or a lack of capacity to perform sufficiently rigorous testing.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.17.03","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations (Auditing) ","risk_subcategory":"Auditor failure","description":"\"Auditors may not publicly disclose risks they find, may be required to not pub- licize shortcomings, or may not receive sufficient cooperation from the relevant internal parties.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"62.18.00","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Category","risk_category":"Model Evaluations (Interpretability/Explainability) ","risk_subcategory":null,"description":null,"entity":"Human","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"62.18.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations (Interpretability/Explainability) ","risk_subcategory":"Misuse of interpretability techniques","description":"\"Interpretability techniques, by enabling a better understanding of the model, could potentially be used for harmful purposes. For example, mechanistic inter- pretability could be used to identify neurons responsible for specific functions, and certain neurons that encode safety-related features may be modified to de- crease its activation or certain information may be censored [24]. Furthermore, interpretability techniques can be used to simulate a white-box attack scenario. In this case, knowing the internal workings of a model aids in the development of adversarial attacks [24].\"","entity":"Human","intent":"Intentional","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"62.18.03","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Model Evaluations (Interpretability/Explainability) ","risk_subcategory":"Adversarial attacks targeting explainable AI techniques","description":"\"Adversarial attacks can affect not only the model’s output but also its corresponding explanation. Current adversarial optimization techniques can intro- duce imperceptible noise to the input image, so that the model’s output does not change but the corresponding explanation is arbitrarily manipulated [61]. Such manipulations are harder to notice, as they are less commonly known compared to standard adversarial attacks targeting the model’s output.\"","entity":"Human","intent":"Intentional","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"62.19.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Attacks on GPAIs/GPAI Failure Modes ","risk_subcategory":"Jailbreak of a model to subvert intended behavior","description":"\"A jailbreak is a type of adversarial input to the model (during deployment) re- sulting in model behavior deviating from intended use. Jailbreaks may be gen- erated automatically in a “white box” setting, where access to internal training parameters is required for creation and optimization of the attack [238]. Other attacks may be “black box” - without access to model internals. In text based generative models, jailbreaks may sometimes be human-readable, with the use of reasoning or role-play to “convince” the model to bypass its safety mechanisms [231].\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.19.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Attacks on GPAIs/GPAI Failure Modes ","risk_subcategory":"Jailbreak of a multimodal model","description":"\"Current generation multimodal (e.g., vision and language) GPAI models are vulnerable to adversarial jailbreak attacks. These attacks can be used to automatically induce a model to produce an arbitrary or specific output with high success rate [227]. Multimodal jailbreaks can also be used to exfiltrate a model’s context window or other model internals [18].\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.19.03","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Attacks on GPAIs/GPAI Failure Modes ","risk_subcategory":"Transferable adversarial attacks from open to closed-source mod- els","description":"\"In some cases, an adversarial attack developed for an open-weights and open- source model (where the weights and architecture are known - a “white box” attack) can be transferable to closed-source models, despite the defenses put in place by the closed-source model provider (such as structured access). These adversarial attacks can be generated automatically [238].\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.19.04","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Attacks on GPAIs/GPAI Failure Modes ","risk_subcategory":"Backdoors or trojan attacks in GPAI models","description":"\"Backdoors can be inserted into GPAI models during their training or fine-tuning, to be exploited during deployment [185, 118]. Attackers inserting the backdoor can be the GPAI model provider themselves or another actor (e.g., by ma- nipulating the training data or the software infrastructure used by the model provider) [222]. Some backdoors can be exploited with minimal overhead, al- lowing attackers to control the model outputs in a targeted way with a high success rate [90].\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.19.05","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Attacks on GPAIs/GPAI Failure Modes ","risk_subcategory":"Text encoding-based attacks","description":"\"Various new or existing text encodings, such as Base64, can be employed to craft jailbreak attacks that bypass safety training [13]. Low-resource language inputs also appear more likely to circumvent a model’s safeguards [229]. Since safety fine-tuning might not involve this encoding data or may only do so to a limited extent, harmful natural language prompts could be translated into less frequently used encodings [214].\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.19.12","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Attacks on GPAIs/GPAI Failure Modes ","risk_subcategory":"Misuse of AI model by user-performed persuasion","description":"\"AI models can be influenced to accept misinformation through persuasive conversations, even when their initial responses are factually correct. Multi-turn persuasion can be more effective than single-turn persuasion attempts in altering the model’s stance [223].\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.27.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Deployment (Model Release) ","risk_subcategory":"Non-decomissionability of models with open weights","description":"\"If the model parameter weights are released or leaked in a security breach, the model cannot be decommissioned because the developer no longer has control over the publicly available model or its use. This prevents effective management and control of an open-sourced or leaked model. Models with publicly available weights are also easier to reconfigure, enabling misuse [178].\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.28.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Cybersecurity ","risk_subcategory":"Interconnectivity with malicious external tools","description":"\"The growing integration and interconnectivity with external tools and plugins increase the risk of exposure to malicious external inputs. This interconnectivity makes it easier for external tools to introduce harmful content [220].\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.28.04","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Cybersecurity ","risk_subcategory":"Model weight leak","description":"\"Model weights or access to them can be leaked when initial access is granted only to a select group of individuals, such as institutional researchers [209]. This risk can increase as more people gain access, and identifying the source of the leak becomes more difficult. The availability of leaked model weights makes various attacks on systems that use the leaked AI model easier to implement, such as finding adversarial examples, elicitation of dangerous capabilities, and extraction of confidential information present in the training data. The avail- ability of model weights might also enable ","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"62.29.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (General) ","risk_subcategory":"High-impact misuses and abuses beyond original purpose","description":"\"Since general-purpose AI systems have a large repertoire of capabilities, mali- cious actors such as foreign actors can use such systems to cause large damage if they gain unrestricted or unmonitored access to those AI systems.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"62.29.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (General) ","risk_subcategory":"Democratizing access to dual-use technologies","description":"\"Access to dual-use technologies can become easier because of GPAI model pro- liferation (in particular, open-source or open-weights models). Non-experts can use such dual-use-capable systems at a minimal cost [194, 100]. Improved model capabilities also contribute to dual-use risks posed by malicious actors. For example, an open-source base model for generating high quality sequence data can be modified to generate candidate protein sequences for toxin synthesis [29].\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"62.29.03","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (General) ","risk_subcategory":"Competitive pressures in GPAI product release","description":"\"In competitive situations, developers of general-purpose AI systems might cut corners on the safety evaluation of their GPAI model and instead spend more time and effort on the capabilities of those systems [183, 69]. This is especially dangerous if the capabilities of such AI systems are correlated with the risk they pose [162].\"","entity":"Human","intent":"Intentional","timing":"Other","domain":6,"subdomain":"6.4"},{"ev_id":"62.29.03a","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Additional evidence","risk_category":"Impacts of AI (General) ","risk_subcategory":"Competitive pressures in GPAI product release","description":null,"entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":6,"subdomain":"6.4"},{"ev_id":"62.30.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Physical) ","risk_subcategory":"AI-based tools attacking critical infrastructure","description":"\"Critical infrastructure can also be damaged without AI integration, for instance, when AI-based tools are used indirectly to aid actions such as in coordinated power outages caused by large-scale user manipulation [159].\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"62.31.01#2","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Financial Impacts) ","risk_subcategory":"Deployment of GPAI agents in finance","description":"\"The deployment of GPAI based agents in the financial sector can negatively impact market stability due to correlated autonomous actions, high intercon- nectedness, or incentive misalignment [4]. Furthermore, such GPAI agents in the  same environment are vulnerable to classical challenges in multi-agent systems [63], such as coordination and security of the agents.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"62.31.02#1","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Societal Impacts) ","risk_subcategory":"Overreliance on AI system undermining user autonomy","description":"\"AI systems can undermine human autonomy, if they allow for habitually trusting the AI’s suggestions without sufficient exercising of human agency. Over time, a user may develop unjustified trust in or dependence on the system, or rely on its advice for tasks outside the system’s domain of expertise [205, 42]. In particular, less confident users (or users in emotional distress) can be more prone to “overtrust” a system [219].\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"62.31.03#1","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Societal Impacts) ","risk_subcategory":"Automatically generating disinformation at scale","description":"\"Disinformation (in various modalities: text, audio, images, video, etc.) can be generated with minimal human oversight and effort. Disinformation tools are relatively cheap and their technology is widely available. Such deployments can be particularly widespread in sensitive political contexts.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"62.31.03#2","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Financial Impacts) ","risk_subcategory":"Use of alternative financial data via AI","description":"\"Alternative financial data of a company is any data about the company not pro- duced by that company. Examples of such data that can benefit from improved collection and aggregation using AI models include stock discussions on social media, product reviews, and satellite imagery. The use of alternative financial data, enabled by the deployment of AI models, may introduce biases and generalization issues due to shorter shelf-life and vary- ing quality (e.g., shorter time series, smaller sample sizes, and dubious claims) due to its origins from various sources, posing financial tail risks (i.e.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"62.31.05","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Societal Impacts) ","risk_subcategory":"Generative AI use in political influence campaigns","description":"\"GPAI tools can be used in automation and scaling of influence campaigns [178]. Public opinion may be manipulated by targeted misleading or manipulative information. This can lead to rising political polarization and diminishing trust in public institutions.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"62.31.08","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Societal Impacts) ","risk_subcategory":"Multimodal deepfakes","description":"\"Deepfakes are media that depict real or non-existent people or events, involving the use of multiple modalities (e.g., images, audio, video). They can also involve the imitation of speech or body movements of real people. Multimodal deepfakes can be used to harass, discredit, intimidate, and extort individuals.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"62.31.09","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Societal Impacts) ","risk_subcategory":"Generation of personalized content for harassment, extortion, or intimidation","description":"\"GPAIs can be misused for the automated generation of content personalized to target select individuals based on their weak spots [30]. Such attacks may be more efficient and more successful in achieving the goals of harassment, extortion, or intimidation.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"62.31.10","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Societal Impacts) ","risk_subcategory":"Misuse for surveillance and population control","description":"\"AI tools can be misused by human or institutional actors for monitoring, control- ling, or suppressing individuals [178]. Massive data collection and automated analysis are often conducted, and AI tools can further exacerbate such practices.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"62.31.11","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Societal Impacts) ","risk_subcategory":"Systemic large-scale manipulation","description":"\"AI systems embedded with systemic biases can manipulate large population segments, particularly when these biases align with the beliefs or behaviors of the targeted group. When weaponized at scale, this manipulation can exacerbate social divisions or cause large-scale disruptions, such as city-wide blackouts (e.g., by the manipulation of power consumption into the peak demand period [159]).\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"62.31.12","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Societal Impacts) ","risk_subcategory":"Diminishing societal trust due to disinformation or manipulation","description":"\"The use of GPAIs may contribute to the proliferation of either deliberate dis- information or unintended misinformation can severely erode trust in public figures and democratic institutions. This diminishing trust can extend to other forms of media, making the public less informed.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"62.31.13","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Societal Impacts) ","risk_subcategory":"Personalized disinformation","description":"\"Automatic generation of disinformation can be personalized to target specific groups or individuals. Such attacks can be more effective in achieving their goals, and their costs can be significantly reduced when using GPAIs.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"62.31.14","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Societal Impacts) ","risk_subcategory":"GPAI assisted impersonation","description":"\"GPAI outputs are not always correctly detected as AI-generated across multiple modalities (text, images, audio, video). A malicious actor can use GPAI outputs directly when communicating, or use AI-informed details to help construct a convincing impersonation (e.g., forging of supporting documents). Even if future countermeasures prove potent enough to detect GPAI-generated content, the risk remains if the countermeasures are not well known, or difficult to access.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"62.32.00","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Category","risk_category":"Impacts of AI (Cyberattacks) ","risk_subcategory":null,"description":"- ","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"62.32.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Cyberattacks) ","risk_subcategory":"Automated discovery and exploitation of software systems","description":"\"GPAIs can be used to aid in the automated discovery of software vulnerabilities [33]. This can empower malicious actors, making their cyberattacks more effi- cient and potentially more damaging. This type of automation allows attackers to expand the scale of their operations at a low cost, increasing the impact of their actions. New malware can be developed automatically, or the known vulnerabilities can be exploited to create more sophisticated attacks.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"62.32.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Cyberattacks) ","risk_subcategory":"Amplification of cyberattacks","description":"\"General-purpose AI models may significantly enhance the magnitude and ef- fectiveness of cyberattacks, by amplifying existing capabilities or resources of malicious actors [3]. For example, GPAI models may be employed to: • Automatically scan open-source codebases and compiled binaries for po- tential vulnerabilities • Apply known exploits flexibly and at scale (e.g., identifying vulnerable computers based on subtle cues in response times or output formats) • Assist with different aspects of cyberattacks, including planning, recon- naissance, exploit searching, remote control, malware impleme","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"62.32.03","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Cyberattacks) ","risk_subcategory":"AI-driven spear phishing attacks","description":"\"Generative models can be misused to target individual users more efficiently by using personalized information [23]. Highly convincing automated fraudulent schemes can exploit the trust of victims by extracting sensitive data and making the deception more likely to succeed. For example, in LLMs, this misuse can be aided by jailbreaking techniques [178].\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"62.33.00","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Category","risk_category":"Impacts of AI (Weapons) ","risk_subcategory":null,"description":"- ","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"62.33.01","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Weapons) ","risk_subcategory":"Misuse of AI systems to assist in the creation of weapons","description":"\"AI systems may be misused to aid in the creation of weapons, such as chemical, biological, radiological, and nuclear (CBRN) weapons, or augment the abilities of existing weapons, such as providing autonomous capabilities to unmanned weapon systems. Current systems do not significantly aid a malicious actor in these tasks, but they do show early signs [117]. This risk can sometimes be mitigated with input and output filtering, but is still susceptible to adversarial techniques (such as jailbreaking or paraphrasing).\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"62.33.02","quick_ref":"Gipiškis2024","paper_title":"Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems","level":"Risk Sub-Category","risk_category":"Impacts of AI (Weapons) ","risk_subcategory":"Misuse of drug-discovery models","description":"\"Models used for drug discovery, such as drug-target affinity prediction models, can be used to identify or develop dangerous toxins. This is particularly concern- ing if the training data contains information related to potentially dangerous proteins and viruses.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"63.06.00","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Category","risk_category":"Selection Pressures","risk_subcategory":null,"description":"\"Selection pressures (Section 3.3): some aspects of training and selection by those deploying and using AI agents can lead to undesirable behaviour;\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.08.03","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Commitment and Trust ","risk_subcategory":"Rigidity and Mistaken Commitments","description":"\"Rigidity and Mistaken Commitments. Even when it is desirable to be able to make threats in order to deter socially harmful behaviour, doing so using AI agents effectively removes the human from the loop, which could prove disastrous in high-stakes contexts (e.g., a false positive in a nuclear sub- marine’s warning system; see also Case Study 11), or when irresponsible actors are enabled in making disproportionate or mistaken commitments.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"63.10.01","quick_ref":"Hammond2025","paper_title":"Multi-Agent Risks from Advanced AI ","level":"Risk Sub-Category","risk_category":"Multi-Agent Security ","risk_subcategory":"Swarm Attacks","description":"\"Swarm Attacks. The need for multi-agent security is foreshadowed by attacks today that benefit from the use of many decentralised agents, such as distributed denial-of-service attacks (Cisco, 2023; Yoachimik & Pacheco, 2024). Such attacks exploit the massive collective resources of individual low- resourced actors, chained into an attack that breaks the assumptions of bandwidth constraints on a single well-resourced agent.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":7,"subdomain":"7.6"},{"ev_id":"64.01.00","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Category","risk_category":"Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness) ","risk_subcategory":null,"description":"-","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"64.01.01","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness) ","risk_subcategory":"Impersonation ","description":"\"Assume the identity of a real person and take actions on their behalf\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"64.01.02","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness) ","risk_subcategory":"Appropriated Likeness","description":"\"Use or alter a person's likeness or other identifying features\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"64.01.03","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness) ","risk_subcategory":"Sockpuppeting ","description":"\"Create synthetic online personas or accounts\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"64.01.04","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness) ","risk_subcategory":"Non-consensual intimate imagery (NCII) ","description":"\"Create sexual explicit material using an adult person’s likeness\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"64.01.05","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics that exploit GenAI capabilities (Realistic depiction of human likeness) ","risk_subcategory":"Child sexual abuse material (CSAM) ","description":"\"Create child sexual explicit material\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"64.02.00","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Category","risk_category":"Misuse tactics that exploit GenAI capabilities (Realistic depictions of non-humans) ","risk_subcategory":null,"description":"-","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"64.02.01","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics that exploit GenAI capabilities (Realistic depictions of non-humans) ","risk_subcategory":"Falisification ","description":"\"Fabricate or falsely represent evidence, incl. reports, IDs, documents\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"64.02.02","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics that exploit GenAI capabilities (Realistic depictions of non-humans) ","risk_subcategory":"Intellectual Property (IP) Infringement ","description":"\"Use a person's IP without their permission\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.3"},{"ev_id":"64.02.03","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics that exploit GenAI capabilities (Realistic depictions of non-humans) ","risk_subcategory":"Counterfeit ","description":"\"Reproduce or imitate an original work, brand or style and pass as real\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"64.03.00","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Category","risk_category":"Misuse tactics that exploit GenAI capabilities (Use of generated content) ","risk_subcategory":null,"description":null,"entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"64.03.01","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics that exploit GenAI capabilities (Use of generated content) ","risk_subcategory":"Scaling and Amplification ","description":"\"Automate, amplify, or scale workflows\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"64.03.02","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics that exploit GenAI capabilities (Use of generated content) ","risk_subcategory":"Targeting & Personalisation ","description":"\"Refine outputs to target individuals with tailored attacks\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"64.04.00","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Category","risk_category":"Misuse tactics to compromise GenAI systems (Model integrity) ","risk_subcategory":null,"description":"-","entity":"Human","intent":"Intentional","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"64.04.01","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics to compromise GenAI systems (Model integrity) ","risk_subcategory":"Prompt injection ","description":"\"Prompt Injections are a form of Adversarial Input that involve manipulating the text instructions given to a GenAI system (Liu et al., 2023). Prompt Injections exploit loopholes in a model’s architec- tures that have no separation between system instructions and user data to produce a harmful output (Perez and Ribeiro, 2022). While researchers may use similar techniques to test the robustness of GenAI models, malicious actors can also leverage them. For example, they might flood a model with manipulative prompts to cause denial-of-service attacks or to bypass an AI detection software.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"64.04.02","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics to compromise GenAI systems (Model integrity) ","risk_subcategory":"Adversarial input ","description":"\"Adversarial Inputs involve modifying individual input data to cause a model to malfunction. These modifications, which are often imperceptible to humans, exploit how the model makes decisions to produce errors (Wallace et al., 2019) and can be applied to text, but also to images, audio, or video (e.g. changing pixels in an image of a panda in a way that causes a model to label it as a gibbon).6\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"64.04.03","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics to compromise GenAI systems (Model integrity) ","risk_subcategory":"Jailbreaking ","description":"\"Jailbreaking aims to bypass or remove restrictions and safety filters placed on a GenAI model completely (Chao et al., 2023; Shen et al., 2023). This gives the actor free rein to generate any output, regardless of its content being harmful, biassed, or offensive. All three of these are tactics that manipulate the model into producing harmful outputs against its design. The difference is that prompt injections and adversarial inputs usually seek to steer the model towards producing harmful or incorrect outputs from one query, whereas jailbreaking seeks to dismantle a model’s safety mechanisms ","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"64.04.04","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics to compromise GenAI systems (Model integrity) ","risk_subcategory":"Model diversion ","description":"\"Model Diversion takes model manipulation one step further, by repurposing (often open-source) generative AI models in a way that diverts them from their intended functionality or from the use cases envisioned by their developers (Lin et al., 2024). An example of this is training the BERT open source model on the DarkWeb to create DarkBert.7\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"64.04.05","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics to compromise GenAI systems (Model integrity) ","risk_subcategory":"Model extraction ","description":"\"Data Exfiltration goes beyond revealing private information, and involves illicitly obtaining the training data used to build a model that may be sensitive or proprietary. Model Extraction is the same attack, only directed at the model instead of the training data — it involves obtaining the architecture, parameters, or hyper-parameters of a proprietary model (Carlini et al., 2024).\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"64.04.06","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics to compromise GenAI systems (Model integrity) ","risk_subcategory":"Steganography ","description":"\"Steganography is the practice of hiding coded messages in GenAI model outputs, which may allow malicious actors to communicate covertly.8\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"64.04.07","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics to compromise GenAI systems (Model integrity) ","risk_subcategory":"Poisoning ","description":"\"Data Poisoning involves deliberately corrupting a model’s training dataset to introduce vulnerabilities, derail its learning process, or cause it to make incorrect predictions (Carlini et al., 2023). For example, the tool Nightshade is a data poisoning tool, which allows artists to add invisible changes to the pixels in their art before uploading online, to break any models that use it for training.9 Such attacks exploit the fact that most GenAI models are trained on publicly available datasets like images and videos scraped from the web, which malicious actors can easily compromise.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"64.05.00","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Category","risk_category":"Misuse tactics to compromise GenAI systems (Data integrity) ","risk_subcategory":null,"description":"-","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"64.05.01","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics to compromise GenAI systems (Data integrity) ","risk_subcategory":"Privacy compromise ","description":"\"Privacy Compromise attacks reveal sensitive or private information that was used to train a model. For example, personally identifiable information or medical records.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"64.05.02","quick_ref":"Marchal2024","paper_title":"Generative AI Misuse: A Taxonomy of Tactics and Insights from Real-World Data","level":"Risk Sub-Category","risk_category":"Misuse tactics to compromise GenAI systems (Data integrity) ","risk_subcategory":"Data exfiltration ","description":"\"Data Exfiltration goes beyond revealing private information, and involves illicitly obtaining the training data used to build a model that may be sensitive or proprietary. Model Extraction is the same attack, only directed at the model instead of the training data — it involves obtaining the architecture, parameters, or hyper-parameters of a proprietary model (Carlini et al., 2024).\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"65.01.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Transparency) ","risk_subcategory":"Lack of training data transparency ","description":"\"Without accurate documentation on how a model's data was collected, curated, and used to train a model, it might be harder to satisfactorily explain the behavior of the model with respect to the data.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"65.01.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Transparency) ","risk_subcategory":"Uncertain data provenance ","description":"\"Data provenance refers to tracing history of data, which includes its ownership, origin, and transformations. Without standardized and established methods for verifying where the data came from, there are no guarantees that the data is the same as the original source and has the correct usage terms.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"65.02.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Data laws) ","risk_subcategory":"Data usage restrictions ","description":"\"Laws and other restrictions can limit or prohibit the use of some data for specific AI use cases.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.02.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Data laws) ","risk_subcategory":"Data acquisition restrictions ","description":"\"Laws and other regulations might limit the collection of certain types of data for specific AI use cases.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.02.03","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Data laws) ","risk_subcategory":"Data transfer restrictions ","description":"\"Laws and other restrictions can limit or prohibit transferring data.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.04.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Fairness) ","risk_subcategory":"Data bias","description":"\"Historical and societal biases that are present in the data are used to train and fine-tune the model.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.1"},{"ev_id":"65.05.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Intellectual property) ","risk_subcategory":"Confidential information in data ","description":"\"Confidential information might be included as part of the data that is used to train or tune the model.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"65.06.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Accuracy) ","risk_subcategory":"Data contamination ","description":"\"Data contamination occurs when incorrect data is used for training. For example, data that is not aligned with model’s purpose or data that is already set aside for other development tasks such as testing and evaluation.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.07.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Value alignment) ","risk_subcategory":"Improper retraining ","description":"\"Using undesirable output (for example, inaccurate, inappropriate, and user content) for retraining purposes can result in unexpected model behavior.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.07.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Value alignment) ","risk_subcategory":"Improper data curation ","description":"\"Improper collection and preparation of training or tuning data includes data label errors and by using data with conflicting information or misinformation.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.08.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Training Data Risks (Robustness) ","risk_subcategory":"Data poisoning ","description":"\"A type of adversarial attack where an adversary or malicious insider injects intentionally corrupted, false, misleading, or incorrect samples into the training or fine-tuning datasets.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"65.09.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Inference risks (Robustness) ","risk_subcategory":"Prompt injection attack ","description":"\"A prompt injection attack forces a generative model that takes a prompt as input to produce unexpected output by manipulating the structure, instructions, or information contained in its prompt.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"65.09.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Inference risks (Robustness) ","risk_subcategory":"Extraction attack ","description":"\"An attribute inference attack is used to detect whether certain sensitive features can be inferred about individuals who participated in training a model. These attacks occur when an adversary has some prior knowledge about the training data and uses that knowledge to infer the sensitive data.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"65.09.03","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Inference risks (Robustness) ","risk_subcategory":"Evasion attack ","description":"\"Evasion attacks attempt to make a model output incorrect results by slightly perturbing the input data that is sent to the trained model.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"65.09.04","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Inference risks (Robustness) ","risk_subcategory":"Prompt leaking ","description":"\"A prompt leak attack attempts to extract a model's system prompt (also known as the system message).\"","entity":"Human","intent":"Intentional","timing":"Other","domain":2,"subdomain":"2.2"},{"ev_id":"65.10.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Inference risks (Multi-category) ","risk_subcategory":"Jailbreaking ","description":"\"A jailbreaking attack attempts to break through the guardrails that are established in the model to perform restricted actions.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"65.10.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Inference risks (Multi-category) ","risk_subcategory":"Prompt priming ","description":"\"Because generative models tend to produce output like the input provided, the model can be prompted to reveal specific kinds of information. For example, adding personal information in the prompt increases its likelihood of generating similar kinds of personal information in its output. If personal data was included as part of the model’s training, there is a possibility it could be revealed.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"65.11.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Inference risks (Privacy) ","risk_subcategory":"Membership inference attack ","description":"\"A membership inference attack repeatedly queries a model to determine whether a given input was part of the model’s training. More specifically, given a trained model and a data sample, an attacker samples the input space, observing outputs to deduce whether that sample was part of the model's training.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"65.11.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Inference risks (Privacy) ","risk_subcategory":"Attribute inference attack ","description":"\"An attribute inference attack repeatedly queries a model to detect whether certain sensitive features can be inferred about individuals who participated in training a model. These attacks occur when an adversary has some prior knowledge about the training data and uses that knowledge to infer the sensitive data.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"65.13.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Inference risks (Accuracy) ","risk_subcategory":"Poor model accuracy ","description":"\"Poor model accuracy occurs when a model’s performance is insufficient to the task it was designed for. Low accuracy might occur if the model is not correctly engineered, or there are changes to the model’s expected inputs.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"65.14.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (misuse) ","risk_subcategory":"Non-disclosure ","description":"\"Content might not be clearly disclosed as AI generated.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.4"},{"ev_id":"65.14.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (misuse) ","risk_subcategory":"Improper usage ","description":"\"Improper usage occurs when a model is used for a purpose that it was not originally designed for.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"65.14.03","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (misuse) ","risk_subcategory":"Spreading toxicity","description":"\"Generative AI models might be used intentionally to generate hateful, abusive, and profane (HAP) or obscene content.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"65.14.04","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (misuse) ","risk_subcategory":"Dangerous use","description":"\"Generative AI models might be used with the sole intention of harming people.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"65.14.05","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (misuse) ","risk_subcategory":"Nonconsensual use","description":"\"Generative AI models might be intentionally used to imitate people through deepfakes by using video, images, audio, or other modalities without their consent.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"65.14.06","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (misuse) ","risk_subcategory":"Spreading disinformation ","description":"\"Generative AI models might be used to intentionally create misleading or false information to deceive or influence a targeted audience.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"65.15.03","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Output risks (Value alignment)","risk_subcategory":"Over- or under-reliance ","description":"\"In AI-assisted decision-making tasks, reliance measures how much a person trusts (and potentially acts on) a model’s output. Over-reliance occurs when a person puts too much trust in a model, accepting a model’s output when the model’s output is likely incorrect. Under-reliance is the opposite, where the person doesn’t trust the model but should.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"65.22.01","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Governance)","risk_subcategory":"Lack of system transparency ","description":"\"Insufficient documentation of the system that uses the model and the model’s purpose within the system in which it is used.\"","entity":"Human","intent":"Other","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"65.22.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Governance)","risk_subcategory":"Unrepresentative risk testing ","description":"\"Testing is unrepresentative when the test inputs are mismatched with the inputs that are expected during deployment.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"65.22.03","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Governance)","risk_subcategory":"Incomplete usage definition ","description":"\"Since foundation models can be used for many purposes, a model’s intended use is important for defining the relevant risks of that model. As the use changes, the relevant risks might correspondingly change.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"65.22.04","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Governance)","risk_subcategory":"Lack of data transparency ","description":"\"Lack of data transparency is due to insufficient documentation of training or tuning dataset details. \"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"65.22.05","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Governance)","risk_subcategory":"Incorrect risk testing ","description":"\"A metric selected to measure or track a risk is incorrectly selected, incompletely measuring the risk, or measuring the wrong risk for the given context.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"65.22.06","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Governance)","risk_subcategory":"Lack of model transparency ","description":"\"Lack of model transparency is due to insufficient documentation of the model design, development, and evaluation process and the absence of insights into the inner workings of the model.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.4"},{"ev_id":"65.22.07","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Governance)","risk_subcategory":"Lack of testing diversity ","description":"\"AI model risks are socio-technical, so their testing needs input from a broad set of disciplines and diverse testing practices.\"","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"65.23.02","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Societal impact)","risk_subcategory":"Impact on education: plagiarism ","description":"\"Easy access to high-quality generative models might result in students that use AI models to plagiarize existing work intentionally or unintentionally.\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"65.23.03","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Societal impact)","risk_subcategory":"Impact on Jobs ","description":"\"Widespread adoption of foundation model-based AI systems might lead to people's job loss as their work is automated if they are not reskilled.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"65.23.04","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Societal impact)","risk_subcategory":"Impact on affected communities ","description":"\"It is important to include the perspectives or concerns of communities that are affected by model outcomes when designing and building models. Failing to include these perspectives makes it difficult to understand the relevant context for the model and to engender trust within these communities.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.3"},{"ev_id":"65.23.05","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Societal impact)","risk_subcategory":"Impact on education: bypassing learning ","description":"\"Easy access to high-quality generative models might result in students that use AI models to bypass the learning process.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"65.23.07","quick_ref":"IBM2025","paper_title":"AI Risk Atlas ","level":"Risk Sub-Category","risk_category":"Non-technical risks (Societal impact)","risk_subcategory":"Human exploitation ","description":"\"When workers who train AI models such as ghost workers are not provided with adequate working conditions, fair compensation, and good health care benefits that also include mental health.\"","entity":"Human","intent":"Other","timing":"Pre-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"66.01.01","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Autonomy","risk_subcategory":"Impersonation / identity theft","description":"\"Theft of an individual, group or organisation’s identity by a third-party in order to defraud, mock or otherwise harm them or another party\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"66.01.02","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Autonomy","risk_subcategory":"IP / copyright / personality / rights loss","description":"\"Misuse or abuse of an individual or organisation’s intellectual property, including copyright, trademarks, and patents. & Loss of or restrictions to the rights of an individual to control the commercial use of their identity, such as name, image, likeness, or other unequivocal identifiers\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"66.02.01","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Political and Economic","risk_subcategory":"Political instability","description":"\"Political unrest caused directly or indirectly by the use or misuse of a technology system\"","entity":"Human","intent":"Other","timing":"Other","domain":6,"subdomain":"6.0"},{"ev_id":"66.02.02","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Political and Economic","risk_subcategory":"Institutional trust loss","description":"\"Erosion of trust in public institutions and weakened checks and balances due to mis/disinformation, influence operations, or real or perceived misuse of generative AI\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"66.04.05","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Societal and Cultural","risk_subcategory":"Cheating / plagiarism","description":"\"Use of generative AI in an academic setting to either cheat or plagiarize\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"66.04.07","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Societal and Cultural","risk_subcategory":"Labor exploitation","description":"\"Use/misuse of labour to help train, develop, manage or optimise a technology system or set of systems, including under-paid and/or offshore\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"66.05.02","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Reputational","risk_subcategory":"Defamation / libel / slander","description":"\"Use of a technology system to create, facilitate or amplify false perception(s) about an individual, group or organisation\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"66.07.01","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Psychological","risk_subcategory":"Sexualization","description":"\"The non-consensual sexualisation of an individual or group using a technology or application\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"66.07.04","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Psychological","risk_subcategory":"Coercion / manipulation","description":"\"Use of a technology system to covertly alter user beliefs and behaviour using nudging, dark patterns and/or other opaque techniques\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"66.07.05","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Psychological","risk_subcategory":"Over-reliance","description":"\"Unfettered and/or obsessive belief in the accuracy or other quality of a technology system, resulting in complacency, lack of critical thinking and other actual or potential negative impacts\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"66.09.04","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Privacy and Security","risk_subcategory":"Secondary use","description":"\"The use of personal data collected for one purpose for a diferent purpose without end-user consent; AI exacerbates secondary use risks by creating new AI capabilities with collected personal data, and (re)creating models from a public dataset.\"","entity":"Human","intent":"Intentional","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"66.09.07","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Privacy and Security","risk_subcategory":"Insecurity","description":"\"carelessness in protecting collected personal data from leaks and improper access due to faulty data storage and data practices\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"66.10.01","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Human Rights and Civil Liberties","risk_subcategory":"Erosion of due process","description":"\"Restrictions to or loss of liberty as a result of use or misuse of a generative AI in a legal process\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"66.11.00","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Category","risk_category":"Physical","risk_subcategory":"-","description":"\"Physical injury to an individual or group, or damage to physical property due to the use of misuse of a technology system or set of systems\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"66.11.01","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Physical","risk_subcategory":"Loss of life","description":"\"Accidental or deliberate loss of life, including suicide, extinction or cessation, due to the use or misuse of a technology system\"","entity":"Human","intent":"Other","timing":"Post-deployment","domain":null,"subdomain":null},{"ev_id":"66.11.03","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Sub-Category","risk_category":"Physical","risk_subcategory":"Self-harm","description":"\"A person who deliberately damages their own body as a direct or indirect result of using a technology system\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"66.12.00","quick_ref":"Li2025","paper_title":"A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents","level":"Risk Category","risk_category":"Environment","risk_subcategory":"-","description":"\"Damage to the environment caused by the use or misuse of a technology system or set of systems\"","entity":"Human","intent":"Other","timing":"Other","domain":null,"subdomain":null},{"ev_id":"67.03.00","quick_ref":"DSIT2023","paper_title":"Capabilities and Risks from Frontier AI","level":"Risk Category","risk_category":"Misuse risks","risk_subcategory":null,"description":"\"Frontier AI may help bad actors to perform cyberattacks, run disinformation campaigns and design biological or chemical weapons. Frontier AI will almost certainly continue to lower the barriers to entry for less sophisticated threat actors.192 We focus here on only a few important misuse risks, but this is not to downplay the importance of others.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"67.03.01","quick_ref":"DSIT2023","paper_title":"Capabilities and Risks from Frontier AI","level":"Risk Sub-Category","risk_category":"Misuse risks","risk_subcategory":"Dual Use Science risks","description":"\"Frontier AI systems have the potential to accelerate advances in the life sciences, from training new scientists to enabling faster scientific workflows. While these capabilities will have tremendous beneficial applications, there is a risk that they can be used for malicious purposes, such as for the development of biological or chemical weapons.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"67.03.02","quick_ref":"DSIT2023","paper_title":"Capabilities and Risks from Frontier AI","level":"Risk Sub-Category","risk_category":"Misuse risks","risk_subcategory":"Cyber ","description":"\"As the programming abilities of AI systems continue to expand, frontier AI is likely to significantly exacerbate existing cyber risks. Most notably, AI systems can be used by potentially anyone to create faster paced, more effective and larger scale cyber intrusion via tailored phishing methods or replicating malware. Frontier AI’s effect on the overall balance between cyber offence and defence is uncertain, as these tools also have many applications in improving the cybersecurity of systems and defenders are mobilising significant resources to utilise frontier AI for defensive purposes.209 I","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"67.03.03","quick_ref":"DSIT2023","paper_title":"Capabilities and Risks from Frontier AI","level":"Risk Sub-Category","risk_category":"Misuse risks","risk_subcategory":"Disinformation and Influence Operations","description":"\"In addition to unintentional degradation of the information environment (discussed in the section on Societal Harms above), frontier AI can be misused to deliberately spread false information to create disruption, persuade people on political issues, or cause other forms of harm or damage.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"67.04.01","quick_ref":"DSIT2023","paper_title":"Capabilities and Risks from Frontier AI","level":"Risk Sub-Category","risk_category":"Loss of control ","risk_subcategory":"Humans might increasingly hand over control to misaligned AI systems","description":"\"Organisations around the world are already deploying misaligned AI systems that are causing harm in unexpected ways.250 Recommendation algorithms increase the consumption of extremist content.251 Medical algorithms have been known to misdiagnose US patients,252 and recommend incorrect prescriptions.253 Still, we hand over more control to them, often because they are still as - or more - effective than human decision making, or because they are cheaper.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":5,"subdomain":"5.2"},{"ev_id":"68.01.00","quick_ref":"Chin2025","paper_title":"Dimensional Characterization and Pathway Modeling for Catastrophic AI Risks","level":"Risk Category","risk_category":"CBRN ","risk_subcategory":null,"description":"\"Chemical, biological, radiological, and nuclear (CBRN) risks are broad classes of threats that have the potential to cause harm to a large number of people. Explosives are also sometimes included in this category, often referred to as CBRNE...The key characteristic of CBRN risk is that it stems from misuse of capable models with a direct pathway to harm, where a malicious actor is able to carry out consequential attacks more efficiently and effectively with the help of AI.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"70.01.01","quick_ref":"Perlo2025","paper_title":"Embodied AI: Emerging Risks and Opportunities for Policy Action","level":"Risk Sub-Category","risk_category":"Physical Risks ","risk_subcategory":"Purposeful or malicious harm","description":"\"EAI systems present distinct physical risks due to their embodiment in the physical world. EAI technologies have already been designed and deployed with lethal intent, such as AI-controlled drones [52, 53]. However, fully autonomous military robots, often integrated with bespoke AI architectures [54, 55], are not yet widely used in combat. While highly or fully autonomous warfare is distinctly possible in the future [56], immediate risks arise from commercially available EAI systems, including AI-controlled quadrupeds and autonomous driving assistants.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"71.01.01","quick_ref":"Tang2025","paper_title":"Risks of AI Scientists: Prioritizing Safeguarding Over Autonomy","level":"Risk Sub-Category","risk_category":"Scientific Domain of Agents","risk_subcategory":"Chemical Risks ","description":"\"Chemical risks involve the exploitation of agents to synthesize chemical weapons, as well as the creation or release of hazardous substances during autonomous chemical experiments. This category also includes the risks arising from the use of advanced materials, such as nanomaterials, which may have unknown or unpredictable chemical properties.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"71.02.01","quick_ref":"Tang2025","paper_title":"Risks of AI Scientists: Prioritizing Safeguarding Over Autonomy","level":"Risk Sub-Category","risk_category":"User Intent ","risk_subcategory":"Malicious and Direct ","description":"\"Directly harmful objective\"","entity":"Human","intent":"Intentional","timing":"Other","domain":4,"subdomain":"4.0"},{"ev_id":"72.01.00","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Category","risk_category":"Misuse Risks ","risk_subcategory":null,"description":"\"Risks arising from intentional exploitation of AI model capabilities by malicious actors to cause harm to individuals, organisations, or society.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"72.01.01","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Misuse Risks ","risk_subcategory":"Cyber Offense Risks","description":"\"AI-enabled cyber offense poses a significant cyber domain security risk by fundamentally transforming the scale, sophistication, and accessibility of cyber-attacks. Unlike traditional cyber threats, AI enables both the automation of existing attack vectors and the creation of entirely new categories of offensive capabilities that can adapt and evolve in real-time. AI can automate and enhance cyber-attacks, including vulnerability discovery and exploitation, password cracking, malicious code generation, sophisticated phishing, network scanning, and social engineering. This could dramatically l","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"72.01.02","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Misuse Risks ","risk_subcategory":"Biological and Chemical Risks","description":"\"The dual-use nature of AI technology presents a critical risk by significantly lowering technical thresholds for malicious non-state actors to design, synthesize, acquire, and deploy CBRNE (Chemical, Biological, Radiological, Nuclear, and Explosive) weapons. This capability poses unprecedented challenges to national security, international non-proliferation regimes, and global security governance.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"72.01.03","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Misuse Risks ","risk_subcategory":"Physical Harm and Injury Risks","description":"\"The integration of general-purpose AI models into embodied systems creates direct physical threats through malicious exploitation of autonomous decision-making capabilities in real-world environments. The risk lies in embodied models' capacity for autonomous action and real-world interaction, and when these capabilities are maliciously exploited they may trigger a series of serious consequences.18\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"72.01.04","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Misuse Risks ","risk_subcategory":"Large-Scale Persuasion and Harmful Manipulation Risks","description":"\"AI systems can be gravely misused to distort public perception and compromise social stability through the generation of synthetic content (e.g., deepfakes, sophisticated fake news) and the strategic manipulation of digital platforms with large user bases to disseminate or precisely target misleading information or ideologies.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"72.03.00","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Category","risk_category":"Accident Risks ","risk_subcategory":null,"description":"\"Risks arising from operational failures, model misjudgments, or improper human operation of AI systems deployed in safety-critical infrastructure, where single points of failure can trigger cascading catastrophic consequences.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"72.04.02","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Market Concentration and Infrastructure Dependencies:","description":"\"Over-reliance on a limited number of dominant AI providers could create critical single points of failure across essential services. Market concentration in AI development may lead to scenarios where technical failures, cyber-attacks, or policy decisions by a few companies could simultaneously disrupt healthcare systems, financial services, transportation networks, and communication infrastructure, creating cascading failures across interconnected critical systems.\"","entity":"Human","intent":"Unintentional","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"72.04.04","quick_ref":"Tse2025","paper_title":"Frontier AI Risk Management Framework (v1.0)","level":"Risk Sub-Category","risk_category":"Systemic Risks ","risk_subcategory":"Social Cohesion and Equity Disruption:","description":"\"Systemic deployment of biased AI systems could exacerbate existing social discrimination and prejudice at unprecedented scales, while unequal access to advanced AI capabilities may widen socioeconomic disparities and create new forms of social stratification that challenge traditional social order.\"","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"73.01.05","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Agentic LLMs Pose Novel Risks ","risk_subcategory":"Safety Risks from Affordances Provided to LLM-agents","description":"\"The capabilities of LLM-agents can be enhanced in significant ways by providing the LLM-agent with novel affordances, e.g. the ability to browse the web (Nakano et al., 2021), to manipulate objects in the physical world (Ahn et al., 2022; Huang et al., 2022a), to create and instruct copies of itself (Richards, 2023), to create and use new tools (Wang et al., 2023a), etc. Affordances can create additional risks, as they often increase the impact area of the language-agent, and they amplify the consequences of an agent’s failures and enable novel forms of failure modes (Ruan et al., 2023; Pan e","entity":"Human","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.2"},{"ev_id":"73.03.00","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Category","risk_category":"Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs","risk_subcategory":null,"description":"\"Like all technologies, LLMs have the possibility for misuse by malicious actors. Malicious use of dual- use capabilities of AI is a recurring concern within literature (Brundage et al., 2018; Hendrycks et al., 2023; Mozes et al., 2023)\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"73.03.01","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs","risk_subcategory":"Misinformation and Manipulation","description":"\"Recent studies have demonstrated that LLMs can be exploited to craft deceptive narratives with levels of persuasiveness similar to human-generated content (Pan et al., 2023b; Spitale et al., 2023), to fabri- cate fake news (Zellers et al., 2019; Zhou et al., 2023f), and to devise automated influence operations aimed at manipulating the perspectives of targeted audiences (Goldstein et al., 2023). LLMs have also been found to be used in malicious social botnets (Yang and Menczer, 2023), powering automated accounts used to disseminate coordinated messages. More broadly, the use of LLMs for the d","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"73.03.02","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs","risk_subcategory":"Cybersecurity","description":"\"LLMs may exacerbate cybersecurity risks in various ways (Newman, 2024). Firstly, LLMs may significantly amplify the effectiveness of deceptive operations aimed at tricking people into disclosing sensitive information or granting adversary access to critical resources. For example, LLMs might prove highly effective at crafting personalized phishing emails or messages at scale that may be harder for an average user to recognize as phishing attempts (Karanjai, 2022; Hazell, 2023). In addition to being directly harmful to the targeted individual, such ‘social engineering’ attacks are often the ba","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"73.03.03","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs","risk_subcategory":"Surveillance and Censorship","description":"\"Content moderation has emerged as one of the key use-cases of LLMs (Weng et al., 2023), indicating the potential of LLMs for surveillance and censorship as well (Edwards, 2023). Surveillance and censorship are one of the primary tools employed by governments with dictatorial tendencies to suppress opposing political and social voices. These censorship measures, however, are often quite crude and can be escaped with little ingenuity...However, LLMs could enable significantly more sophisticated surveillance and censorship operations at scale (Feldstein, 2019). Multimodal-LLMs or LLMs combined w","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"73.03.04","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs","risk_subcategory":"Warfare and Physical Harm","description":"\"The use of AI in warfare is highly alarming and may pose dangers to human safety (Hendrycks et al., 2023). Autonomous drone warfare is being aggressively pursued as a tactic in the current war in Ukraine (Meaker, 2023), and may already have been used on human targets (Hambling, 2023). The use of AI- based facial recognition has been documented in the targeting of Palestinians in Gaza (International, 2023). LLMs have already been productized in limited ways for the purposes of warfare planning (Tarantola, 2023). Furthermore, active research is being carried out to develop multimodal-LLMs that ","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"73.03.05","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs","risk_subcategory":"Hazardous Biological and Chemical Technologies","description":"\"AI systems such as LLMs, chemical LLMs (Skinnider et al., 2021; Moret et al., 2023), and other LLM- based biological design tools might soon facilitate the production of bioweapons, chemical weapons, and other hazardous technologies. In particular, LLMs might enable actors with less expertise to more easily synthesize dangerous pathogens, while customized chemical and biological design tools might be more concerning in terms of expanding the capabilities of sophisticated actors (e.g. states) (Sandbrink, 2023). Gopal et al. (2023) and Soice et al. (2023) demonstrated that people with little ba","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"73.03.06","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs","risk_subcategory":"Domain-Specific Misuses","description":"\"Improvements in LLMs may exert greater pressure to apply LLMs to various domains, such as health and education (Eloundou et al., 2023). Crude efforts to use LLMs in such domains, however, may incur harm and should be discouraged strongly. In particular, it is important to guard against different ways in which LLMs may be misused within any domain. One famous episode of misuse within the health sector is a mental health non-profit experimenting LLM-based therapy on its users without their informed consent (Xiang, 2023a). Within the education sector, LLMs may be misused in various ways that mig","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"73.04.02","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"LLM-Systems Can Be Untrustworthy","risk_subcategory":"Inconsistent Performance across and within Domains","description":"\"Estimating true capabilities of an LLM is a difficult task (c.f. Section 3.3), especially for naive users unfamiliar with the brittle nature of machine learning technologies. Exaggeration of model capabilities by the developers (Lambert, 2023; Blair-Stanek et al., 2023), and issues such as task-contamination (Roberts et al., 2023b), underrepresentation of tasks or domains (Wu et al., 2023a; McCoy et al., 2023), and prompt-sensitivity (Anthropic, 2023d) may cause a user to misestimate the true capabilities of a model. This lack of reliability can undermine user trust or cause harm if a user ba","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"73.04.03","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"LLM-Systems Can Be Untrustworthy","risk_subcategory":"Overreliance","description":"\"If a user begins to excessively trust an LLM, this may cause them to develop an overreliance on the LLM. Overreliance can result in automation bias (Kupfer et al., 2023), and can cause errors of omission (user choosing not to verify the validity of a response) and errors of commission (user believing and acting on the basis of the LLM’s response, even if it contradicts their own knowledge) (Skitka et al., 1999). It can be particularly dangerous in domains where the user may lack relevant expertise to robustly scrutinize the LLM responses. This is particularly a source of risk for LLMs because","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"73.07.02","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Jailbreaks and Prompt Injections Threaten Security of LLMs","risk_subcategory":"“Model Psychology” Attacks","description":"\"LLMs are vulnerable to “psychological” tricks (Li et al., 2023e; Shen et al., 2023), which can be exploited by attackers. Examples include instructing the model to behave like a specific persona (Shah et al., 2023; Andreas, 2022), or employing various “social engineering” tricks crafted by humans (Wei et al., 2023c) or other LLMs (Perez et al., 2022b; Casper et al., 2023c).\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"73.07.04","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Sub-Category","risk_category":"Jailbreaks and Prompt Injections Threaten Security of LLMs","risk_subcategory":"Attacking LLMs via Additional Modalities a","description":"\"LLMs can now process modalities other than text, e.g. images or video frames (OpenAI, 2023c; Gemini Team, 2023). Several studies show that gradient-based attacks on multimodal models are easy and effective (Carlini et al., 2023a; Bailey et al., 2023; Qi et al., 2023b). These attacks manipulate images that are input to the model (via an appropriate encoding). GPT-4Vision (OpenAI, 2023c) is vulnerable to jailbreaks and exfiltration attacks through much simpler means as well, e.g. writing jailbreaking text in the image (Willison, 2023a; Gong et al., 2023). For indirect prompt injection, the atta","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"73.08.00","quick_ref":"Anwar2024","paper_title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","level":"Risk Category","risk_category":"Vulnerability to Poisoning and Backdoors","risk_subcategory":null,"description":"\"The previous section explored jailbreaks and other forms of adversarial prompts as ways to elicit harmful capabilities acquired during pretraining. These methods make no assumptions about the training data. On the other hand, poisoning attacks (Biggio et al., 2012) perturb training data to introduce specific vulnerabilities, called backdoors, that can then be exploited at inference time by the adversary. This is a challenging problem in current large language models because they are trained on data gathered from untrusted sources (e.g. internet), which can easily be poisoned by an adversary (","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"74.01.01","quick_ref":"Wang2025","paper_title":"A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy","level":"Risk Sub-Category","risk_category":"Inherent Risk ","risk_subcategory":"Privacy - Membership Inference Attack (MIA)","description":"\"inferring whether a given text record is used for training LLM\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"74.01.02","quick_ref":"Wang2025","paper_title":"A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy","level":"Risk Sub-Category","risk_category":"Inherent Risk ","risk_subcategory":"Privacy - Data Extraction Attack (DEA)","description":"\"extracting the text records that exist in the training dataset\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"74.01.03","quick_ref":"Wang2025","paper_title":"A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy","level":"Risk Sub-Category","risk_category":"Inherent Risk ","risk_subcategory":"Privacy -  Prompt Inversion Attack (PIA)","description":"\"stealing the private prompting texts\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"74.01.04","quick_ref":"Wang2025","paper_title":"A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy","level":"Risk Sub-Category","risk_category":"Inherent Risk ","risk_subcategory":"Privacy - Attribute Inference Attack (AIA)","description":"\"deducing the private or sensitive information from training texts, prompting texts or external texts\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"74.01.05","quick_ref":"Wang2025","paper_title":"A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy","level":"Risk Sub-Category","risk_category":"Inherent Risk ","risk_subcategory":"Privacy - Model Extraction Attack (MEA)","description":"\"replicating the parameters of the LLM,\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"74.02.02","quick_ref":"Wang2025","paper_title":"A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy","level":"Risk Sub-Category","risk_category":"Malicious Use ","risk_subcategory":"Jailbreak in LLM Malicious Use - Poisoning Training Data ","description":"\"In the data collecting and pre-training phase, malicious adversaries can Jailbreak LLMs through poisoning their training data to make the model to output harmful content.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"74.02.03","quick_ref":"Wang2025","paper_title":"A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy","level":"Risk Sub-Category","risk_category":"Malicious Use ","risk_subcategory":"Jailbreak in LLM Malicious Use - Backdoor Attack ","description":"\"However, there are still ones who can leave holes in the training dataset, making LLMs appear safe on average, but generate harmful content under other specific conditions. This kind of attack can be categorized as \"backdoor attack\". Evan et al. developed a backdoor model that behaves as expected when trained, but exhibits different and potentially harmful behavior when deployed [81]. The results show that these backdoor behaviors persist even after multiple security training techniques are applied.\"","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"74.02.04","quick_ref":"Wang2025","paper_title":"A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy","level":"Risk Sub-Category","risk_category":"Malicious Use ","risk_subcategory":"Jailbreak in LLM Malicious Use - White & Black Box Attacks ","description":"\"In the fine-tuning and alignment phase, elaborately- designed instruction datasets can be utilized to fine-tune LLMs to drive them to perform undesirable behaviors, such as generating harmful information or content that violates ethical norms, and thus achieve a jailbreak. Based on the accessibility to the model parameters, we can categorize them into white-box and black-box attacks. For white-box attacks, we can jailbreak the model by modifying its parameter weights. In [107], Lermen et al. used LoRA to fine-tune the Llama2’s 7B, 13B, and 70B as well as Mixtral on AdvBench and RefusalBench d","entity":"Human","intent":"Intentional","timing":"Pre-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"74.02.05","quick_ref":"Wang2025","paper_title":"A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy","level":"Risk Sub-Category","risk_category":"Malicious Use ","risk_subcategory":"Jailbreak in LLM Malicious Use - Prompt Attacks ","description":"\"In the prompting and reasoning phase, dialog can push LLMs into confused or overly compliant states, raising the risk of producing harmful outputs when confronted with harmful questions. Most of the jailbreak methods in this phase are black-boxed and can be categorized into four main groups based on the type of method: Prompt Injection [154], Role Play, Adversarial Prompting, and Prompt Form Transformation.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"}]}