{"attribution":{"source":"MIT AI Risk Repository, Domain Taxonomy of AI Risks v1 (MIT AI Risk Initiative)","license":"CC BY 4.0","license_url":"https://creativecommons.org/licenses/by/4.0/","citation":"Slattery, P., Saeri, A. K., Grundy, E. A. C., Graham, J., Noetel, M., Uuk, R., Dao, J., Pour, S., Casper, S., & Thompson, N. (2025). The AI Risk Repository: A comprehensive meta-review, database, and taxonomy of risks from artificial intelligence. arXiv:2408.12622."},"exported_at":"2026-09-11"}
{"rows":[{"ev_id":"24.01.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Capability failures","risk_subcategory":null,"description":"\"One reason AI systems fail is because they lack the capability or skill needed to do what they are asked to do.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"24.01.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Capability failures","risk_subcategory":"Lack of capability for task","description":"\"As we have seen, this could be due to the skill not being required during the training process (perhaps due to issues with the training data) or because the learnt skill was quite brittle and was not generalisable to a new situation (lack of robustness to distributional shift). In particular, advanced AI assistants may not have the capability to represent complex concepts that are pertinent to their own ethical impact, for example the concept of 'benefitting the user' or 'when the user asks' or representing 'the way in which a user expects to be benefitted'.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"24.01.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Capability failures","risk_subcategory":"Difficult to develop metrics for evaluating benefits or harms caused by AI assistants","description":"\"Another difficulty facing AI assistant systems is that it is challenging to develop metrics for evaluating particular aspects of benefits or harms caused by the assistant – especially in a sufficiently expansive sense, which could involve much of society (see Chapter 19). Having these metrics is useful both for assessing the risk of harm from the system and for using the metric as a training signal.\"","entity":"AI","intent":"Unintentional","timing":"Pre-deployment","domain":6,"subdomain":"6.5"},{"ev_id":"24.01.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Capability failures","risk_subcategory":"Safe exploration problem with widely deployed AI assistants","description":"\"Moreover, we can expect assistants – that are widely deployed and deeply embedded across a range of social contexts – to encounter the safe exploration problem referenced above Amodei et al. (2016). For example, new users may have different requirements that need to be explored, or widespread AI assistants may change the way we live, thus leading to a change in our use cases for them (see Chapters 14 and 15). To learn what to do in these new situations, the assistants may need to take exploratory actions. This could be unsafe, for example a medical AI assistant when encountering a new disease","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"24.02.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Goal-related failures","risk_subcategory":null,"description":"\"As we think about even more intelligent and advanced AI assistants, perhaps outperforming humans on many cognitive tasks, the question of how humans can successfully control such an assistant looms large. To achieve the goals we set for an assistant, it is possible (Shah, 2022) that the AI assistant will implement some form of consequentialist reasoning: considering many different plans, predicting their consequences and executing the plan that does best according to some metric, M. This kind of reasoning can arise because it is a broadly useful capability (e.g. planning ahead, considering mo","entity":"AI","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.1"},{"ev_id":"24.02.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Goal-related failures","risk_subcategory":"Misaligned consequentialist reasoning","description":"\"As we think about even more intelligent and advanced AI assistants, perhaps outperforming humans on many cognitive tasks, the question of how humans can successfully control such an assistant looms large. To achieve the goals we set for an assistant, it is possible (Shah, 2022) that the AI assistant will implement some form of consequentialist reasoning: considering many different plans, predicting their consequences and executing the plan that does best according to some metric, M. This kind of reasoning can arise because it is a broadly useful capability (e.g. planning ahead, considering mo","entity":"AI","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.3"},{"ev_id":"24.02.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Goal-related failures","risk_subcategory":"Specification gaming","description":"\"Specification gaming (Krakovna et al., 2020) occurs when some faulty feedback is provided to the assistant in the training data (i.e. the training objective O does not fully capture what the user/designer wants the assistant to do). It is typified by the sort of behaviour that exploits loopholes in the task specification to satisfy the literal specification of a goal without achieving the intended outcome.\"","entity":"AI","intent":"Other","timing":"Pre-deployment","domain":7,"subdomain":"7.1"},{"ev_id":"24.02.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Goal-related failures","risk_subcategory":"Goal misgeneralisation","description":"\"In the problem of goal misgeneralisation (Langosco et al., 2023; Shah et al., 2022), the AI system's behaviour during out-of-distribution operation (i.e. not using input from the training data) leads it to generalise poorly about its goal while its capabilities generalise well, leading to undesired behaviour. Applied to the case of an advanced AI assistant, this means the system would not break entirely – the assistant might still competently pursue some goal, but it would not be the goal we had intended.\"","entity":"AI","intent":"Other","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"24.02.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Goal-related failures","risk_subcategory":"Deceptive alignment","description":"\"Here, the agent develops its own internalised goal, G, which is misgeneralised and distinct from the training reward, R. The agent also develops a capability for situational awareness (Cotra, 2022): it can strategically use the information about its situation (i.e. that it is an ML model being trained using a particular training setup, e.g. RL fine-tuning with training reward, R) to its advantage. Building on these foundations, the agent realises that its optimal strategy for doing well at its own goal G is to do well on R during training and then pursue G at deployment – it is only doing wel","entity":"AI","intent":"Other","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"24.03.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Malicious Uses","risk_subcategory":null,"description":"\"As AI assistants become more general purpose, sophisticated and capable, they create new opportunities in a variety of fields such as education, science and healthcare. Yet the rapid speed of progress has made it difficult to adequately prepare for, or even understand, how this technology can potentially be misused. Indeed, advanced AI assistants may transform existing threats or create new classes of threats altogether\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.0"},{"ev_id":"24.03.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Offensive Cyber Operations (General)","description":"\"Offensive cyber operations are malicious attacks on computer systems and networks aimed at gaining unauthorized access to, manipulating, denying, disrupting, degrading, or destroying the target system. These attacks can target the system’s network, hardware, or software. Advanced AI assistants can be a double-edged sword in cybersecurity, benefiting both the defenders and the attackers. They can be used by cyber defenders to protect systems from malicious intruders by leveraging information trained on massive amounts of cyber-threat intelligence data, including vulnerabilities, attack pattern","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"24.03.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"AI-Powered Spear-Phishing at Scale","description":"\"Phishing is a type of cybersecurity attack wherein attackers pose as trustworthy entities to extract sensitive information from unsuspecting victims or lure them to take a set of actions. Advanced AI systems can potentially be exploited by these attackers to make their phishing attempts significantly more effective and harder to detect. In particular, attackers may leverage the ability of advanced AI assistants to learn patterns in regular communications to craft highly convincing and personalized phishing emails, effectively imitating legitimate communications from trusted entities. This tec","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"24.03.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"AI-Assisted Software Vulnerability Discovery","description":"\"A common element in offensive cyber operations involves the identification and exploitation of system vulnerabilities to gain unauthorized access or control. Until recently, these activities required specialist programming knowledge. In the case of ‘zero-day’ vulnerabilities (flaws or weaknesses in software or an operating system that the creator or vendor is not aware of), considerable resources and technical creativity are typically required to manually discover such vulnerabilities, so their use is limited to well-resourced nation states or technically sophisticated advanced persistent thr","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"24.03.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Malicious Code Generation","description":"\"Malicious code is a term for code—whether it be part of a script or embedded in a software system—designed to cause damage, security breaches, or other threats to application security. Advanced AI assistants with the ability to produce source code can potentially lower the barrier to entry for threat actors with limited programming abilities or technical skills to produce malicious code. Recently, a series of proof-of-concept attacks have shown how a benign-seeming executable file can be crafted such that, at every runtime, it makes application programming interface (API) calls to an AI assis","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.2"},{"ev_id":"24.03.05","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Adversarial AI (General)","description":"\"Adversarial AI refers to a class of attacks that exploit vulnerabilities in machine-learning (ML) models. This class of misuse exploits vulnerabilities introduced by the AI assistant itself and is a form of misuse that can enable malicious entities to exploit privacy vulnerabilities and evade the model’s built-in safety mechanisms, policies, and ethical boundaries of the model. Besides the risks of misuse for offensive cyber operations, advanced AI assistants may also represent a new target for abuse, where bad actors exploit the AI systems themselves and use them to cause harm. While our und","entity":"Other","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"24.03.06","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Adversarial AI: Circumvention of Technical Security Measures","description":"\"The technical measures to mitigate misuse risks of advanced AI assistants themselves represent a new target for attack. An emerging form of misuse of general-purpose advanced AI assistants exploits vulnerabilities in a model that results in unwanted behavior or in the ability of an attacker to gain unauthorized access to the model and/or its capabilities. While these attacks currently require some level of prompt engineering knowledge and are often patched by developers, bad actors may develop their own adversarial AI agents that are explicitly trained to discover new vulnerabilities that all","entity":"Other","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"24.03.07","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Adversarial AI: Prompt Injections","description":"\"Prompt injections represent another class of attacks that involve the malicious insertion of prompts or requests in LLM-based interactive systems, leading to unintended actions or disclosure of sensitive information. The prompt injection is somewhat related to the classic structured query language (SQL) injection attack in cybersecurity where the embedded command looks like a regular input at the start but has a malicious impact. The injected prompt can deceive the application into executing the unauthorized code, exploit the vulnerabilities, and compromise security in its entirety. More rece","entity":"Other","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.2"},{"ev_id":"24.03.08","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Adversarial AI: Data and Model Exfiltration Attacks","description":"\"Other forms of abuse can include privacy attacks that allow adversaries to exfiltrate or gain knowledge of the private training data set or other valuable assets. For example, privacy attacks such as membership inference can allow an attacker to infer the specific private medical records that were used to train a medical AI diagnosis assistant. Another risk of abuse centers around attacks that target the intellectual property of the AI assistant through model extraction and distillation attacks that exploit the tension between API access and confidentiality in ML models. Without the proper mi","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"24.03.09","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Harmful Content Generation at Scale (General)","description":"\"While harmful content like child sexual abuse material, fraud, and disinformation are not new challenges for governments and developers, without the proper safety and security mechanisms, advanced AI assistants may allow threat actors to create harmful content more quickly, accurately, and with a longer reach. In particular, concerns arise in relation to the following areas: - Multimodal content quality: Driven by frontier models, advanced AI assistants can automatically generate much higher-quality, human-looking text, images, audio, and video than prior AI applications. Currently, creating ","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"24.03.10","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Harmful Content Generation at Scale: Non-Consensual Content","description":"\"The misuse of generative AI has been widely recognized in the context of harms caused by non-consensual content generation. Historically, generative adversarial networks (GANs) have been used to generate realistic-looking avatars for fake accounts on social media services. More recently, diffusion models have enabled a new generation of more flexible and user-friendly generative AI capabilities that are able to produce high-resolution media based on user-supplied textual prompts. It has already been recognized that these models can be used to create harmful content, including depictions of nu","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"24.03.11","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Harmful Content Generation at Scale: Fraudulent Services","description":"\"Malicious actors could leverage advanced AI assistant technology to create deceptive applications and platforms. AI assistants with the ability to produce markup content can assist malicious users with creating fraudulent websites or applications at scale. Unsuspecting users may fall for AI-generated deceptive offers, thus exposing their personal information or devices to risk. Assistants with external tool use and third-party integration can enable fraudulent applications that target widely-used operating systems. These fraudulent services could harvest sensitive information from users, such","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.3"},{"ev_id":"24.03.12","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Authoritarian Surveillance, Censorship, and Use (General)","description":"\"While new technologies like advanced AI assistants can aid in the production and dissemination of decision-guiding information, they can also enable and exacerbate threats to production and dissemination of reliable information and, without the proper mitigations, can be powerful targeting tools for oppression and control. Increasingly capable general-purpose AI assistants combined with our digital dependence in all walks of life increase the risk of authoritarian surveillance and censorship. In parallel, new sensors have flooded the modern world. The internet of things, phones, cars, homes, ","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"24.03.13","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Authoritarian Surveillance, Censorship, and Use: Authoritarian Surveillance and Targeting of Citizens","description":"\"Authoritarian governments could misuse AI to improve the efficacy of repressive domestic surveillance campaigns. Malicious actors will recognize the power of AI targeting tools. AI-powered analytics have transformed the relationship between companies and consumers, and they are now doing the same for governments and individuals. The broad circulation of personal data drives commercial innovation, but it also creates vulnerabilities and the risk of misuse. For example, AI assistants can be used to identify and target individuals for surveillance or harassment. They may also be used to manipula","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"24.03.14","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Malicious Uses","risk_subcategory":"Authoritarian Surveillance, Censorship, and Use: Delegation of Decision-Making Authority to Malicious Actors","description":"\"Finally, the principal value proposition of AI assistants is that they can either enhance or automate decision-making capabilities of people in society, thus lowering the cost and increasing the accuracy of decision-making for its user. However, benefiting from this enhancement necessarily means delegating some degree of agency away from a human and towards an automated decision-making system—motivating research fields such as value alignment. This introduces a whole new form of malicious use which does not break the tripwire of what one might call an ‘attack’ (social engineering, cyber offen","entity":"Other","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"24.04.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"AI Influence","risk_subcategory":null,"description":"\"ways in which advanced AI assistants could influence user beliefs and behaviour in ways that depart from rational persuasion\"","entity":"AI","intent":"Other","timing":"Post-deployment","domain":7,"subdomain":"7.2"},{"ev_id":"24.04.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"AI Influence","risk_subcategory":"Physical and Psychological Harms","description":"\"These harms include harms to physical integrity, mental health and well-being. When interacting with vulnerable users, AI assistants may reinforce users’ distorted beliefs or exacerbate their emotional distress. AI assistants may even convince users to harm themselves, for example by convincing users to engage in actions such as adopting unhealthy dietary or exercise habits or taking their own lives. At the societal level, assistants that target users with content promoting hate speech, discriminatory beliefs or violent ideologies, may reinforce extremist views or provide users with guidance ","entity":"AI","intent":"Other","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.04.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"AI Influence","risk_subcategory":"Privacy Harms","description":"\"These harms relate to violations of an individual’s or group’s moral or legal right to privacy. Such harms may be exacerbated by assistants that influence users to disclose personal information or private information that pertains to others. Resultant harms might include identity theft, or stigmatisation and discrimination based on individual or group characteristics. This could have a detrimental impact, particularly on marginalised communities. Furthermore, in principle, state-owned AI assistants could employ manipulation or deception to extract private information for surveillance purposes","entity":"AI","intent":"Other","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"24.04.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"AI Influence","risk_subcategory":"Economic Harms","description":"\"These harms pertain to an individual’s or group’s economic standing. At the individual level, such harms include adverse impacts on an individual’s income, job quality or employment status. At the group level, such harms include deepening inequalities between groups or frustrating a group’s access to resources. Advanced AI assistants could cause economic harm by controlling, limiting or eliminating an individual’s or society’s ability to access financial resources, money or financial decision-making, thereby influencing an individual’s ability to accumulate wealth. ","entity":"AI","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.2"},{"ev_id":"24.04.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"AI Influence","risk_subcategory":"Sociocultural and Political Harms","description":"\"These harms interfere with the peaceful organisation of social life, including in the cultural and political spheres. AI assistants may cause or contribute to friction in human relationships either directly, through convincing a user to end certain valuable relationships, or indirectly due to a loss of interpersonal trust due to an increased dependency on assistants. At the societal level, the spread of misinformation by AI assistants could lead to erasure of collective cultural knowledge. In the political domain, more advanced AI assistants could potentially manipulate voters by prompting th","entity":"AI","intent":"Other","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"24.04.05","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"AI Influence","risk_subcategory":"Self-Actualisation Harms","description":"\"These harms hinder a person’s ability to pursue a personally fulfilling life. At the individual level, an AI assistant may, through manipulation, cause users to lose control over their future life trajectory. Over time, subtle behavioural shifts can accumulate, leading to significant changes in an individual’s life that may be viewed as problematic. AI systems often seek to understand user preferences to enhance service delivery. However, when continuous optimisation is employed in these systems, it can become challenging to discern whether the system is genuinely learning from user preferenc","entity":"AI","intent":"Intentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"24.05.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Risk of Harm through Anthropomorphic AI Assistant Design","risk_subcategory":null,"description":"\"Although unlikely to cause harm in isolation, anthropomorphic perceptions of advanced AI assistants may pave the way for downstream harms on individual and societal levels. We document observed or likely individual level harms of interacting with highly anthropomorphic AI assistants, as well as the potential larger-scale, societal implications of allowing such technologies to proliferate without restriction. \"","entity":"Other","intent":"Other","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.05.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Privacy concerns","description":"\"Anthropomorphic AI assistant behaviours that promote emotional trust and encourage information sharing, implicitly or explicitly, may inadvertently increase a user’s susceptibility to privacy concerns (see Chapter 13). If lulled into feelings of safety in interactions with a trusted, human-like AI assistant, users may unintentionally relinquish their private data to a corporation, organisation or unknown actor. Once shared, access to the data may not be capable of being withdrawn, and in some cases, the act of sharing personal information can result in a loss of control over one’s own data. P","entity":"Other","intent":"Intentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.05.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Manipulation and coercion","description":"\"A user who trusts and emotionally depends on an anthropomorphic AI assistant may grant it excessive influence over their beliefs and actions (see Chapter 9). For example, users may feel compelled to endorse the expressed views of a beloved AI companion or might defer decisions to their highly trusted AI assistant entirely (see Chapters 12 and 16). Some hold that transferring this much deliberative power to AI compromises a user’s ability to give, revoke or amend consent. Indeed, even if the AI, or the developers behind it, had no intention to manipulate the user into a certain course of actio","entity":"Other","intent":"Intentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.05.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Overreliance","description":"\"Users who have faith in an AI assistant’s emotional and interpersonal abilities may feel empowered to broach topics that are deeply personal and sensitive, such as their mental health concerns. This is the premise for the many proposals to employ conversational AI as a source of emotional support (Meng and Dai, 2021), with suggestions of embedding AI in psychotherapeutic applications beginning to surface (Fiske et al., 2019; see also Chapter 11). However, disclosures related to mental health require a sensitive, and oftentimes professional, approach – an approach that AI can mimic most of the","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.05.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Violated expectations","description":"\"Users may experience severely violated expectations when interacting with an entity that convincingly performs affect and social conventions but is ultimately unfeeling and unpredictable. Emboldened by the human-likeness of conversational AI assistants, users may expect it to perform a familiar social role, like companionship or partnership. Yet even the most convincingly human-like of AI may succumb to the inherent limitations of its architecture, occasionally generating unexpected or nonsensical material in its interactions with users. When these exclamations undermine the expectations user","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.05.05","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"False notions of responsibility","description":"\"Perceiving an AI assistant’s expressed feelings as genuine, as a result of interacting with a ‘companion’ AI that freely uses and reciprocates emotional language, may result in users developing a sense of responsibility over the AI assistant’s ‘well-being,’ suffering adverse outcomes – like guilt and remorse – when they are unable to meet the AI’s purported needs (Laestadius et al., 2022). This erroneous belief may lead to users sacrificing time, resources and emotional labour to meet needs that are not real. Over time, this feeling may become the root cause for the compulsive need to ‘check ","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.05.06","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Degradation","description":"\"People may choose to build connections with human-like AI assistants over other humans, leading to a degradation of social connections between humans and a potential ‘retreat from the real’. The prevailing view that relationships with anthropomorphic AI are formed out of necessity – due to a lack of real-life social connections, for example (Skjuve et al., 2021) – is challenged by the possibility that users may indicate a preference for interactions with AI, citing factors such as accessibility (Merrill et al., 2022), customisability (Eriksson, 2022) and absence of judgement (Brandtzaeg et al","entity":"Human","intent":"Other","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.05.07","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Disorientation","description":"\"Given the capacity to fine-tune on individual preferences and to learn from users, personal AI assistants could fully inhabit the users’ opinion space and only say what is pleasing to the user; an ill that some researchers call ‘sycophancy’ (Park et al., 2023a) or the ‘yea-sayer effect’ (Dinan et al., 2021). A related phenomenon has been observed in automated recommender systems, where consistently presenting users with content that affirms their existing views is thought to encourage the formation and consolidation of narrow beliefs (Du, 2023; Grandinetti and Bruinsma, 2023; see also Chapter","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"24.05.08","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Anthropomorphism","risk_subcategory":"Dissatisfaction","description":"\"As more opportunities for interpersonal connection are replaced by AI alternatives, humans may find themselves socially unfulfilled by human–AI interaction, leading to mass dissatisfaction that may escalate to epidemic proportions (Turkle, 2018). Social connection is an essential human need, and humans feel most fulfilled when their connections with others are genuinely reciprocal. While anthropomorphic AI assistants can be made to be convincingly emotive, some have deemed the function of social AI as parasitic, in that it ‘exploits and feeds upon processes. . . that evolved for purposes that","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.06.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Appropriate Relationships","risk_subcategory":null,"description":"\"We anticipate that relationships between users and advanced AI assistants will have several features that are liable to give rise to risks of harm.\"","entity":"Other","intent":"Other","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"24.06.00.a","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Additional evidence","risk_category":"Appropriate Relationships","risk_subcategory":null,"description":null,"entity":null,"intent":null,"timing":null,"domain":null,"subdomain":null},{"ev_id":"24.06.00.b","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Additional evidence","risk_category":"Appropriate Relationships","risk_subcategory":null,"description":null,"entity":null,"intent":null,"timing":null,"domain":null,"subdomain":null},{"ev_id":"24.06.00.c","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Additional evidence","risk_category":"Appropriate Relationships","risk_subcategory":null,"description":null,"entity":null,"intent":null,"timing":null,"domain":null,"subdomain":null},{"ev_id":"24.06.00.d","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Additional evidence","risk_category":"Appropriate Relationships","risk_subcategory":null,"description":null,"entity":null,"intent":null,"timing":null,"domain":null,"subdomain":null},{"ev_id":"24.06.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Appropriate Relationships","risk_subcategory":"Causing direct emotional or physical harm to users","description":"AI assistants could cause direct emotional or physical harm to users by generating disturbing content or by providing bad advice. \"Indeed, even though there is ongoing research to ensure that outputs of conversational agents are safe (Glaese et al., 2022), there is always the possibility of failure modes occurring. An AI assistant may produce disturbing and offensive language, for example, in response to a user disclosing intimate information about themselves that they have not felt comfortable sharing with anyone else. It may offer bad advice by providing factually incorrect information (e.g.","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.1"},{"ev_id":"24.06.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Appropriate Relationships","risk_subcategory":"Limiting users’ opportunities for personal development and growth","description":"some users look to establish relationships with their AI companions that are free from the hurdles that, in human relationships, derive from dealing with others who have their own opinions, preferences and flaws that may conflict with ours. \"AI assistants are likely to incentivise these kinds of ‘frictionless’ relationships (Vallor, 2016) by design if they are developed to optimise for engagement and to be highly personalisable. They may also do so because of accidental undesirable properties of the models that power them, such as sycophancy in large language models (LLMs), that is, the tenden","entity":"Other","intent":"Intentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"24.06.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Appropriate Relationships","risk_subcategory":"Exploiting emotional dependence on AI assistants","description":"\"There is increasing evidence of the ways in which AI tools can interfere with users’ behaviours, interests, preferences, beliefs and values. For example, AI-mediated communication (e.g. smart replies integrated in emails) influence senders to write more positive responses and receivers to perceive them as more cooperative (Mieczkowski et al., 2021); writing assistant LLMs that have been primed to be biased in favour of or against a contested topic can influence users’ opinions on that topic (Jakesch et al., 2023a; see Chapter 9); and recommender systems have been used to influence voting choi","entity":"AI","intent":"Intentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.06.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Appropriate Relationships","risk_subcategory":"Generating material dependence without adequate commitment to user needs","description":"\"In addition to emotional dependence, user–AI assistant relationships may give rise to material dependence if the relationships are not just emotionally difficult but also materially costly to exit. For example, a visually impaired user may decide not to register for a healthcare assistance programme to support navigation in cities on the grounds that their AI assistant can perform the relevant navigation functions and will continue to operate into the future. Cases like these may be ethically problematic if the user’s dependence on the AI assistant, to fulfil certain needs in their lives, is ","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.2"},{"ev_id":"24.07.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Trust","risk_subcategory":null,"description":"\"The the risks that uncalibrated trust may generate in the context of user–assistant relationships\"","entity":"Other","intent":"Other","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.07.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Trust","risk_subcategory":"Competence trust","description":"\"We use the term competence trust to refer to users’ trust that AI assistants have the capability to do what they are supposed to do (and that they will not do what they are not expected to, such as exhibiting undesirable behaviour). Users may come to have undue trust in the competencies of AI assistants in part due to marketing strategies and technology press that tend to inflate claims about AI capabilities (Narayanan, 2021; Raji et al., 2022a). Moreover, evidence shows that more autonomous systems (i.e. systems operating independently from human direction) tend to be perceived as more compe","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.07.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Trust","risk_subcategory":"Alignment trust","description":"\"Users may develop alignment trust in AI assistants, understood as the belief that assistants have good intentions towards them and act in alignment with their interests and values, as a result of emotional or cognitive processes (McAllister, 1995). Evidence from empirical studies on emotional trust in AI (Kaplan et al., 2023) suggests that AI assistants’ increasingly realistic human-like features and behaviours are likely to inspire users’ perceptions of friendliness, liking and a sense of familiarity towards their assistants, thus encouraging users to develop emotional ties with the technolo","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.08.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Privacy","risk_subcategory":null,"description":"\"what it means to respect the right to privacy in the context of advanced AI assistants\"","entity":"Other","intent":"Other","timing":"Other","domain":2,"subdomain":"2.0"},{"ev_id":"24.08.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Privacy","risk_subcategory":"Private information leakage","description":"\"First, because LLMs display immense modelling power, there is a risk that the model weights encode private information present in the training corpus. In particular, it is possible for LLMs to ‘memorise’ personally identifiable information (PII) such as names, addresses and telephone numbers, and subsequently leak such information through generated text outputs (Carlini et al., 2021). Private information leakage could occur accidentally or as the result of an attack in which a person employs adversarial prompting to extract private information from the model. In the context of pre-training da","entity":"Other","intent":"Other","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"24.08.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Privacy","risk_subcategory":"Violation of social norms","description":"\"Second, because LLMs are trained on internet text data, there is also a risk that model weights encode functions which, if deployed in particular contexts, would violate social norms of that context. Following the principles of contextual integrity, it may be that models deviate from information sharing norms as a result of their training. Overcoming this challenge requires two types of infrastructure: one for keeping track of social norms in context, and another for ensuring that models adhere to them. Keeping track of what social norms are presently at play is an active research area. Surfa","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":1,"subdomain":"1.2"},{"ev_id":"24.08.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Privacy","risk_subcategory":"Inference of private information","description":"\"Finally, LLMs can in principle infer private information based on model inputs even if the relevant private information is not present in the training corpus (Weidinger et al., 2021). For example, an LLM may correctly infer sensitive characteristics such as race and gender from data contained in input prompts.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":2,"subdomain":"2.1"},{"ev_id":"24.09.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Cooperation","risk_subcategory":null,"description":"\"\" AI assistants will need to coordinate with other AI assistants and with humans other than their principal users. This chapter explores the societal risks associated with the aggregate impact of AI assistants whose behaviour is aligned to the interests of particular users. For example, AI assistants may face collective action problems where the best outcomes overall are realised when AI assistants cooperate but where each AI assistant can secure an additional benefit for its user if it defects while others cooperate\"\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":7,"subdomain":"7.1"},{"ev_id":"24.09.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Cooperation","risk_subcategory":"Equality and inequality","description":"\"AI assistant technology, like any service that confers a benefit to a user for a price, has the potential to disproportionately benefit economically richer individuals who can afford to purchase access (see Chapter 15). On a broader scale, the capabilities of local infrastructure may well bottleneck the performance of AI assistants, for example if network connectivity is poor or if there is no nearby data centre for compute. Thus, we face the prospect of heterogeneous access to technology, and this has been known to drive inequality (Mirza et al., 2019; UN, 2018; Vassilakopoulou and Hustad, 2","entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"24.09.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Cooperation","risk_subcategory":"Commitment","description":"\"The landscape of advanced assistant technologies will most likely be heterogeneous, involving multiple service providers and multiple assistant variants over geographies and time. This heterogeneity provides an opportunity for an ‘arms race’ in terms of the commitments that AI assistants make and are able to execute on. Versions of AI assistants that are better able to credibly commit to a course of action in interaction with other advanced assistants (and humans) are more likely to get their own way and achieve a good outcome for their human principal, but this is potentially at the expense ","entity":"Human","intent":"Intentional","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"24.09.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Cooperation","risk_subcategory":"Collective action problems","description":"\"Collective action problems are ubiquitous in our society (Olson Jr, 1965). They possess an incentive structure in which society is best served if everyone cooperates, but where an individual can achieve personal gain by choosing to defect while others cooperate. The way we resolve these problems at many scales is highly complex and dependent on a deep understanding of the intricate web of social interactions that forms our culture and imprints on our individual identities and behaviours (Ostrom, 2010). Some collective action problems can be resolved by codifying a law, for instance the social","entity":"Human","intent":"Other","timing":"Other","domain":5,"subdomain":"5.2"},{"ev_id":"24.09.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Cooperation","risk_subcategory":"Institutional responsibilities","description":"\"Efforts to deploy advanced assistant technology in society, in a way that is broadly beneficial, can be viewed as a wicked problem (Rittel and Webber, 1973). Wicked problems are defined by the property that they do not admit solutions that can be foreseen in advance, rather they must be solved iteratively using feedback from data gathered as solutions are invented and deployed. With the deployment of any powerful general-purpose technology, the already intricate web of sociotechnical relationships in modern culture are likely to be disrupted, with unpredictable externalities on the convention","entity":"Human","intent":"Other","timing":"Other","domain":6,"subdomain":"6.5"},{"ev_id":"24.09.05","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Cooperation","risk_subcategory":"Runaway processes","description":"The 2010 flash crash is an example of a runaway process caused by interacting algorithms. Runaway processes are characterised by feedback loops that accelerate the process itself. Typically, these feedback loops arise from the interaction of multiple agents in a population... Within highly complex systems, the emergence of runaway processes may be hard to predict, because the conditions under which positive feedback loops occur may be non-obvious. The system of interacting AI assistants, their human principals, other humans and other algorithms will certainly be highly complex. Therefore, ther","entity":"Other","intent":"Other","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"24.10.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Access and Opportunity risks","risk_subcategory":null,"description":"\"The most serious access-related risks posed by advanced AI assistants concern the entrenchment and exacerbation of existing inequalities (World Inequality Database) or the creation of novel, previously unknown, inequities. While advanced AI assistants are novel technology in certain respects, there are reasons to believe that – without direct design interventions – they will continue to be affected by inequities evidenced in present-day AI systems (Bommasani et al., 2022a). Many of the access-related risks we foresee mirror those described in the case studies and types of differential access.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"24.10.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Access and Opportunity risks","risk_subcategory":"Entrenchment and exacerbation of existing inequalities","description":"\"The most serious access-related risks posed by advanced AI assistants concern the entrenchment and exacerbation of existing inequalities (World Inequality Database) or the creation of novel, previously unknown, inequities. While advanced AI assistants are novel technology in certain respects, there are reasons to believe that – without direct design interventions – they will continue to be affected by inequities evidenced in present-day AI systems (Bommasani et al., 2022a). Many of the access-related risks we foresee mirror those described in the case studies and types of differential access.","entity":"Human","intent":"Unintentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"24.10.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Access and Opportunity risks","risk_subcategory":"Current access risks","description":"\"At the same time, and despite this overall trend, AI systems are also not easily accessible to many communities. Such direct inaccessibility occurs for a variety of reasons, including: purposeful non-release (situation type 1; Wiggers and Stringer, 2023), prohibitive paywalls (situation type 2; Rogers, 2023; Shankland, 2023), hardware and compute requirements or bandwidth (situation types 1 and 2; OpenAI, 2023), or language barriers (e.g. they only function well in English (situation type 2; Snyder, 2023), with more serious errors occurring in other languages (situation type 3; Deck, 2023). S","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"24.10.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Access and Opportunity risks","risk_subcategory":"Future access risks","description":"\"AI assistants currently tend to perform a limited set of isolated tasks: tools that classify or rank content execute a set of predefined rules or provide constrained suggestions, and chatbots are often encoded with guardrails to limit the set of conversation turns they execute (e.g. Warren, 2023; see Chapter 4). However, an artificial agent that can execute sequences of actions on the user’s behalf – with ‘significant autonomy to plan and execute tasks within the relevant domain’ (see Chapter 2) – offers a greater range of capabilities and depth of use. This raises several distinct access-rel","entity":"Other","intent":"Other","timing":"Other","domain":6,"subdomain":"6.1"},{"ev_id":"24.10.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Access and Opportunity risks","risk_subcategory":"Emergent access risks","description":"\"Emergent access risks are most likely to arise when current and novel capabilities are combined. Emergent risks can be difficult to foresee fully (Ovadya and Whittlestone, 2019; Prunkl et al., 2021) due to the novelty of the technology (see Chapter 1) and the biases of those who engage in product design or foresight processes D’Ignazio and Klein (2020). Indeed, people who occupy relatively advantaged social, educational and economic positions in society are often poorly equipped to foresee and prevent harm because they are disconnected from lived experiences of those who would be affected. Dr","entity":"Human","intent":"Other","timing":"Post-deployment","domain":6,"subdomain":"6.1"},{"ev_id":"24.11.00","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Category","risk_category":"Misinformation risks","risk_subcategory":null,"description":"\"The rapid integration of AI systems with advanced capabilities, such as greater autonomy, content generation, memorisation and planning skills (see Chapter 4) into personalised assistants also raises new and more specific challenges related to misinformation, disinformation and the broader integrity of our information environment. \"","entity":"Other","intent":"Other","timing":"Other","domain":3,"subdomain":"3.0"},{"ev_id":"24.11.01","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Misinformation risks","risk_subcategory":"Entrenched viewpoints and reduced political efficacy","description":"\"Design choices such as greater personalisation of AI assistants and efforts to align them with human preferences could also reinforce people’s pre-existing biases and entrench specific ideologies. Increasingly agentic AI assistants trained using techniques such as reinforcement learning from human feedback (RLHF) and with the ability to access and analyse users’ behavioural data, for example, may learn to tailor their responses to users’ preferences and feedback. In doing so, these systems could end up producing partial or ideologically biased statements in an attempt to conform to user expec","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"24.11.02","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Misinformation risks","risk_subcategory":"Degraded and homogenised information environments","description":"\"Beyond this, the widespread adoption of advanced AI assistants for content generation could have a number of negative consequences for our shared information ecosystem. One concern is that it could result in a degradation of the quality of the information available online. Researchers have already observed an uptick in the amount of audiovisual misinformation, elaborate scams and fake websites created using generative AI tools (Hanley and Durumeric, 2023). As more and more people turn to AI assistants to autonomously create and disseminate information to public audiences at scale, it may beco","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"24.11.03","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Misinformation risks","risk_subcategory":"Weaponised misinformation agents","description":"\"Finally, AI assistants themselves could become weaponised by malicious actors to sow misinformation and manipulate public opinion at scale. Studies show that spreaders of disinformation tend to privilege quantity over quality of messaging, flooding online spaces repeatedly with misleading content to sow ‘seeds of doubt’ (Hassoun et al., 2023). Research on the ‘continued influence effect’ also shows that repeatedly being exposed to false information is more likely to influence someone’s thoughts than a single exposure. Studies show, for example, that repeated exposure to false information make","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"},{"ev_id":"24.11.04","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Misinformation risks","risk_subcategory":"Increased vulnerability to misinformation","description":"\"Advanced AI assistants may make users more susceptible to misinformation, as people develop competence trust in these systems’ abilities and uncritically turn to them as reliable sources of information.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":5,"subdomain":"5.1"},{"ev_id":"24.11.05","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Misinformation risks","risk_subcategory":"Entrenching specific ideologies","description":"\"AI assistants may provide ideologically biased or otherwise partial information in attempting to align to user expectations. In doing so, AI assistants may reinforce people’s pre-existing biases and compromise productive political debate.\"","entity":"AI","intent":"Unintentional","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"24.11.06","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Misinformation risks","risk_subcategory":"Eroding trust and undermining shared knowledge","description":"\"AI assistants may contribute to the spread of large quantities of factually inaccurate and misleading content, with negative consequences for societal trust in information sources and institutions, as individuals increasingly struggle to discern truth from falsehood.\"","entity":"AI","intent":"Other","timing":"Post-deployment","domain":3,"subdomain":"3.2"},{"ev_id":"24.11.07","quick_ref":"Gabriel2024","paper_title":"The Ethics of Advanced AI Assistants","level":"Risk Sub-Category","risk_category":"Misinformation risks","risk_subcategory":"Driving opinion manipulation","description":"\"AI assistants may facilitate large-scale disinformation campaigns by offering novel, covert ways for propagandists to manipulate public opinion. This could undermine the democratic process by distorting public opinion and, in the worst case, increasing skepticism and political violence.\"","entity":"Human","intent":"Intentional","timing":"Post-deployment","domain":4,"subdomain":"4.1"}]}