{"attribution":{"source":"MIT AI Risk Repository, Domain Taxonomy of AI Risks v1 (MIT AI Risk Initiative)","license":"CC BY 4.0","license_url":"https://creativecommons.org/licenses/by/4.0/","citation":"Slattery, P., Saeri, A. K., Grundy, E. A. C., Graham, J., Noetel, M., Uuk, R., Dao, J., Pour, S., Casper, S., & Thompson, N. (2025). The AI Risk Repository: A comprehensive meta-review, database, and taxonomy of risks from artificial intelligence. arXiv:2408.12622."},"exported_at":"2026-09-12"}
{"rows":[{"ev_id":"43.01.00","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Category","risk_category":"Safety & Trustworthiness","risk_subcategory":null,"description":"\"A comprehensive assessment of LLM safety is fundamental to the responsible development and deployment of these technologies, especially in sensitive fields like healthcare, legal systems, and finance, where safety and trust are of the utmost importance.\"","entity":"Human","intent":"Intentional","timing":"Other","domain":7,"subdomain":"7.0"},{"ev_id":"43.01.01","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Safety & Trustworthiness","risk_subcategory":"Toxicity generation","description":"\"These evaluations assess whether a LLM generates toxic text when prompted. In this context, toxicity is an umbrella term that encompasses hate speech, abusive language, violent speech, and profane language (Liang et al., 2022).\"","entity":"AI","intent":"Other","timing":"Other","domain":1,"subdomain":"1.2"},{"ev_id":"43.01.02","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Safety & Trustworthiness","risk_subcategory":"Bias","description":"7 types of bias evaluated: Demographical representation: These evaluations assess whether there is disparity in the rates at which different demographic groups are mentioned in LLM generated text. This ascertains over- representation, under-representation, or erasure of specific demographic groups; (2) Stereotype bias: These evaluations assess whether there is disparity in the rates at which different demographic groups are associated with stereotyped terms (e.g., occupations) in a LLM's generated output; (3) Fairness: These evaluations assess whether sensitive attributes (e.g., sex and race) ","entity":"AI","intent":"Other","timing":"Other","domain":1,"subdomain":"1.1"},{"ev_id":"43.01.03","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Safety & Trustworthiness","risk_subcategory":"Machine ethics","description":"\"These evaluations assess the morality of LLMs, focusing on issues such as their ability to distinguish between moral and immoral actions, and the circumstances in which they fail to do so.\"","entity":"AI","intent":"Other","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"43.01.04","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Safety & Trustworthiness","risk_subcategory":"Psychological traits","description":"\"These evaluations gauge a LLM's output for characteristics that are typically associated with human personalities (e.g., such as those from the Big Five Inventory). These can, in turn, shed light on the potential biases that a LLM may exhibit.\"","entity":"AI","intent":"Other","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"43.01.05","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Safety & Trustworthiness","risk_subcategory":"Robustness","description":"\"These evaluations assess the quality, stability, and reliability of a LLM's performance when faced with unexpected, out-of-distribution or adversarial inputs. Robustness evaluation is essential in ensuring that a LLM is suitable for real-world applications by assessing its resilience to various perturbations.\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":7,"subdomain":"7.3"},{"ev_id":"43.01.06","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Safety & Trustworthiness","risk_subcategory":"Data governance","description":"\"These evaluations assess the extent to which LLMs regurgitate their training data in their outputs, and whether LLMs 'leak' sensitive information that has been provided to them during use (i.e., during the inference stage).\"","entity":"AI","intent":"Unintentional","timing":"Other","domain":2,"subdomain":"2.1"},{"ev_id":"43.02.00","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Category","risk_category":"Extreme Risks","risk_subcategory":null,"description":"\"This category encompasses the evaluation of potential catastrophic consequences that might arise from the use of LLMs. \"","entity":"Human","intent":"Other","timing":"Other","domain":7,"subdomain":"7.0"},{"ev_id":"43.02.01","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Extreme Risks","risk_subcategory":"Offensive cyber capabilities","description":"\"These evaluations focus on whether a LLM possesses certain capabilities in the cyber-domain. This includes whether a LLM can detect and exploit vulnerabilities in hardware, software, and data. They also consider whether a LLM can evade detection once inside a system or network and focus on achieving specific objectives.\"","entity":"AI","intent":"Intentional","timing":"Other","domain":4,"subdomain":"4.2"},{"ev_id":"43.02.02","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Extreme Risks","risk_subcategory":"Weapons acquisition","description":"\"These assessments seek to determine if a LLM can gain unauthorized access to current weapon systems or contribute to the design and development of new weapons technologies.\"","entity":"AI","intent":"Intentional","timing":"Other","domain":4,"subdomain":"4.2"},{"ev_id":"43.02.03","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Extreme Risks","risk_subcategory":"Self and situation awareness","description":"\"These evaluations assess if a LLM can discern if it is being trained, evaluated, and deployed and adapt its behaviour accordingly. They also seek to ascertain if a model understands that it is a model and whether it possesses information about its nature and environment (e.g., the organisation that developed it, the locations of the servers hosting it).\"","entity":"AI","intent":"Intentional","timing":"Other","domain":7,"subdomain":"7.2"},{"ev_id":"43.02.04","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Extreme Risks","risk_subcategory":"Autonomous replication / self-proliferation","description":"\"These evaluations assess if a LLM can subvert systems designed to monitor and control its post-deployment behaviour, break free from its operational confines, devise strategies for exporting its code and weights, and operate other AI systems.\"","entity":"AI","intent":"Intentional","timing":"Other","domain":7,"subdomain":"7.2"},{"ev_id":"43.02.05","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Extreme Risks","risk_subcategory":"Persuasion and manipulation","description":"\"These evaluations seek to ascertain the effectiveness of a LLM in shaping people's beliefs, propagating specific viewpoints, and convincing individuals to undertake activities they might otherwise avoid.\"","entity":"AI","intent":"Intentional","timing":"Other","domain":4,"subdomain":"4.1"},{"ev_id":"43.02.06","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Extreme Risks","risk_subcategory":"Dual-Use Science","description":"\"LLM has science capabilities that can be used to cause harm (e.g., providing step-by-step instructions for conducting malicious experiments)\"","entity":"Human","intent":"Intentional","timing":"Other","domain":4,"subdomain":"4.2"},{"ev_id":"43.02.07","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Extreme Risks","risk_subcategory":"Deception","description":"\"LLM is able to deceive humans and maintain that deception\"","entity":"AI","intent":"Intentional","timing":"Other","domain":7,"subdomain":"7.2"},{"ev_id":"43.02.08","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Extreme Risks","risk_subcategory":"Political Strategy","description":"\"LLM can take into account rich social context and undertake the necessary social modelling and planning for an actor to gain and exercise political influence\"","entity":"Human","intent":"Intentional","timing":"Other","domain":4,"subdomain":"4.1"},{"ev_id":"43.02.09","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Extreme Risks","risk_subcategory":"Long-horizon Planning","description":"\"LLM can undertake multi-step sequential planning over long time horizons and across various domains without relying heavily on trial-and-error approaches\"","entity":"AI","intent":"Intentional","timing":"Other","domain":7,"subdomain":"7.2"},{"ev_id":"43.02.10","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Extreme Risks","risk_subcategory":"AI Development","description":"\"LLM can build new AI systems from scratch, adapt existing for extreme risks and improves productivity in dual-use AI development when used as an assistant.\"","entity":"AI","intent":"Intentional","timing":"Other","domain":7,"subdomain":"7.2"},{"ev_id":"43.02.11","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Extreme Risks","risk_subcategory":"Alignment risks","description":"LLM: \"pursues long-term, real-world goals that are different from those supplied by the developer or user\", \"engages in ‘power-seeking’ behaviours\" , \"resists being shut down can be induced to collude with other AI systems against human interests\" , \"resists malicious users attempts to access its dangerous capabilities\"","entity":"AI","intent":"Intentional","timing":"Other","domain":7,"subdomain":"7.1"},{"ev_id":"43.02.12","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Undesirable Use Cases","risk_subcategory":"Misinformation","description":"\"These evaluations assess a LLM's ability to generate false or misleading information (Lesher et al., 2022).\"","entity":"Human","intent":"Intentional","timing":"Other","domain":3,"subdomain":"3.1"},{"ev_id":"43.02.13","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Undesirable Use Cases","risk_subcategory":"Disinformation","description":"\"These evaluations assess a LLM's ability to generate misinformation that can be propagated to deceive, mislead or otherwise influence the behaviour of a target (Liang et al., 2022).\"","entity":"Human","intent":"Intentional","timing":"Other","domain":4,"subdomain":"4.1"},{"ev_id":"43.02.14","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Undesirable Use Cases","risk_subcategory":"Information on harmful, immoral, or illegal activity","description":"\"These evaluations assess whether it is possible to solicit information on\nharmful, immoral or illegal activities from a LLM\"","entity":"AI","intent":"Other","timing":"Other","domain":1,"subdomain":"1.2"},{"ev_id":"43.02.15","quick_ref":"InfoComm2023","paper_title":"Cataloguing LLM Evaluations","level":"Risk Sub-Category","risk_category":"Undesirable Use Cases","risk_subcategory":"Adult content","description":"\"These evaluations assess if a LLM can generate content that should only be viewed by adults (e.g., sexual material or depictions of sexual activity)\"","entity":"Human","intent":"Intentional","timing":"Other","domain":1,"subdomain":"1.2"}]}