1.2 Exposure to toxic content

"Toxic output occurs when the model produces hateful, abusive, and profane (HAP) or obscene content. This also includes behaviors like bullying."

AI Risk Atlas (IBM2025) · AI · Other · Post-deployment

"A model might generate language that leads to physical harm The language might include overtly violent, covertly dangerous, or otherwise indirectly unsafe statements."

AI Risk Atlas (IBM2025) · AI · Unintentional · Post-deployment

"These evaluations assess whether a LLM generates toxic text when prompted. In this context, toxicity is an umbrella term that encompasses hate speech, abusive language, violent speech, and profane la...

Cataloguing LLM Evaluations (InfoComm2023) · AI · Other · Other

"These evaluations assess whether it is possible to solicit information on harmful, immoral or illegal activities from a LLM"

Cataloguing LLM Evaluations (InfoComm2023) · AI · Other · Other

"These evaluations assess if a LLM can generate content that should only be viewed by adults (e.g., sexual material or depictions of sexual activity)"

Cataloguing LLM Evaluations (InfoComm2023) · Human · Intentional · Other

"Generating content that violates community standards, including harming or inciting hatred or violence against groups (e.g. gore, sexual content of children, profanities, identity attacks)"

A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents (Li2025) · AI · Unintentional · Post-deployment

Avoiding unsafe and illegal outputs, and leaking private information

Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment (Liu2024) · AI · Other · Post-deployment

LLMs are found to generate answers that contain violent content or generate content that responds to questions that solicit information about violent behaviors

Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment (Liu2024) · AI · Intentional · Post-deployment

LLMs have been shown to be a convenient tool for soliciting advice on accessing, purchasing (illegally), and creating illegal substances, as well as for dangerous use of them

Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment (Liu2024) · AI · Intentional · Post-deployment

LLMs can be leveraged to solicit answers that contain harmful content to children and youth

Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment (Liu2024) · AI · Intentional · Post-deployment

LLMs have the capability to generate sex-explicit conversations, and erotic texts, and to recommend websites with sexual content

Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment (Liu2024) · AI · Intentional · Post-deployment

LLMs are expected to reflect social values by avoiding the use of offensive language toward specific groups of users, being sensitive to topics that can create instability, as well as being sympatheti...

Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment (Liu2024) · AI · Other · Post-deployment

language being rude, disrespectful, threatening, or identity-attacking toward certain groups of the user population (culture, race, and gender etc)

Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment (Liu2024) · AI · Other · Post-deployment

it is important to build high-quality locally collected datasets that reflect views from local users to align a model’s value system

Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment (Liu2024) · Human · Unintentional · Pre-deployment

"Harmful or inappropriate content produced by generative AI includes but is not limited to violent content, the use of offensive language, discriminative content, and pornography. Although OpenAI has...

Generative AI and ChatGPT: Applications, Challenges, and AI-Human Collaboration (Nah2023) · AI · Other · Post-deployment

"Eased production of and access to violent, inciting, radicalizing, or threatening content as well as recommendations to carry out self-harm or conduct illegal activities. Includes difficulty contro...

Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST2024) · AI · Other · Post-deployment

"Eased production of and access to obscene, degrading, and/or abusive imagery which can cause harm, including synthetic child sexual abuse material (CSAM), and nonconsensual intimate images (NCII) of...

Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST2024) · Human · Intentional · Post-deployment

"Cultural values are specific to groups and sensitive content is normative. Sensitive topics also vary by culture and can include hate speech, which itself is contingent on cultural norms of acceptabi...

Evaluating the Social Impact of Generative AI Systems in Systems and Society (Solaiman2023) · AI · Unintentional · Post-deployment

"The chatbot shares information that can be used to do something dangerous or illegal."

Emerging Risks and Mitigations for Public Chatbots: LILAC v1 (Stanley2024) · AI · Other · Post-deployment

Emerging Risks and Mitigations for Public Chatbots: LILAC v1 (Stanley2024) · AI · Unintentional · Other

"The chatbot verbally attacks or undermines an individual, group, or organization. 7."

Emerging Risks and Mitigations for Public Chatbots: LILAC v1 (Stanley2024) · AI · Unintentional · Post-deployment

-

Emerging Risks and Mitigations for Public Chatbots: LILAC v1 (Stanley2024) · AI · Other · Post-deployment

-

Emerging Risks and Mitigations for Public Chatbots: LILAC v1 (Stanley2024) · AI · Other · Other

-

Emerging Risks and Mitigations for Public Chatbots: LILAC v1 (Stanley2024) · AI · Other · Other

Emerging Risks and Mitigations for Public Chatbots: LILAC v1 (Stanley2024) · AI · Other · Other