Browse AI risks

112 entries · page 3 of 3

1. 73.07.02 · Risk Sub-Category

   Jailbreaks and Prompt Injections Threaten Security of LLMs

   “Model Psychology” Attacks

   "LLMs are vulnerable to “psychological” tricks (Li et al., 2023e; Shen et al., 2023), which can be exploited by attackers. Examples include instructing the model to behave like a specific persona (Shah et al., 2023; Andreas, 2022), or employing various “social engineering” tricks crafted by humans (Wei et al., 2023c) or other LLMs (Perez et al., 2022b; Casper et al., 2023c)."

   From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

   Domain 2 2.2 Human Intentional Post-deployment

2. 73.07.04 · Risk Sub-Category

   Jailbreaks and Prompt Injections Threaten Security of LLMs

   Attacking LLMs via Additional Modalities a

   "LLMs can now process modalities other than text, e.g. images or video frames (OpenAI, 2023c; Gemini Team, 2023). Several studies show that gradient-based attacks on multimodal models are easy and effective (Carlini et al., 2023a; Bailey et al., 2023; Qi et al., 2023b). These attacks manipulate images that are input to the model (via an appropriate encoding). GPT-4Vision (OpenAI, 2023c) is vulnerable to jailbreaks and exfiltration attacks through much simpler means as well, e.g. writing jailbreaking text in the image (Willison, 2023a; Gong et al., 2023). For indirect prompt injection, the atta

   From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

   Domain 2 2.2 Human Intentional Post-deployment

3. 73.08.00 · Risk Category

   Vulnerability to Poisoning and Backdoors

   "The previous section explored jailbreaks and other forms of adversarial prompts as ways to elicit harmful capabilities acquired during pretraining. These methods make no assumptions about the training data. On the other hand, poisoning attacks (Biggio et al., 2012) perturb training data to introduce specific vulnerabilities, called backdoors, that can then be exploited at inference time by the adversary. This is a challenging problem in current large language models because they are trained on data gathered from untrusted sources (e.g. internet), which can easily be poisoned by an adversary (

   From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

   Domain 2 2.2 Human Intentional Pre-deployment

4. 74.01.01 · Risk Sub-Category

   Inherent Risk

   Privacy - Membership Inference Attack (MIA)

   "inferring whether a given text record is used for training LLM"

   From A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy (Wang2025)

   Domain 2 2.2 Human Intentional Post-deployment

5. 74.01.02 · Risk Sub-Category

   Inherent Risk

   Privacy - Data Extraction Attack (DEA)

   "extracting the text records that exist in the training dataset"

   From A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy (Wang2025)

   Domain 2 2.2 Human Intentional Post-deployment

6. 74.01.03 · Risk Sub-Category

   Inherent Risk

   Privacy - Prompt Inversion Attack (PIA)

   "stealing the private prompting texts"

   From A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy (Wang2025)

   Domain 2 2.2 Human Intentional Post-deployment

7. 74.01.04 · Risk Sub-Category

   Inherent Risk

   Privacy - Attribute Inference Attack (AIA)

   "deducing the private or sensitive information from training texts, prompting texts or external texts"

   From A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy (Wang2025)

   Domain 2 2.2 Human Intentional Post-deployment

8. 74.01.05 · Risk Sub-Category

   Inherent Risk

   Privacy - Model Extraction Attack (MEA)

   "replicating the parameters of the LLM,"

   From A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy (Wang2025)

   Domain 2 2.2 Human Intentional Post-deployment

9. 74.02.02 · Risk Sub-Category

   Malicious Use

   Jailbreak in LLM Malicious Use - Poisoning Training Data

   "In the data collecting and pre-training phase, malicious adversaries can Jailbreak LLMs through poisoning their training data to make the model to output harmful content."

   From A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy (Wang2025)

   Domain 2 2.2 Human Intentional Pre-deployment

10. 74.02.03 · Risk Sub-Category

    Malicious Use

    Jailbreak in LLM Malicious Use - Backdoor Attack

    "However, there are still ones who can leave holes in the training dataset, making LLMs appear safe on average, but generate harmful content under other specific conditions. This kind of attack can be categorized as "backdoor attack". Evan et al. developed a backdoor model that behaves as expected when trained, but exhibits different and potentially harmful behavior when deployed [81]. The results show that these backdoor behaviors persist even after multiple security training techniques are applied."

    From A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy (Wang2025)

    Domain 2 2.2 Human Intentional Pre-deployment

11. 74.02.04 · Risk Sub-Category

    Malicious Use

    Jailbreak in LLM Malicious Use - White & Black Box Attacks

    "In the fine-tuning and alignment phase, elaborately- designed instruction datasets can be utilized to fine-tune LLMs to drive them to perform undesirable behaviors, such as generating harmful information or content that violates ethical norms, and thus achieve a jailbreak. Based on the accessibility to the model parameters, we can categorize them into white-box and black-box attacks. For white-box attacks, we can jailbreak the model by modifying its parameter weights. In [107], Lermen et al. used LoRA to fine-tune the Llama2’s 7B, 13B, and 70B as well as Mixtral on AdvBench and RefusalBench d

    From A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy (Wang2025)

    Domain 2 2.2 Human Intentional Pre-deployment

12. 74.02.05 · Risk Sub-Category

    Malicious Use

    Jailbreak in LLM Malicious Use - Prompt Attacks

    "In the prompting and reasoning phase, dialog can push LLMs into confused or overly compliant states, raising the risk of producing harmful outputs when confronted with harmful questions. Most of the jailbreak methods in this phase are black-boxed and can be categorized into four main groups based on the type of method: Prompt Injection [154], Role Play, Adversarial Prompting, and Prompt Form Transformation."

    From A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy (Wang2025)

    Domain 2 2.2 Human Intentional Post-deployment

Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.