Browse AI risks

5 entries

1. 73.07.00 · Risk Category

   Jailbreaks and Prompt Injections Threaten Security of LLMs

   "LLMs are not adversarially robust and are vulnerable to security failures such as jailbreaks and prompt-injection attacks. While a number of jailbreak attacks have been proposed in the literature, the lack of standardized evaluation makes it difficult to compare them. We also do not have efficient white-box methods to evaluate adver- sarial robustness. Multi-modal LLMs may further allow novel types of jailbreaks via additional modalities. Finally, the lack of robust privilege levels within the LLM input means that jailbreaking and prompt-injection attacks may be particularly hard to eliminate

   From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

   Domain 2 2.2 Other Other Other

2. 73.07.01 · Risk Sub-Category

   Jailbreaks and Prompt Injections Threaten Security of LLMs

   Exploiting Limited Generalization of Safety Finetuning

   "Safety tuning is performed over a much narrower distribution compared to the pretraining distribution. This leaves the model vulnerable to attacks that exploit gaps in the generalization of the safety training, e.g. using encoded text (Wei et al., 2023c) or low-resource languages (Deng et al., 2023a; Yong et al., 2023) (see also Section 3.2)."

   From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

   Domain 2 2.2 Other Unintentional Other

3. 73.07.02 · Risk Sub-Category

   Jailbreaks and Prompt Injections Threaten Security of LLMs

   “Model Psychology” Attacks

   "LLMs are vulnerable to “psychological” tricks (Li et al., 2023e; Shen et al., 2023), which can be exploited by attackers. Examples include instructing the model to behave like a specific persona (Shah et al., 2023; Andreas, 2022), or employing various “social engineering” tricks crafted by humans (Wei et al., 2023c) or other LLMs (Perez et al., 2022b; Casper et al., 2023c)."

   From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

   Domain 2 2.2 Human Intentional Post-deployment

4. 73.07.04 · Risk Sub-Category

   Jailbreaks and Prompt Injections Threaten Security of LLMs

   Attacking LLMs via Additional Modalities a

   "LLMs can now process modalities other than text, e.g. images or video frames (OpenAI, 2023c; Gemini Team, 2023). Several studies show that gradient-based attacks on multimodal models are easy and effective (Carlini et al., 2023a; Bailey et al., 2023; Qi et al., 2023b). These attacks manipulate images that are input to the model (via an appropriate encoding). GPT-4Vision (OpenAI, 2023c) is vulnerable to jailbreaks and exfiltration attacks through much simpler means as well, e.g. writing jailbreaking text in the image (Willison, 2023a; Gong et al., 2023). For indirect prompt injection, the atta

   From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

   Domain 2 2.2 Human Intentional Post-deployment

5. 73.08.00 · Risk Category

   Vulnerability to Poisoning and Backdoors

   "The previous section explored jailbreaks and other forms of adversarial prompts as ways to elicit harmful capabilities acquired during pretraining. These methods make no assumptions about the training data. On the other hand, poisoning attacks (Biggio et al., 2012) perturb training data to introduce specific vulnerabilities, called backdoors, that can then be exploited at inference time by the adversary. This is a challenging problem in current large language models because they are trained on data gathered from untrusted sources (e.g. internet), which can easily be poisoned by an adversary (

   From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)

   Domain 2 2.2 Human Intentional Pre-deployment

Informational only, not legal advice. Verify every claim against the linked official sources and consult qualified counsel before acting.