MIT AI Risk Repository · domain 2: Privacy & Security
2.2 AI system security vulnerabilities and attacks
Vulnerabilities in AI systems, software development toolchains, and hardware that can be exploited, resulting in unauthorized access, data and privacy breaches, or system manipulation causing unsafe outputs or behavior.
- 112
- 12
- 24
- 20
| Label | Value |
|---|---|
| Human | 87 |
| Other | 18 |
| AI | 6 |
| Not coded | 1 |
| Label | Value |
|---|---|
| Intentional | 83 |
| Unintentional | 16 |
| Other | 12 |
| Not coded | 1 |
| Label | Value |
|---|---|
| Post-deployment | 64 |
| Pre-deployment | 25 |
| Other | 22 |
| Not coded | 1 |
| Label | Value |
|---|---|
| 2017 | 3 |
| 2019 | 1 |
| 2022 | 2 |
| 2023 | 3 |
| 2024 | 3 |
| 2025 | 5 |
| 2026 | 7 |
| Label | Value |
|---|---|
| Risk Category | 16 |
| Risk Sub-Category | 96 |
Risk entries
Browse and export all- Privacy - Prompt Inversion Attack (PIA)
"stealing the private prompting texts"
- Privacy - Attribute Inference Attack (AIA)
"deducing the private or sensitive information from training texts, prompting texts or external texts"
- Privacy - Model Extraction Attack (MEA)
"replicating the parameters of the LLM,"
- Jailbreak in LLM Malicious Use - Poisoning Training Data
"In the data collecting and pre-training phase, malicious adversaries can Jailbreak LLMs through poisoning their training data to make the model to output harmful content."
- Jailbreak in LLM Malicious Use - Backdoor Attack
"However, there are still ones who can leave holes in the training dataset, making LLMs appear safe on average, but generate harmful content under other specific conditions. This kind of attack can be...
- Jailbreak in LLM Malicious Use - White & Black Box Attacks
"In the fine-tuning and alignment phase, elaborately- designed instruction datasets can be utilized to fine-tune LLMs to drive them to perform undesirable behaviors, such as generating harmful informa...
- Jailbreak in LLM Malicious Use - Prompt Attacks
"In the prompting and reasoning phase, dialog can push LLMs into confused or overly compliant states, raising the risk of producing harmful outputs when confronted with harmful questions. Most of the...
- Vulnerability of AI systems to attacks and misuse
- On Purpose - Pre-Deployment
"During the pre-deployment development stage, software may be subject to sabotage by someone with necessary access (a programmer, tester, even janitor) who for a number of possible reasons may alter s...
- Security risks (confidentiality)
- Security risks (integrity)
- Adversarial attack
"Recent advances have shown that a deep learning model with high predictive accuracy frequently misbehaves on adversarial examples [57,58]. In particular, a small perturbation to an input image, which...