MIT AI Risk Repository · Risk Sub-Category · 73.03.02
Cybersecurity
Category: Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs
Description
"LLMs may exacerbate cybersecurity risks in various ways (Newman, 2024). Firstly, LLMs may significantly amplify the effectiveness of deceptive operations aimed at tricking people into disclosing sensitive information or granting adversary access to critical resources. For example, LLMs might prove highly effective at crafting personalized phishing emails or messages at scale that may be harder for an average user to recognize as phishing attempts (Karanjai, 2022; Hazell, 2023). In addition to being directly harmful to the targeted individual, such ‘social engineering’ attacks are often the ba
From Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).
Classification
- Domain
- 4. Malicious actors
- Causal entity
- Human
- Intent
- Intentional
- Timing
- Post-deployment
Subdomain definition: Using AI systems to gain a personal advantage over others such as through cheating, fraud, scams, blackmail or targeted manipulation of beliefs or behavior. Examples include AI-facilitated plagiarism for research or education, impersonating a trusted or fake individual for illegitimate financial benefit, or creating humiliating or sexual imagery.
Real-world incidents in this subdomain
- Italian Mediaset Journalist Safiria Leccese's Image Was Reportedly Used in a Purportedly AI-Generated Fake Loan Scam
- Scammers Reportedly Used AI-Cloned Daughter's Voice to Defraud Bay Area Mother in Fake Kidnapping Call
- Texas Man Arturo Hernandez Allegedly Published AI-Generated Deepfake Pornography Depicting Women in TAKE IT DOWN Act Case
- Guelph, Ontario, Woman Reportedly Lost $14,000 in Purported Deepfake MrBeast Cryptocurrency Scam
- Purportedly AI-Recreated Clips from Beastie Boys' 'Sabotage' Video Reportedly Appeared in FBI Promotional Video Posted by Kash Patel
- Ahmedabad Aadhaar Fraud Racket Reportedly Used Purportedly AI-Generated Deepfakes to Change Businessman's Linked Mobile Number
How other frameworks describe this risk
Other entries from Anwar2024
- Agentic LLMs Pose Novel Risks
- Natural Language Underspecifies Goals
- Goal-Directedness Incentivizes Undesirable Behaviors
- Safety Risks from Affordances Provided to LLM-agents
- Multi-Agent Safety Is Not Assured by Single-Agent Safety
- Foundationality May Cause Correlated Failures
- Groups of LLM-Agents May Show Emergent Functionality
- Collusion between LLM-Agents
- Dual-Use Capabilities Enable Malicious Use and Misuse of LLMs
- Misinformation and Manipulation
- Cybersecurity
- Cybersecurity