MIT AI Risk Repository · Risk Sub-Category · 24.03.02

AI-Powered Spear-Phishing at Scale

Category: Malicious Uses

Open the MIT Risk Navigator All entries from Gabriel2024

Description

"Phishing is a type of cybersecurity attack wherein attackers pose as trustworthy entities to extract sensitive information from unsuspecting victims or lure them to take a set of actions. Advanced AI systems can potentially be exploited by these attackers to make their phishing attempts significantly more effective and harder to detect. In particular, attackers may leverage the ability of advanced AI assistants to learn patterns in regular communications to craft highly convincing and personalized phishing emails, effectively imitating legitimate communications from trusted entities. This tec

From The Ethics of Advanced AI Assistants (Gabriel2024), as extracted by the MIT AI Risk Repository (CC BY 4.0).

Classification

Domain: 4. Malicious actors
Subdomain: 4.1 Disinformation, surveillance, and influence at scale
Causal entity: Human
Intent: Intentional
Timing: Post-deployment

Subdomain definition: Using AI systems to conduct large-scale disinformation campaigns, malicious surveillance, or targeted and sophisticated automated censorship and propaganda, with the aim to manipulate political processes, public opinion and behavior.

Real-world incidents in this subdomain

Browse all incidents in this subdomain

How other frameworks describe this risk

Coercion/manipulation
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Electoral interference
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Political manipulation
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Political and Economic
A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms (Abercrombie2024)
Biased influence through citizen screening and tailored propaganda
The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)
Political
The Rise of Artificial Intelligence - Future Outlooks and Emerging Risks (Allianz2018)
Surveillance and Censorship
Foundational Challenges in Assuring Alignment and Safety of Large Language Models (Anwar2024)
Disinformation and manipulation of public opinion
International Scientific Report on the Safety of Advanced AI (Bengio2024)

Other entries from Gabriel2024

Capability failures Risk Category · 7.3
Lack of capability for task Risk Sub-Category · 7.3
Difficult to develop metrics for evaluating benefits or harms caused by AI assistants Risk Sub-Category · 6.5
Safe exploration problem with widely deployed AI assistants Risk Sub-Category · 7.3
Goal-related failures Risk Category · 7.1
Misaligned consequentialist reasoning Risk Sub-Category · 7.3
Specification gaming Risk Sub-Category · 7.1
Goal misgeneralisation Risk Sub-Category · 7.1
Deceptive alignment Risk Sub-Category · 7.1
Malicious Uses Risk Category · 4.0
Offensive Cyber Operations (General) Risk Sub-Category · 4.2
AI-Assisted Software Vulnerability Discovery Risk Sub-Category · 4.2