Prompt injection

Why it matters

• Agents that read email, browse the web or call tools turn a successful injection into real actions: data exfiltration, transactions or code changes.
• Resilience against manipulated inputs is a legal requirement for high-risk AI systems under the EU AI Act (Article 15) and a standard red-team test case.
• It cannot be patched once and closed: it needs design controls and recurring testing in the release process.

In detail

Direct prompt injection comes from the user's own input (often called jailbreaking); indirect prompt injection arrives through external content the model processes, such as web pages, documents, emails or tool results. Because instructions and data share one channel, filtering cannot fully prevent it; mitigations limit what a successful injection can do: least-privilege tools, separating untrusted content, validating outputs and human approval for consequential actions. Exposure of the system prompt itself is a separate OWASP entry (LLM07 System Prompt Leakage).

Related concepts

• Agentic AIAgentic AI is an AI system that pursues a goal by planning and taking actions with some autonomy, such as call...
• Red teaming (AI)AI red teaming is structured adversarial testing in which testers try to make an AI system fail, by producing...
• OWASP Top 10 for LLM ApplicationsThe OWASP Top 10 for LLM Applications is the Open Worldwide Application Security Project's list of the ten mos...