The 3-Bucket Threat Model
Shadow AI, attacks on AI, and agentic tool-use attacks — the framework that organizes all of AI security.
Everything in AI security falls into three primary categories:
Bucket 1: AI Access / Shadow AI
“Employees using AI in ways the company can’t see, with data they shouldn’t be sharing.”
Examples:
- Engineers pasting proprietary code into ChatGPT to debug
- HR pasting candidate data into AI summarizers
- Sensitive docs uploaded to free AI tools
- Personal AI accounts being used with corporate data
Mental model: This is DLP (Data Loss Prevention) for the AI era.
Defense focus: Visibility, governance, browser/endpoint inspection.
Bucket 2: Attacks on the AI Itself
“Adversaries manipulating the LLM to misbehave.”
Examples:
- Prompt injection (direct + indirect)
- Jailbreaks (bypassing safety training)
- Model manipulation (adversarial inputs)
- Data poisoning (corrupting training data or RAG corpora)
Mental model: Classical ML adversarial attacks, applied to LLMs.
Defense focus: Input/output classifiers, content filters.
Bucket 3: Agentic / Tool-Use Attacks (Frontier Risks)
“Adversaries exploiting agents that take actions on behalf of users.”
Examples:
- Tool abuse (tricking agents into calling dangerous tools)
- MCP server compromise (malicious tools)
- Agent hijacking (subverting agent goals mid-task)
- Excessive agency (over-permissioned agents)
- Confused deputy (agent acting on attacker’s behalf)
Mental model: Combines prompt injection with tool execution. Damage scales with the capabilities and permissions of the agent.
Defense focus: Runtime inspection, behavioral monitoring, authorization layers.
Why Bucket 3 Carries the Greatest Blast Radius
Each bucket scales in damage potential:
| Bucket | Damage Potential | Reason |
|---|---|---|
| 1. Shadow AI | Data leak | One-time leak, often recoverable |
| 2. Attack on LLM | Misinformation, info leak | Bad answers, limited blast radius |
| 3. Attack on Agent | Real-world action taken | Agent can delete files, send money, exfiltrate at scale |
Key insight: An LLM attack changes what the AI says. An agent attack changes what the AI does. That distinction is the entire reason agentic AI security is a separate, important category.