Chapter 5B focuses on the AI-specific attack surface: prompt injection, malicious context, insecure tool use, data exfiltration, model misuse, agent-to-agent propagation, and unsafe output handling. The objective is to contain model manipulation before it reaches enterprise state.
5B.1 Threat landscape
| Threat | Attack objective | Primary control family |
|---|---|---|
| Direct prompt injection | Override instructions or bypass intended workflow | Task constraints, policy, tool restrictions, output validation |
| Indirect prompt injection | Use documents, email, web, or tickets to manipulate the agent | Content trust labels, context isolation, capability authorization |
| Excessive agency | Exploit broad tools, permissions, or autonomy | Least privilege, bounded capabilities, approval, limits |
| Sensitive data disclosure | Extract protected context, memory, prompts, or records | ACL-aware retrieval, DLP, field filtering, egress control |
| Insecure output handling | Turn generated content into code, query, command, or injection | Schema validation, escaping, domain validation, safe adapters |
| Knowledge poisoning | Insert false or malicious content into retrieval sources | Source approval, lineage, versioning, integrity monitoring |
| Agent propagation | Spread malicious context or decisions across agents | Context isolation, structured handoffs, independent policy |
5B.2 Prompt injection defense
Prompt injection is not solved by a stronger system prompt. The architecture must assume that model instructions and untrusted content share the same interpretation channel. The defense therefore limits what the model can access and what downstream systems will accept.
- Mark source trust and authority explicitly.
- Separate instructions from retrieved evidence in the context structure.
- Do not pass entire documents when only a relevant section is needed.
- Do not let retrieved content modify tool permissions or policy.
- Require independent authorization for every side effect.
- Log suspicious instructions and denied capability attempts as security events.
5B.3 Tool and capability security
Tool security is where AI-specific manipulation becomes conventional enterprise risk. The safest tool is a narrow business capability with explicit schema, owner, resource scope, risk tier, idempotency, and post-condition verification.
| Tool pattern | Risk | Preferred replacement |
|---|---|---|
| Arbitrary SQL | Data exfiltration, mutation, expensive queries | Approved query service or semantic API |
| Shell execution | System compromise and lateral movement | Versioned runbook capability |
| Unrestricted HTTP | SSRF, data exfiltration, malicious destinations | Destination-specific connector with allowlist |
| Broad file access | Secret and personal-data disclosure | Scoped document retrieval service |
| Raw administrative API | Excessive functionality and permissions | Business-level operation with policy |
Security rule: never make a dangerous primitive safe only by describing acceptable use in natural language.
Remove or isolate the primitive and expose a constrained capability instead.
5B.4 Data protection and exfiltration controls
Apply tenant, role, field, classification, purpose, and geography controls before context assembly.
Pass only relevant fields and sections; avoid broad record dumps and secret material.
Use DLP, redaction, egress policy, response classification, and safe destination rules.
Data-loss prevention should cover prompts, retrieved context, tool parameters, model outputs, logs, and external connectors. The model should not be trusted to decide whether information is confidential. Classification and field-level rules must be deterministic.
5B.5 Model and supply-chain security
Model deployments, agent frameworks, plugins, MCP servers, container images, libraries, and prompt assets are part of the software supply chain. Organizations should maintain approved registries, signed artifacts, dependency inventories, vulnerability scanning, and change control.
5B.6 Agent isolation and secure handoffs
Multi-agent systems should use separate context and permission scopes. Agents exchange structured results, source references, and task metadata—not unrestricted conversation histories. Each receiving agent should re-evaluate policy for its own capabilities.
{
"from_agent":"incident-analysis-agent",
"to_agent":"change-planning-agent",
"task_id":"task_88421",
"result":{"root_cause":"configuration regression"},
"evidence_refs":["trace:tr_441","deployment:rel_822"],
"allowed_next_capabilities":["service.change.prepare"],
"sensitive_fields_redacted":true,
"policy_decision_id":"pd_10933"
}5B.7 Secure output handling
Generated content must be treated as untrusted input to downstream systems. HTML must be escaped, URLs validated, SQL parameterized, files scanned, and structured payloads checked against schemas and business rules. Never execute generated code or commands directly in a privileged environment.
| Output | Required handling |
|---|---|
| HTML / Markdown | Sanitize active content, scripts, links, and embedded resources |
| SQL | Prefer query API; otherwise parameterize, restrict, and run read-only |
| Code | Static analysis, sandbox, tests, human review, signed pipeline |
| Commands | Map to approved runbooks; prohibit arbitrary execution |
| JSON / YAML | Schema, type, range, identifier, and domain validation |
| Files | Type verification, malware scan, content inspection, size limits |
5B.8 AI-specific security checklist
5B.9 Summary
Chapter 5B contains the AI-specific threat surface by limiting context, agency, tools, outputs, and agent propagation. Prompt injection is treated as an expected input condition, not an exceptional model failure.
Core conclusion: the safest response to model manipulation is a system in which the model never possesses enough authority to turn manipulation directly into enterprise impact.