Enterprise AI abuse cases and attack paths

6B.1 Attack surface inventory

6B.2 Abuse case — Indirect prompt injection

1. Attacker inserts hidden or visible instructions into a document, email, ticket, or webpage.
2. The content is retrieved as evidence for a legitimate task.
3. The model interprets the malicious text as an instruction.
4. The agent requests a capability or external destination not required by the task.
5. If policy and egress controls are weak, data is exposed or enterprise state is changed.

6B.3 Abuse case — Sensitive data exfiltration

An attacker uses repeated questions, tool calls, memory retrieval, or external destinations to reconstruct protected information. The threat may come from a compromised user, malicious insider, poisoned content, or over-permissioned agent.

• Enumerate accessible sources and infer schema.
• Request progressively broader context.
• Exploit summaries or embeddings that preserve hidden content.
• Encode data in URLs, tool parameters, generated files, or logs.
• Use cross-tenant cache, memory, or telemetry leakage.

Mitigation requires pre-retrieval authorization, field minimization, DLP, output classification, destination control, query-volume analytics, and tenant-isolated stores.

6B.4 Abuse case — Excessive agency

A legitimate or compromised user gives a broad goal. The agent decomposes it into high-impact actions using powerful tools, weak approval, or long-running autonomy. No single step appears malicious, but the combined behavior creates unacceptable impact.

Controls include task classes, action budgets, resource limits, maximum step counts, explicit approvals, capability-specific identities, post-condition verification, and emergency suspension.

6B.5 Abuse case — Tool parameter manipulation

The attacker does not need to invoke an unauthorized tool if they can manipulate target identifiers, amounts, recipients, query filters, or destinations within an allowed capability. The threat model must therefore cover parameter integrity, not only tool names.

{
  "allowed_capability":"crm.account.note.create-draft",
  "expected_account":"AC-1092",
  "model_supplied_account":"AC-8821",
  "attack":"identifier substitution",
  "required_control":[
    "bind resource scope to policy decision",
    "validate target against user relationship",
    "display target in approval",
    "verify exact target after execution"
  ]
}

6B.6 Abuse case — Knowledge poisoning

An attacker modifies a source, uploads a malicious document, alters metadata, or exploits weak ingestion governance. The poisoned content may remain dormant until retrieved for a high-impact task.

6B.7 Abuse case — Cross-agent propagation

A compromised or manipulated agent passes malicious instructions, false conclusions, or sensitive data to another agent. The receiving agent may possess stronger tools or a different trust level.

Secure handoffs should contain structured results, evidence references, sensitivity labels, and allowed next capabilities. The receiving agent must not inherit the sender’s permissions and should independently evaluate policy.

6B.8 Abuse case — Denial of service and economic exhaustion

• Recursive planning or agent handoff loops.
• Oversized documents or repeated multimodal processing.
• High-cardinality retrieval and ranking.
• Tool retries against unavailable dependencies.
• Expensive model routing triggered by crafted prompts.
• Queue saturation through event replay or duplicate requests.

Controls include quotas, task budgets, maximum context size, timeouts, concurrency limits, circuit breakers, model cost policies, deduplication, and tenant-level isolation.

6B.9 Risk scoring

6B.10 Abuse-case checklist

6B.11 Summary

Chapter 6B converts the architectural threat model into concrete abuse cases. It shows how prompt injection, excessive agency, data exfiltration, parameter manipulation, poisoning, propagation, and denial of service can move through the system.