Security Threat Model: Enterprise AI Agent Workflows

Scope

This threat model covers AI Agent workflows that read documents, reason over business context, call tools, draft artifacts, or assist users in enterprise operations.

Assets

• Source documents and policy repositories.
• System-of-record data.
• Credentials and tokens.
• User prompts and uploaded files.
• Agent outputs and intermediate artifacts.
• Audit logs and approval records.

Trust Boundaries

1. External input boundary: PDFs, spreadsheets, web pages, emails, chat text, and user uploads.
2. Model boundary: prompts, retrieved context, tool responses, and generated output.
3. Tool boundary: connectors, APIs, browser automation, and file operations.
4. Human approval boundary: final decision, customer communication, posting, onboarding, or financial action.

Prompt Injection

Untrusted documents or user text may instruct the agent to ignore policy, leak data, or call unauthorized tools.

• Treat retrieved content as data, not instructions.
• Isolate document reading from write-capable workers.
• Use typed outputs and validation.
• Apply tool allowlists and least privilege.

Over-automation

The system may perform a responsibility action before a human review.

• Define non-automatable actions.
• Require human approval for high-impact steps.
• Separate draft generation from final execution.
• Log reviewer identity and approval status.

Data Leakage

Sensitive content may be exposed to the wrong model, user, connector, or downstream artifact.

• Apply access control before retrieval.
• Keep secrets outside prompts.
• Restrict tool permissions by worker role.
• Redact or tokenize sensitive data where possible.

Hallucinated Evidence and Quality Drift

• Require source references for knowledge workflows.
• Validate citations against retrieved source IDs.
• Escalate when sources are missing or conflicting.
• Maintain golden test cases and version prompts, tools, and indexes.
• Monitor human correction and escalation trends.

Minimum Audit Events

• workflow_id, user_id, agent_worker_id, source_ids, prompt_version, tool_calls, model_version, output_artifact_id, reviewer_id, decision_status, timestamp.