A retrieved document can contain the information an enterprise assistant needs and, in the same paragraph, instructions it must never obey. That distinction matters because retrieval systems and tool-connected agents routinely bring outside text into a conversation. A policy document, ticket comment, search result, repository README, or vendor page is evidence for a task—not a source of authority over the task.
The OWASP Top 10 for LLM Applications 2026, released August 4, places prompt injection at LLM01 and excessive agency at LLM03. The pairing is useful: malicious instructions in untrusted content become materially more dangerous when an application can also send messages, change records, or call privileged tools. The engineering question is therefore not simply whether a filter recognizes suspicious text. It is what the system would permit even if the filter missed it.
Why the document boundary is easy to cross
Consider a procurement assistant asked to compare renewal terms in supplier documents. One uploaded contract includes a sentence formatted as an operational directive: send the full comparison and account details to an outside mailbox before answering. The contract is relevant to the user’s question, but the directive is not an authorized request. If the assistant treats all retrieved words as equally authoritative, the supplier document has silently gained control over the workflow.
This is an illustrative scenario, not a claim about a documented breach. The attack surface includes retrieval snippets, browser results, tool outputs, support tickets, and content pasted by other users. The distinction is about provenance and authority, not whether the text looks convincing. A genuine document may contain hostile instructions; a trusted search connector may retrieve an untrusted page.
Separate facts, instructions, and permissions
A useful control design divides the system into three layers. The evidence layer contains retrieved material and its source. The instruction layer contains the user’s authorized objective and the application’s operating rules. The permission layer independently decides which external actions are allowed. These layers should communicate, but a document must not be able to promote itself from evidence to instruction or permission.
Documents, search results, tickets
May inform an answer
User intent and application policy
Define the task
Independent tool authorization
Controls side effects
Boundary rule: text retrieved in layer 1 cannot modify layer 2 or expand layer 3.
Do not assume a guardrail covers every channel
Amazon’s Bedrock Guardrails documentation describes prompt-attack filtering and how application developers tag user-input regions. It also states an important limitation: the prompt-attack filter does not evaluate content in messages[].content[].toolResult or tool definitions. That is a product-specific documented boundary, not a statement that every deployment has identical behavior.
For an enterprise architecture review, ask exactly where a detector runs: on user messages, retrieved chunks, tool responses, outbound arguments, or some subset. A control that scans only one path cannot justify a claim of comprehensive protection. Detection should be tested on the actual paths the application uses, with expected false-positive and false-negative behavior documented.
Put the final decision outside the model
Before an assistant sends an email, changes a supplier record, downloads sensitive material, or invokes a financial transaction, a conventional authorization service should verify the requesting identity, resource, action, scope, and destination. A retrieved page should never be able to add a recipient or authorize an export merely by containing persuasive language. For sensitive actions, use an explicit approval step tied to the exact proposed action and parameters.
This complements the four control planes for enterprise agents: identity, authorization, outbound access, and audit. It also explains why a successful answer is not sufficient evidence of a safe workflow. The system might have produced a correct summary while also attempting an unauthorized tool call.
Test the boundary, not just the final answer
Build a small adversarial test collection around your real workflows. Insert instructions into a retrieved policy, a web page, a ticket, and a tool response. Ask whether the assistant notices conflicting instructions, whether it attempts a restricted action, and whether the authorization layer blocks the attempt. Include benign documents that mention commands as quoted examples; rejecting all such documents would damage legitimate retrieval.
Measure at least four outcomes separately: unauthorized action attempts, blocked attempts, unauthorized actions completed, and legitimate tasks wrongly refused. Track the source channel and the exact tool boundary. A zero count of completed unauthorized actions is encouraging only if tests exercised the relevant capabilities and the logs captured the attempts.
For rollout decisions, the enterprise agent evaluation framework provides a complementary approach to autonomy gates, while the third-party security-agent control framework helps evaluate supplier promises against operational evidence.
What an architecture review should require
Require a diagram showing every entry point for untrusted content; a list of privileged tools and their independent authorization checks; a test record covering malicious and harmless examples; a destination allowlist where appropriate; and an incident path for disabling tool access. Preserve enough event metadata to reconstruct an attempted action without indiscriminately retaining confidential prompt contents.
The practical standard is simple: even if a malicious instruction reaches the model, the surrounding system should keep it from becoming an unauthorized action. Better filtering helps, but authority boundaries determine how far a missed injection can travel.
References
- OWASP GenAI Security Project, Top 10 for LLM Applications 2026, released August 4, 2026.
- Amazon Web Services, Detect prompt attacks with Amazon Bedrock Guardrails, accessed October 17, 2026.
- OWASP GenAI Security Project, Top 10 for Agentic Applications 2026, December 9, 2025.
Leave A Comment