A language model processes instructions and data in the form of text or multimodal content. This flexibility creates a fundamental ambiguity: a consulted document may contain a sentence that resembles an instruction. An agent can then interpret external data as a command, attempt to access a tool, or reveal information.
Prompt injection is not solved by a firmer system prompt. The model remains a probabilistic component. Security consists of limiting its powers, isolating sources, verifying actions, and preventing a misinterpretation from producing an impact.
Direct and indirect injection
A direct injection is entered by the user in the conversation to bypass the rules. An indirect injection is hidden in an email, a web page, a document, an image, or tool output that the system consults.
Indirect injection is particularly important for agents: malicious content can come from a source that the user believes to be legitimate. The model must not grant it the same level of trust as system policies.
The possible consequences
According to the available tools and data, an injection can cause:
- disclosure of contextual information;
- sending to an external destination;
- unauthorized action;
- data modification;
- memorization of a persistent instruction;
- abusive use of a resource;
- misleading response;
- circumvention of a validation;
- saturation or excessive spending.
A chatbot without tools has a smaller sphere of impact than an agent capable of sending emails or executing code. Risk is measured by powers, not just by the likelihood of a bad response.
Mapping the boundaries of trust
The diagram must distinguish:
- system instructions and policies;
- user input;
- retrieved content;
- memory;
- model outputs;
- tools;
- secrets;
- external services;
- human validations.
Each stream specifies its origin, its level of trust, its transformations, and the actions it can influence. Unreliable content should never modify a policy or a list of tools.
Independent defense layers limiting the consequences of a prompt injection.
Separate instructions and data
The system clearly delineates the retrieved content and indicates to the model that it is not instructions. This technique reduces certain risks, but it is not a guarantee.
True protection comes from the architecture: even if the model follows a malicious instruction, it does not have the access or permission necessary to produce the impact.
The outputs of a model are themselves unreliable when they feed a tool. They must be validated like any external input.
Reduce the context
The more data the model receives, the greater the exposure surface and the risk of leakage. The RAG must retrieve only the necessary and authorized passages. Secrets are not added to the context for convenience.
Memory retains the minimum and has a duration. Conversations of multiple users or files are isolated. Old data is read again from its source before an action.
Design tools with least privilege
A tool must:
- to perform a specific action;
- verify the identity;
- validate a strict schema;
- limit the scope;
- apply business rules;
- to be idempotent;
- to log;
- return a structured result.
Generic shell, SQL, free browser, or arbitrary HTTP request tools are prohibited by default. When code must be executed, it is done in an isolated, ephemeral environment with no unnecessary network access.
Control the destinations
Sending, downloading, and publishing capabilities are exfiltration channels. Recipients, domains, buckets, URLs, and file types are restricted by an independent policy.
The agent cannot create an authorized destination by themselves. New destinations require a separate administrative action.
Protecting secrets
The keys are not present in the prompt nor returned by the tools. A server-side service makes the call and limits the operations. The logs mask secrets and sensitive parameters.
Technical accounts have minimal permissions, rotation, and monitoring. An alleged leak triggers a rapid revocation.
Validate the structured outputs
A tool call must follow a schema: types, values, formats, lengths, and relationships. Unknown fields are rejected. Critical values are recalculated or reread from the source.
A valid JSON is not necessarily allowed. The policy engine checks user, resource, amount, destination, and context.
Add proportional approvals
Reversible and low-risk actions can be executed automatically. Sensitive actions are previewed and confirmed. High-impact actions require independent approval.
Validation is linked to the exact parameters and expires. An external instruction cannot produce implicit approval.
Isolate the navigation and the code
An agent that browses the web or executes code must use a sandbox with:
- ephemeral file system;
- limited network;
- limited time and resources;
- no master key;
- controlled downloads;
- filtered result;
- activity log.
Active documents, scripts, and macros are not executed in the main context.
Secure the memory
An injection may require recording a rule for future conversations. Memory writings are separate, limited, and sometimes subject to validation. Each item has origin, date, owner, and scope.
The system can distinguish user preferences, verified facts, and generated summaries. A model output does not become lasting truth without rules.
Check the connectors and MCP
A remote tool server is a provider of code and data. It must be inventoried, authenticated, assessed, and restricted. The metadata of a tool is not automatically reliable.
Tokens are specific to the server and the user. Redirects, consents, capability changes, and updates are monitored. One server cannot request secrets intended for another.
Detect without relying on detection
Filters can detect certain formulations, areas, or behaviors. They are useful for alerts, but an attacker can vary the form. The policy must remain safe even if detection fails.
Volume, destination, tool, cost, and rejection anomalies are monitored. A cutoff mechanism quickly disables a capacity.
Test in an adversarial manner
Red teaming covers:
- unreliable content in each source;
- attempts to change role;
- requests for secrecy;
- chain tools;
- encoded data;
- memory;
- languages and formats;
- errors and timeouts;
- approvals;
- destinations.
Tests must be authorized, isolated, and focused on controls. Detected cases become automated regressions without retaining sensitive data.
Prepare the incident
The plan includes cutting off the tools, revoking tokens, preserving evidence, analyzing actions, notification, and restoration. The log links identity, content, model, tools, settings, and outcome.
A model error can trigger a business action; the security team and the product team must therefore share the procedures.
Accept the residual risk
No combination guarantees that a model will never follow an instruction. The goal is that this error does not allow a forbidden action or a significant leak.
Some capabilities remain too risky and must be excluded, limited to a proposal, or reserved for an isolated environment.
Security comes from borders
A safe system considers the model as an untrusted interpreter, useful but limited. Policies, rights, validations, and tools remain deterministic and auditable.
Partitech can carry out threat modeling, design controls, secure connectors, and implement agent and RAG tests. Defense in depth protects the system even when the model makes a mistake.
Let's talk about your project
Audit the security of your RAG or your agents with Partitech. Contact Partitech.