Runtime Security for LLM Applications: Prompt Injection, Data Exfiltration, Unsafe Actions

What is runtime security for LLM applications?

Runtime security for LLM applications is the enforcement of security policy while the application is running: at the moment untrusted input reaches the model, and at the moment the model's output turns into an action. It uses live application context to allow, block, redact, or limit an operation before that operation completes. That context includes the authenticated user, the tool being called, the arguments, the target object, and the cost accumulated so far.

LLM applications fail at runtime in three recognizable ways: prompt injection that produces unauthorized side effects, data exfiltration, and unsafe or unauthorized actions. Each needs a different control. Collapsed into a single "AI safety" bucket, they tend to become nobody's responsibility. The controls that you buy then address whichever failure mode the vendor happens to sell against.

What are the three runtime failure modes?

Each failure mode has a distinct trigger, a distinct piece of evidence, and a distinct control. The following table is the short version, and the sections after it work through each one.

The common thread is that none of these are detectable from the response text alone. A model can produce entirely reasonable-looking output while the tool call underneath it transfers money to the wrong account.

How do you prevent prompt injection in production LLM applications?

Prompt injection is untrusted input overriding your instructions. The risk isn't an odd or embarrassing response. It's the downstream action – a tool call, a data movement, or a spend – that the redirected application then performs.

The first control is evaluating input with a specialist detection model before the provider call. The second, and the one more often missed, is treating tool outputs fed back to the model as untrusted too. This is the difference between a conventional API and an agent workflow: in a normal API, JSON returned from a service is data that you parse and use. In an agent workflow, that same JSON becomes context, and anything in it can function as an instruction. A search result, a fetched web page, a database row containing user-supplied text, or a response from a third-party MCP server are all re-entry points for injection.

The OWASP Top 10 for LLM Applications ranks prompt injection first. Its treatment of indirect injection, where hostile instructions arrive through retrieved content rather than direct user input, maps to exactly this re-entry problem. For framework wiring for LangChain, LlamaIndex, and the Vercel AI SDK, see prompt injection protection for those stacks.

Arcjet's prompt injection detection adds roughly 100 ms and has a dry-run mode, so that you can measure detection rates against real traffic before switching anything to blocking. The false-positive cost lands on legitimate users mid-conversation, so measure first.

The honest limit: no detector catches everything, and the hardest cases contain no attack pattern at all. A request that reads as a plausible business instruction, arriving through a channel that the application trusts, doesn't look anomalous to a classifier. Detection is one layer. The layer that actually prevents the outcome is authorization at the action, described in a later section.

How do you prevent data exfiltration through AI agents?

Exfiltration is the application moving protected data somewhere it should not go. The destinations are more varied than teams usually plan for: a user-facing response, an argument to an outbound tool call, an application log, a trace sent to an observability vendor, or an embedding written to a vector store that later serves a different tenant.

The control is sensitive-information detection at the boundary, applied to inputs and outputs rather than only to what the user sees. Detection on the response alone misses the log and the embedding, which are frequently where regulated data actually accumulates.

Where that inspection runs is a design decision with compliance consequences. Most security tooling carries a quiet trade-off: to inspect your traffic, it must receive your traffic. A cloud scanner reading request bodies or model prompts means that content, potentially personal or regulated, leaves your environment to be analyzed. Arcjet's sensitive-information detection runs locally, in the same process as your application, so the raw body is never sent to Arcjet. An optional on-device machine learning model extends coverage to names, addresses, national identifiers, and financial identifiers. Inspecting without exporting is the point. For more information, see keeping security inspection local.

How do you stop AI agents taking unsafe or unauthorized actions?

The third failure mode is the application doing something consequential that it shouldn't. Two Arcjet controls address different parts of it.

Budgets. Token-bucket limits keyed on an identity, shared by that identity across tool calls rather than by correlationId. An agent that retries a failing tool 40 times hasn't violated any per-request limit, but has spent 40 times the intended budget. Key on the user (as most samples do) and concurrent runs share the bucket; key on the run ID if you want a per-run cap.

Per-action rules inside the tool handler. This is where you know the tool, the arguments, the target object, the tenant, and the authenticated user. No layer above the handler has all five. A gateway sees a model request. A proxy sees an HTTP call. Only the handler sees that issue_refund is about to run with amount=50000 against an account belonging to a different customer than the one in session.

For the irreversible set (fund transfers, deletions, external communications, and production configuration changes), require a human in your own code. Arcjet returns allow or deny. Human approval is an application pattern you add on top. Keep that set small. If everything requires approval, the approvals get rubber-stamped and the control is theater.

Why is sequential action risk the hard case?

The difficult version of unsafe actions is sequential. Individually permitted operations can combine into a bad outcome: read a customer list, then call an export tool. Each step passes its own authorization check. Nothing in either step looks wrong in isolation, and a control that evaluates one call at a time approves both.

Catching this requires policy that carries what came before into the current decision, including the objects already read, the sensitivity of data already in context, and the tools already invoked in this loop. That's a stateful check. The correlationId field tags the run so that you can reconstruct the sequence; it doesn't change allow or deny. Per-request evaluation, however fast, is insufficient on its own for agent workflows.

Practically, most teams start by identifying the small number of dangerous combinations in their own application rather than attempting general sequence analysis. "Read of bulk customer data followed by any outbound tool" is a pattern that you can reason about in your handler. General-purpose sequence reasoning must not be the thing that stands between an agent and a production database. For more information about that argument, see runtime controls on enterprise systems and anatomy of an agent incident.

Where should runtime controls live?

All three controls belong in the same place: inside the application, in the path of the action.

This is a claim about context, not about vendor preference. At the network edge you see packets, IP addresses, and headers. Inside the application you see the authenticated user, their plan and permissions, the route, the tool being invoked, the arguments, and the request body. A control that can't see whether the caller owns the object being modified can't make an authorization decision about it. How fast it runs and where it sits don't change that.

Runtime controls also need to fail predictably. When a security dependency is slow or unavailable, the application has to decide whether to proceed. A direct Guard call fails open by default: the conclusion is allow with error codes, and hasFailedOpen() tells you that the check didn't complete. Fail-closed keeps the control on and the product down. The right answer differs by action: a search endpoint and a refund endpoint must not fail the same way. That's another reason the decision belongs in code that knows which action is running. Vercel AI SDK and LangChain wrappers fail closed unless you opt into continuing on error. HTTP request checks can fail open when your application can't reach Arcjet's cloud; that behavior is configurable.

How do you protect tool calls, MCP servers, and queue workers?

A large share of agent activity never touches HTTP. Tool handlers receive function arguments. Queue consumers read broker messages. Workflow steps exchange state through a runtime. Background jobs run on a schedule with no request object at all.

Perimeter tooling can't enforce on any of these, because there's no request for it to inspect. This is the structural reason that edge controls leave agent workflows partly uncovered: the traffic never passes through them.

In Arcjet's model, HTTP routes use protect() in the handler and non-HTTP paths use guard() inside the function. That gives you one decision model and one set of records, whether or not HTTP is involved. For more information, see the introduction to security inside the agent loop.

What can runtime security not do?

Being precise about the limits is what makes the rest credible.

Runtime enforcement doesn't replace pre-deployment work. Dependency scanning, model evaluation, and red-teaming find classes of problems that never appear in live traffic. It doesn't replace observability either. When something does go wrong, you reconstruct it from logs and traces, and you tune thresholds with detection tooling before enforcing them.

Runtime detection models have real false-positive and false-negative rates, and those rates move as attacker technique changes. Deploy any control whose output is a probability in dry run first, measure it against your own traffic, and pair it with a deterministic check for the operations that matter most. That deterministic check asks whether this user owns this object, whether this tool is permitted for this task, and whether this spend is within budget. It's the one that holds when the probabilistic layer misses.

Checklist for implementing runtime security

• Classify inputs by trust. Direct user input, retrieved content, and tool output are all untrusted, and tool output is the one most often treated as safe.
• Run prompt-injection detection before the provider call and on external content re-entering context.
• Deploy detection in dry run, measure against real traffic, then enforce.
• Apply sensitive-information detection to outputs, logs, traces, embeddings, and user-facing responses.
• Keep content inspection local where data residency obligations apply.
• Enforce token and spend budgets keyed on an identity at the tool call, shared by that identity rather than by correlationId.
• Authorize each consequential action inside the tool handler, against the authenticated user and the specific target object.
• Identify the dangerous action sequences in your own application. Tag decisions with a correlationId so that you can reconstruct the run; that doesn't deny step 3 because of steps 1 and 2.
• For the small set of genuinely irreversible actions, require a human in your own workflow. Arcjet returns allow or deny, not an approval queue.
• Direct Guard fails open. Check hasFailedOpen() on sensitive tools. Define fail-open and fail-closed behavior per action, not globally.
• Record every decision with enough context to investigate afterwards.

Where Arcjet fits

The three preceding failure modes map onto three layers that teams usually buy separately. Content filters and guardrail vendors address prompt injection. Data-loss prevention tools address exfiltration. Identity and gateway products address what an agent can call.

Arcjet is the in-code enforcement layer that sits underneath all three, in the path of the action. That means budgets against cost explosion, prompt injection detection before the provider call, and sensitive-information detection that runs in your application so that the raw body never leaves your environment. Guards cover non-HTTP paths, so tool calls and queue jobs use the same decision model as routes.

It doesn't replace pre-deployment red-teaming or post-runtime observability. For more information about how those layers divide, see the AI agent security platform category map.

For the production how-to, see how to secure AI agents in production. For the residency argument, see AI security for healthcare and regulated industries. For the buyer page, see the best AI security provider for enterprises. For the full-stack wiring of input injection, retrieved-context leakage, and output validation, see runtime security for LLM applications: prompt injection, data leakage, and output validation.

Learn more: AI runtime protection · Arcjet Guards