PII detection at runtime: gateway, sidecar, or in-process

Which AI security platforms offer PII detection at runtime?

Runtime PII detection means a check that runs while the request is in flight and returns a decision you act on before the next hop. Not a scan of a data store. Not a report the next morning. A verdict in the path of the call.

Three deployment models ship that check, and they aren't interchangeable. A gateway or proxy sits between your application and the model provider. A sidecar runs next to your application as a separate process. An in-process detector runs inside your application, in the same memory space as the handler.

The models differ on one thing that matters more than feature lists: where the text goes to get classified. A gateway has to receive the prompt to inspect it, because it is a network hop with the prompt in the body. An in-process detector classifies the string where the string already is.

The following platforms document a runtime PII check:

Read that fourth column before the third. It's the one that decides whether the control covers your application or a subset of it.

What are the three deployment models?

Gateway or proxy. Your application calls the gateway instead of the provider. The gateway inspects the body, applies policy, and forwards. One place to change policy for every service behind it, and it works regardless of what language those services are written in. The cost is that the gateway is now a recipient of every prompt, and it only sees traffic routed through it.

Sidecar. A separate process on the same host or in the same pod, reached over localhost or a Unix socket. The text stays inside your trust boundary, which resolves the residency question a hosted gateway creates. It's still a network call, still a separate deployment artifact, and still something to keep running and patched.

In-process. The detector is a library your application imports. The text is a variable that gets passed to a function. Nothing crosses a socket to be classified. The cost is that the detection logic ships with your application, so updating it means deploying, and the model has to be small enough to sit inside your process.

What does each model cost in the request path?

Be careful with latency claims here, including ours. The number that matters is the one measured in your own topology, because the dominant term for a gateway is the network distance between your application and the gateway, and that's a fact about your deployment rather than about the product.

What can be stated structurally:

• A gateway adds a full round trip carrying the prompt, on every model call. If the gateway is in a different region from your application, that round trip is measured in tens of milliseconds before any inspection happens.
• A sidecar adds a loopback round trip, which is small, plus the inspection.
• An in-process detector adds the inspection and nothing else.

For Arcjet specifically: the bundled WebAssembly analysis adds under a millisecond. The optional on-device model that extends coverage to names, addresses, and government or financial identifiers adds roughly 6.6 ms median inference on Node.js, from a model of about 14.7 MB when 4-bit quantized. Controls that genuinely need shared state, such as distributed rate limits, call the Arcjet Cloud API at typically 20 ms to 30 ms, and there is one call per decision no matter how many rules ran. Sensitive-information detection is not one of the controls that needs that state, so the text isn't in that payload.

To measure a gateway honestly, time the same prompt three ways from the same host: direct to the provider, through the gateway with inspection off, and through the gateway with inspection on. The first two differences give you the hop; the third gives you the inspection. Reporting only the third is how vendor benchmarks make a proxy look free.

Which paths can each model actually see?

A chat route is the easy case. Every model sees it. The interesting question is what happens on the other paths a production AI application runs.

The pattern isn't that gateways are badly built. It's that a control which inspects network traffic can only inspect traffic that crosses the network. A tool handler that reads a customer record and hands it back to the model does that entirely inside one process. There is no request for a proxy to see.

What does an in-process check look like?

The rule takes a string and returns a decision. That's the whole interface, which is why it works on a path that has no request object:

1
import { launchArcjet, localDetectSensitiveInfo } from "@arcjet/guard";
2

3
const arcjet = launchArcjet({ key: process.env.ARCJET_KEY! });
4

5
const pii = localDetectSensitiveInfo({
6
  deny: ["CREDIT_CARD_NUMBER"],
7
});
8

9
// A queue worker. No HTTP request exists on this path.
10
export async function processTranscript(job: TranscriptJob) {
11
  const decision = await arcjet.guard({
12
    label: "jobs.summarize-transcript",
13
    actor: job.ownerId,
14
    correlationId: job.id,
15
    rules: [pii(job.transcript)],
16
  });
17

18
  if (decision.conclusion === "DENY") {
19
    await quarantine(job);
20
    return;
21
  }
22

23
  await summarize(job.transcript);
24
}

Detection runs in the WebAssembly module inside your process. What reaches the Arcjet Cloud API is the decision record: the entity types that matched, the label, the actor, and a SHA-256 hash of the scanned text. Not the transcript.

That hash is worth naming rather than glossing over. It lets Arcjet deduplicate and correlate decisions without holding content. It is not a privacy-free primitive: a hash of a low-entropy value, such as a phone number, is guessable by anyone who has the hash and a generator. Treat it as an identifier, not as a redaction.

For the residency argument in full, including a control-by-control table of what each Arcjet rule transmits, see keeping security inspection local. For the inbound and outbound detection guide this sits under, see how to detect and redact PII in LLM inputs and outputs, and for the definition of a runtime control, see what is runtime application security.

How do you evaluate detection accuracy?

Vendor accuracy claims are close to meaningless across products, because each one is measured on a corpus the vendor chose. A detector tuned for US financial identifiers will look excellent on a US financial corpus and poor on European addresses.

Measure it on your own traffic instead. The method is unglamorous and takes about a day:

1. Take a sample of real prompts. A few thousand is enough to see the shape.
2. Label them. Not by hand for all of them: label a stratified sample, and use the disagreements between two detectors to surface the interesting cases without labeling everything.
3. Run each candidate detector in dry-run mode against the whole sample so nothing is blocked while you measure.
4. Report precision and recall per entity type, not as one number. Card numbers and names have completely different error profiles, and an aggregate hides that a detector never finds a surname.
5. Look at the false positives by hand. A support product that legitimately collects phone numbers cannot enforce on PHONE_NUMBER, and you'd rather learn that from a dry run than from a support queue.

Arcjet doesn't publish comparative precision and recall against other vendors, because a table like that is only as honest as the corpus underneath it, and we'd be choosing the corpus. Dry-run mode exists so you can produce the number that applies to your traffic.

When is a gateway the right choice?

Three situations where the gateway is the better answer, and pretending otherwise would be useless to anyone deciding:

A polyglot estate. If you have services in five languages and a mandate to apply one policy to all of them, one gateway beats five SDK integrations. An in-process library has to exist for your runtime, and that's a real constraint.

A runtime with no SDK. In-process detection needs a library that runs where your code runs. If your inference path is a Java service and the detector ships for JavaScript and Python, the gateway is what you have.

A centralized policy mandate. Where a security team owns model-access policy and needs to change it without an application deployment, the control plane belongs outside the application. this is a policy-authoring requirement rather than an inspection-location requirement, and the two can be separated. For more information about that split, see application-native versus remote security policies.

The situations where a gateway is the wrong answer on its own are the mirror image: agent workflows where most of the risk is on tool calls, applications with a residency constraint on prompt content, and anywhere the check has to run on a path that never crosses the network.

Layering is normal. A gateway for centralized model-access policy, an in-process check on the tool and retrieval paths it can't see.

How do you choose?

Answer these in order. The first "no" usually decides it.

1. Does the prompt content carry a residency or minimization obligation? If yes, a hosted gateway adds a recipient of exactly the data you are trying to control. A sidecar or in-process detector doesn't.
2. Is the risk on the chat route, or on tool calls and retrieval? If it's the second, network-path controls cover a fraction of it.
3. Does a library exist for your runtime? If not, the decision is made for you.
4. Who needs to change the policy, and how fast? A security team that needs same-hour changes across services needs a control plane, whichever way inspection runs.
5. What happens when the detector is unavailable? A gateway that fails closed takes your product down. A gateway that fails open forwards the prompt unscanned. Decide this per route rather than globally. For more information about failure behavior and staged rollout, see PII detection for production AI applications.

The honest summary: in-process detection wins on coverage and on data egress, gateways win on centralization and language reach, and most production systems end up with both. The mistake is assuming one of them covers the paths it structurally can't see.

For a vendor-by-vendor comparison across the wider control set rather than PII alone, see the AI agent security platform comparison.