How to stop users sending PII to an LLM

How do I stop users from sending PII to an LLM?

You can't stop a user typing a card number into a text box. What you can do is stop the string reaching the provider, and that decision belongs in the handler that makes the provider call.

The word in the question is stop, and it's doing real work. Redaction doesn't stop anything: it transforms the prompt and lets the turn continue, which is the right answer for a resume-rewriting box and the wrong answer for a payments bot. Blocking stops the turn and tells the user something. Warning stops nothing but changes behavior over time. These are three different products, not three settings.

So the question underneath the question is: for each class of data your users might paste, which of those three do you want? That's a policy decision with a UX consequence, and it's the part teams skip.

Which data classes should you block, redact, or warn on?

Work through your own data classes rather than adopting someone else's list. The following table is the shape of the decision, using classes that come up in most products.

Two rules keep this from getting away from you. Block the classes where a legitimate turn never contains them, because there the false-positive cost is near zero. Redact the classes where the model needs the sentence but not the value.

The check is the same call regardless of which branch you take. What differs is what you do with the decision:

1
import { launchArcjet, localDetectSensitiveInfo } from "@arcjet/guard";
2
import { rampart } from "@arcjet/sensitive-info-rampart";
3

4
// SSN needs the on-device model backend. Without it, the rule throws.
5
const arcjet = launchArcjet({
6
  key: process.env.ARCJET_KEY!,
7
  sensitiveInfoBackend: rampart(),
8
});
9

10
// Blocked outright: no legitimate turn in this product contains these.
11
const blockList = localDetectSensitiveInfo({
12
  deny: ["CREDIT_CARD_NUMBER", "SSN"],
13
});
14

15
// Measured but not enforced while you tune the threshold.
16
const watchList = localDetectSensitiveInfo({
17
  deny: ["EMAIL", "PHONE_NUMBER"],
18
  mode: "DRY_RUN",
19
});
20

21
const decision = await arcjet.guard({
22
  label: "chat.user-message",
23
  actor: session.userId,
24
  correlationId: conversationId,
25
  rules: [blockList(message), watchList(message)],
26
});
27

28
if (decision.conclusion === "DENY") {
29
  return refuseTurn();
30
}

Running a live list and a dry-run list side by side is how you move a class from warn to block on evidence rather than on a guess.

Why isn't client-side enforcement enough?

A regex in the browser that greys out the send button is a usability feature. It is not a control.

The reasons are the ordinary ones. Your API accepts requests that didn't come from your form. A user on a slow connection can submit before the script loads. Anyone can open the network tab. And a mobile client ships on its own release cycle, so the "current" policy is whatever version each user last updated to.

There's a subtler failure too. Client-side checks train the team into thinking the problem is handled, so the server-side check never gets written, and the paths that were never going to have a client at all – a queue job, a webhook, an MCP tool – stay uncovered.

Keep the client-side hint. It's genuinely better UX to catch the paste before the user hits send. Just don't count it.

Which paths does a user reach the model through?

The chat box is one entrance. Inventory the others before deciding you're covered:

• File uploads, where the PII is in an attached PDF or spreadsheet rather than the message.
• Voice transcripts, where a caller reads a card number aloud and a transcription service produces the string.
• Email or ticket ingestion, where a customer's message becomes model context without a human ever seeing it.
• Webhooks from other systems, carrying records that were never intended for a model.
• Tool results and retrieved documents, which re-enter the prompt on the next turn.
• Background jobs that summarize yesterday's conversations.

Each of these ends in the same provider call. A check placed at the provider call covers all of them; a check placed on the HTTP chat route covers one.

For the tool-call and agent-loop version of this problem, see how to prevent PII leakage from AI agents.

Does the current message need scanning, or the whole thread?

Scan the latest user message by default. Scan the thread when earlier turns still travel.

The distinction bites in a specific way. A user pastes a card on Tuesday. You block the turn. On Wednesday the client sends the full transcript with each request, as most chat clients do, and Tuesday's paste rides along inside messages while today's message is clean. The block fired once; the string has now gone to the provider every turn since.

If your client replays history, either scan what you're about to send rather than what you just received, or strip denied turns from the stored transcript at the point you block them. Blocking a turn and then storing it is the common shape of this bug.

How do you log a block without logging the PII?

This is the part that goes wrong quietly, and almost nothing written about PII detection covers it.

You've just built a detector that finds card numbers. It fires. Now you want to know how often it fires and on what, so you log the event. If that log line includes the matched span, the matched text, or the message that triggered it, you have taken a card number out of a request that was going to be deleted and written it into a log store with a 90-day retention, replicated to your observability vendor, and searchable by everyone with a dashboard login. The detector is now the leak.

The same failure has several forms:

• Putting the matched span in the log line, for debuggability.
• Returning the span to the client in an error body, so support can see what happened.
• Attaching the message to a trace or a span attribute.
• Sending the full request to an error tracker when the block throws.
• Storing the blocked message in your own database, to show the user what they tried to send.

What to log instead:

• The entity types that matched. ["CREDIT_CARD_NUMBER"] tells you what fired without telling you the number.
• A count and, if you need it, the character offsets and lengths of the matched spans. Enough to reconstruct where, not what.
• The rule label, the actor identifier, and a correlation identifier for the conversation, so you can find the sequence later.
• The decision outcome and the mode, so a dry-run detection is distinguishable from an enforced block.

1
const result = blockList.result(decision);
2

3
log.info(
4
  {
5
    label: "chat.user-message",
6
    actor: session.userId,
7
    correlationId: conversationId,
8
    conclusion: decision.conclusion,
9
    // Types only. Never the matched text.
10
    detectedEntityTypes: result?.detectedEntityTypes ?? [],
11
  },
12
  "sensitive info detected on inbound message",
13
);

Two cautions on the tempting middle ground. Hashing the span is not the same as omitting it: a phone number or a card number has little enough entropy that a plain hash is reversible by anyone who can generate candidates, so if you hash, use a keyed HMAC with a secret that isn't in the log store. And detection metadata can itself be an oracle. A log that records "3 entities, offsets 12, 47, 88" against a message you also stored elsewhere is a reconstruction aid. Keep the metadata with the decision, not next to a copy of the content.

Then check the paths you didn't write. Your error tracker probably captures request bodies by default. Your tracing library probably records HTTP payloads at some sampling rate. Turn both off for the routes that carry prompts, and verify it by triggering a block in staging and searching for the string.

What happens when you block too much?

Users don't stop having the problem you just refused to help with. They open a personal ChatGPT account and paste the same document there.

That's the shadow AI failure, and an aggressive block list produces it reliably. The paste that was going to your application, where you had a detector, a log, a retention policy, and a data processing agreement, now goes to a consumer account you have no relationship with. Measured as "blocks per week", the control looks like it's working. Measured as "did the data leave", it made things worse.

The mitigations aren't technical:

• Tune the block list to the classes where refusal is genuinely justified, and redact the rest. A block that fires on every third message teaches users to route around the product.
• Say what to do instead. "I can't process card details here. Use the payments page." beats a flat refusal, and it's the difference between a user completing the task in your product and completing it somewhere else.
• Give the legitimate case a path. If people keep pasting resumes, the answer might be a resume feature with redaction built in rather than a wall.
• Watch the drop-off, not just the block count. If blocked sessions end rather than continue, the users went somewhere.

What should the user see?

Generic, actionable, and no detail about the detector.

Don't name the entity type, don't highlight the span, and don't say which rule fired. A user probing your filter is learning from every one of those, and a well-meaning "we found a credit card number in your message" is a free oracle. It also risks echoing the value back into a page that gets screenshotted into a support ticket.

"I can't process that kind of information here" plus a next step is enough for the honest user and useless to the other one.

Return the same shape on every path. An HTTP route returns a 400 with a generic body. A tool handler throws an error the agent can act on without the span in the message. A queue job quarantines and moves on. The one thing not to do is return a detailed error on the API and a generic one in the UI, because the API is the one being probed.

How do you roll this out without breaking real turns?

Dry run first, on production traffic, for long enough to see a weekend.

Look at the detections by hand. You'll find the classes you expected and at least one you didn't, usually a legitimate business use of an entity you were about to block. Move classes from dry run to live one at a time, and keep the rest measuring.

Then decide what happens when the detector doesn't answer. A timeout is not a clean result, and treating it as one means the control silently disappears during exactly the incident where you'd want it. For fail-open versus fail-closed, staged rollout, and where the check sits in a latency budget, see PII detection for production AI applications.

And keep the boundary clear: a clean PII check is not an authorization decision. It means the message didn't contain a denied entity. It doesn't mean this user may read this record. For more information about that, see how to stop AI agents accessing data they should not.