How to prevent LLMs surfacing confidential employee or customer data

How do you prevent LLMs from surfacing confidential employee or customer data?

This is a different problem from stopping a user pasting a card number, and treating it as the same problem is why so many teams have a PII control that doesn't help here.

User-submitted PII arrives from outside and you decide whether to let it through. Confidential data disclosure runs the other way: the data is already yours, the application fetched it through a code path you wrote, and the failure is that it reached a person who shouldn't see it. Nothing was pasted. Nothing was injected, necessarily. A retrieval returned a document, and the document was outside the asking user's permission scope.

The controls follow from that. An inbound scan on the user's message is looking in the wrong direction. What you need is authorization on retrieval, isolation in memory, and an output check that knows which tenant it's answering for.

This is LLM02, Sensitive Information Disclosure, in the OWASP GenAI LLM Top 10 2026. Naming it that way is useful when you need the risk register entry to match something a security reviewer already recognizes.

Which three paths surface data the user shouldn't see?

The third row is the uncomfortable one. Once a value is in the weights, no output filter is a reliable remedy: you're pattern-matching against a paraphrase the model can produce in unbounded variations. The control for fine-tuning is dataset hygiene, before training, and it's the one place in this article where runtime enforcement genuinely has nothing to offer.

How do you make retrieval permission-aware?

The default RAG pipeline has one index with everything in it, and the model is trusted to only use what's relevant. That's a relevance mechanism doing an authorization job, and it doesn't do it.

The concrete failure: an internal assistant indexes the company wiki, including HR pages. An employee asks "what's the salary band for a staff engineer?" and the retrieval works exactly as designed. The document was relevant. It was also restricted, and nothing in the pipeline knew that.

Three layers, in order of how much they buy you.

Filter at the query. Every retrieval carries the asking user's permissions as a filter on the search, so restricted documents can't be in the candidate set. This is the load-bearing control, and it requires that permissions were captured when the index was built. Retrofitting them is the expensive part, and it's the part that gets deferred.

Re-check on the result. Filters have bugs and indexes go stale. Before a chunk becomes model context, verify that the asking user can still read the source document, from the source system rather than from the index. This catches the document whose permissions changed after indexing, which is the most common form of stale authorization in a RAG system.

Never rely on the prompt. "Only use documents the user is allowed to see" in a system prompt is not a control. The model has already been given the document. Instructing it not to use the thing it can see is a request, and untrusted content in context can argue the other way.

There's a design decision hiding in the second layer that's worth surfacing. When a restricted document is filtered out, does the assistant say "I don't have information about that" or "there is information but you can't see it"? The first is a small lie that leaks nothing. The second confirms the document exists, which is itself disclosure in an HR or legal context. Pick one deliberately and apply it consistently, because inconsistency between the two is an oracle.

For the broader retrieval security model, see how to secure a RAG application.

How do you enforce a tenant boundary?

Multi-tenant AI features fail in a specific way: the tenant key exists on the primary data path, because that path was built before anyone added a model, and is missing on the AI path, because the AI path was built quickly.

Places the key goes missing, in rough order of frequency:

• The vector store, where embeddings are written without tenant metadata and retrieved with a similarity search that spans everything.
• The prompt cache, keyed on the prompt hash rather than on tenant plus prompt hash, so one tenant's cached completion is served to another.
• Conversation summaries, written to a shared table with a user identifier but no tenant identifier.
• Evaluation and debugging datasets, assembled from production traffic across tenants and then used in a context where they get read back.
• Background jobs, which run without a session and therefore without the tenant scoping that the request path provides.

The check that catches the residue is an output-side one: before the completion goes back, verify that no identifier in it belongs to a different tenant than the one in session.

1
/**
2
 * Backstop before the completion is returned. Runs entirely in your process:
3
 * the completion is not sent anywhere to be checked.
4
 */
5
export async function assertTenantBoundary(
6
  completion: string,
7
  tenantId: string,
8
) {
9
  const referenced = extractAccountIds(completion);
10
  const foreign = await accountIdsOutsideTenant(referenced, tenantId);
11

12
  if (foreign.length > 0) {
13
    log.error(
14
      { tenantId, conversationId, foreignCount: foreign.length },
15
      "completion referenced accounts outside the caller's tenant",
16
    );
17
    throw new TenantBoundaryError();
18
  }
19
}

Log the count and the tenant, never the identifiers themselves. A log line naming the accounts that leaked has moved the leak into your log store, which is the failure covered in how to stop users sending PII to an LLM.

This is a backstop, not the control. The control is that the restricted chunk never entered the context. An output check that fires means a retrieval filter failed, so treat a detection here as an incident rather than as the system working.

How long should the model remember?

Persistence is where a correct single-turn decision becomes a lasting problem.

A conversation summary written on Tuesday keeps a value the original turn redacted, because summarization ran on the raw transcript rather than the redacted one. A cache holds a completion past the point the underlying record was deleted, so a deletion request is honored in the database and not in the cache. A scratchpad accumulates identifiers across an agent run and gets logged in full when the run throws.

Three rules that address most of it:

• Run detection on what you're about to persist, not only on what you display. The persisted copy outlives the request.
• Treat a summary as new content with its own classification, rather than as a derivative that inherits the original's handling.
• Give every derived store a retention period and a deletion path that a data subject request can actually reach. A vector store with no delete-by-source-document operation is a store you can't honor an erasure request against, and that's a design decision made at build time whether or not anyone made it deliberately.

Where does prompt injection fit?

Everything so far assumes an ordinary question and a pipeline that hands over more than it should. Injection is the version where someone is trying.

Untrusted content in context can instruct the model to repeat what's in its context window, to summarize documents it was given, or to encode data into an outbound tool call. If the retrieval already put a restricted document in front of the model, injection is how it gets read out.

Which reinforces the ordering rather than adding a new control. Injection detection reduces how often the attempt succeeds; it doesn't make it safe to put a restricted document in context. Authorization on retrieval is what makes the attempt not matter. For the injection side, see runtime security for LLM applications, and for the outbound path, see how to prevent data exfiltration through AI agents.

What's different about employee data?

Internal assistants get built with less scrutiny than customer-facing ones, and they're pointed at the highest-sensitivity corpus in the company.

The wiki contains performance notes. The ticketing system contains employee relations cases. The shared drive contains the compensation spreadsheet from three reorganizations ago that nobody re-permissioned. An assistant indexed over "all the internal docs" has been given all of it, and the population that can query it is everyone.

Two things help disproportionately. Start the index from an explicit allow-list of sources rather than from everything, because the default of indexing all and excluding later means the exclusion list is always behind. And test with a low-privilege account rather than the account that built it, since the person building the assistant usually has broad access and therefore never sees the failure.

Checklist

• Filter every retrieval by the asking user's permissions, at the query.
• Re-check permissions on the result, against the source system, before a chunk becomes context.
• Decide whether a filtered document is acknowledged or silently absent, and be consistent.
• Key every vector write, cache entry, and summary to a tenant, and filter every read.
• Run an output-side tenant boundary check as a backstop, and treat a hit as an incident.
• Run detection on what you persist, not only on what you render.
• Give derived stores a retention period and a working deletion path.
• Keep confidential data out of fine-tuning sets, because no runtime control removes it later.
• Test the assistant with a low-privilege account.