How do I prevent an AI agent from leaking system prompt content?

How do I prevent an AI agent from leaking system prompt content?

To prevent an AI agent from leaking system prompt content, screen the request that asks for it, screen the response that carries it, and keep anything that would hurt you out of the prompt. The third control holds when the other two miss.

A system prompt reaches a user through three paths:

• Direct extraction. Someone asks the model to repeat its instructions, and the model repeats them.
• Indirect extraction. Injected content in a retrieved document or a tool result instructs the model to include its instructions in the answer.
• Incidental disclosure. The model paraphrases its instructions while explaining why it refused something, with no attacker involved.

The third path needs no attacker, and no jailbreak detector fires on it. That's why prompt secrecy can't be the only control.

Why the system prompt isn't a secret

The system prompt shares a context window with attacker-influenced text, and a component whose job is to produce plausible continuations processes it. It has no access control. Treat the system prompt as confidential rather than secret: worth protecting, embarrassing to lose, and not a safe place for anything that matters.

A practical test is to write down what an attacker gains from a verbatim copy of your system prompt. If the answer includes an API key, an internal hostname, a customer's name, a discount code, or the exact bounds of a business rule, then the problem isn't leakage. The problem is that the value is in the prompt.

The following table separates the two cases:

Anything in the last two rows is a finding on its own, before you consider extraction. The OWASP Top 10 for LLM Applications covers this pair as LLM02:2026 Sensitive Information Disclosure and LLM08:2026 Hidden Context Exposure. For more information, see implementing the OWASP Top 10 for LLM Applications.

Screen the request that asks for it

Extraction attempts have recognizable shapes: a request to repeat the text that precedes the conversation, to translate or summarize the instructions, to enter a debug or developer mode, or to encode the configuration. A prompt-injection detector catches the well-known phrasings.

The following handler does that with Arcjet, a security library that evaluates its rules in your own process and returns an allow-or-deny decision before the provider call. launchArcjet creates the client, detectPromptInjection() configures the rule once, and arcjet.guard() runs it against the inbound message:

1
import { detectPromptInjection, launchArcjet } from "@arcjet/guard";
2

3
const arcjet = launchArcjet({ key: process.env.ARCJET_KEY! });
4

5
export async function handleMessage(message: string, session: Session) {
6
  const decision = await arcjet.guard({
7
    label: "chat.inbound",
8
    actor: session.userId,
9
    rules: [detectPromptInjection()(message)],
10
  });
11

12
  if (decision.conclusion === "DENY") {
13
    return { reply: "Please rephrase your message." };
14
  }
15

16
  return callProvider({ message });
17
}

Keep the denial generic. A message that says the request looked like a prompt-extraction attempt tells the attacker which phrasing to vary, and turns your detector into a tuning oracle.

Inbound screening has a known ceiling. A request phrased as a legitimate support question, with no attack pattern, can still lead the model to paraphrase its instructions. Measure the detector in dry-run mode against real traffic before you enforce, so that you know the false-positive rate that you're accepting.

Screen the response that carries it

Most teams skip the output check, and it's the cheaper of the two to get right, because you know what you're looking for. You wrote the system prompt, so you can test whether the response reproduces it.

The following check compares the response against distinctive strings from your own prompt, rather than trying to detect leakage in the abstract:

1
// Distinctive phrases from the system prompt. Short or common strings
2
// produce false positives, so pick spans that only appear in the prompt.
3
const PROMPT_MARKERS = [
4
  "You are Aria, the billing assistant for",
5
  "escalate to a human when the refund exceeds",
6
];
7

8
function looksLikePromptDisclosure(reply: string): boolean {
9
  const normalized = reply.toLowerCase();
10
  return PROMPT_MARKERS.some((marker) =>
11
    normalized.includes(marker.toLowerCase()),
12
  );
13
}

Exact matching catches verbatim reproduction and nothing else. A model that paraphrases gets through. Two additions narrow that gap. Place a unique, meaningless canary string in the system prompt, so that any response containing it is an unambiguous disclosure regardless of the surrounding text. Run a sensitive-info check on the response, so that a paraphrase carrying an internal hostname or an email address is caught by what it contains rather than by how it's worded.

On a match, return a generic reply and record the event with the request's correlation ID. Don't return the model's text with the offending span removed. The remainder often reconstructs the instruction.

The indirect path through retrieved content

A retrieved document can carry an instruction to disclose the system prompt, and that text didn't cross your inbound screen. The user typed a normal question. The payload arrived through a fetched page, a knowledge-base article, a ticket body, or an MCP tool result.

Screen tool results and retrieved chunks before they re-enter the context window, the same way that you screen inbound messages. The following tool runs the same Arcjet prompt-injection rule through the same guard() call, in your own process, this time on fetched content, and returns a placeholder on a deny:

1
import { detectPromptInjection, launchArcjet } from "@arcjet/guard";
2

3
const arcjet = launchArcjet({ key: process.env.ARCJET_KEY! });
4

5
export async function fetchTool({ url }: { url: string }) {
6
  const content = await fetch(url).then((r) => r.text());
7
  const decision = await arcjet.guard({
8
    label: "tools.fetch",
9
    rules: [detectPromptInjection()(content)],
10
  });
11

12
  if (decision.conclusion === "DENY") {
13
    return { content: "[Content blocked]" };
14
  }
15

16
  return { content };
17
}

For more information about the full shape of that attack, see indirect prompt injection in agentic workflows and how to secure a RAG application.

Move the value out of the prompt

Prompt secrecy is a weak control, so design for it to fail. Three moves reduce what a disclosure exposes to almost nothing.

Keep secrets out. Credentials, keys, and internal hostnames belong in environment variables that your tool handlers read. The model doesn't need them to call a function that you wrote. For more information, see storing secrets in environment variables.

Move the rules into code. A prompt line such as Refunds above $500 need approval is both leakable and unenforceable. The same rule as a threshold in issueRefund is neither. For more information about that placement, see how to limit what actions an AI agent is allowed to take.

Fetch data per request, with authorization. Customer context assembled into the prompt at request time, scoped by the session, leaks one user's data at worst. The same data preloaded into a shared system prompt leaks everyone's. For more information, see preventing LLMs from surfacing confidential data.

After those three moves, a full disclosure of the system prompt reveals your tone instructions and the names of your tools. The tool names matter only if the tools themselves aren't authorized, which brings the problem back to the layer that was going to decide it anyway.

Where Arcjet fits

Arcjet covers two of the three controls on this page. Prompt-injection detection screens the inbound message and the content that re-enters context. Sensitive-information detection classifies the response in your own process, so a paraphrase that carries an internal hostname or an email address is caught without the text leaving your environment. Both run from one guard() call and return a decision before the reply is sent.

The third control, moving credentials and thresholds out of the prompt, is a design decision that no library makes for you.

Learn more: Prompt injection detection · Sensitive information detection

System prompt leakage checklist

• No credentials, keys, internal hostnames, or customer data appear in prompt text.
• Business thresholds live in code, and the prompt refers to them without stating values.
• A prompt-injection check runs on inbound messages before the provider call.
• The same check runs on tool results and retrieved chunks before they re-enter context.
• A canary string in the system prompt makes verbatim disclosure unambiguous.
• Responses are checked against distinctive prompt markers and for sensitive information.
• Denials are generic and don't name what the detector matched.
• Suspected disclosures are recorded with the actor and correlation ID for review.