How to stop an AI agent sending PHI to third parties

How do you stop an AI agent sending PHI to a third party?

Check the text that the model wrote for each outbound tool call inside the tool, before the tool sends anything, and refuse the call when that text carries a patient identifier. With Arcjet, the tool calls guard() and a published policy decides: a server-side detector finds identifiers such as medical record numbers and member IDs, and the policy denies the send.

Protected health information (PHI) is health information that can be linked to a specific person. Under the US Health Insurance Portability and Accountability Act (HIPAA), health information is identifiable when it carries an identifier, such as a name, a medical record number (MRN), a health plan member ID, or a phone number. To stop an AI agent sending PHI, you look for those identifiers at the moment the agent sends text out of your organization.

This article follows one agent: an assistant for a clinic or a health-tech product that summarizes a patient's chart and can then send a message, open a ticket in an external helpdesk, post to team chat, or call a partner API. For checks on what users paste into a model, see how to stop users sending PII to an LLM. For every point where an agent can leak personal data, see how to prevent PII leakage from AI agents.

The procedure has the following steps:

1. Decide which tools send data outside your organization, and which recipients a business associate agreement (BAA) covers.
2. Declare a policy with a SERVER string input for the outgoing text and a boolean input for whether it leaves your organization.
3. Add a server-side sensitive information detector that denies the PHI types plus SSN, EMAIL, and PHONE_NUMBER.
4. Write Rego that denies a PHI identifier anywhere and any listed identifier when the text leaves your organization.
5. Call guard() in the tool before it sends.
6. Return a refusal that the model can act on without repeating the PHI.
7. Publish the policy in dry run, review its decisions, and then set the rules live.

Why is the outbound tool call where PHI leaves?

The model writes a tool call's arguments after every inbound check has already run. A user's request can be clean while the message that the model composes from the chart carries the patient's MRN, member ID, and phone number. The tool call is the last point that your code controls before that text reaches another organization.

Take a front-desk coordinator who asks the agent to tell the billing vendor that a claim was rejected. The request names no patient. The agent opens the chart, writes a summary, and calls tickets.create with a description that includes MRN 00418827, Member ID XJH448210931, and the patient's mobile number. The inbound check saw a sentence about a rejected claim. The helpdesk vendor receives a patient record.

HIPAA calls this a disclosure: the release or transfer of information outside the entity that holds it, as defined in 45 CFR 160.103. Its rules apply to covered entities, which are health plans, health care clearinghouses, and providers that transmit health information electronically in HIPAA transactions. The minimum necessary standard in 45 CFR 164.502(b) requires reasonable efforts to limit PHI to the minimum needed for the purpose. A ticket about a rejected claim needs the claim number, not the MRN and the phone number.

A system prompt line such as "never put identifiers in tickets" doesn't hold this boundary, because text planted in a referral letter that the agent reads can override it. For how injected text steers outbound calls, see prompt injection in healthcare AI agents.

Which tools send PHI outside your organization?

Any tool that delivers text to a system that your organization doesn't run can send PHI out. List every tool the agent can call, note where each one sends its arguments, and for each call decide whether the recipient is inside your organization, covered by a BAA, or neither.

A business associate is a company that creates, receives, maintains, or transmits PHI on behalf of a covered entity, such as a billing service. Under 45 CFR 164.502(e), a covered entity can share PHI with one once it has satisfactory assurance, in a written contract, that the associate will safeguard it. That contract is the BAA, and 45 CFR 164.504(e) sets out what it must contain. The clinic's inventory looks like the following table:

Compute the external flag in your own code, from the recipient and from configuration that you control. Never let the model supply it, because a model that can pass external: false can switch off the stricter rule.

Some tools exist to send PHI. A referral to a specialist carries identifiers by design, and minimum necessary doesn't apply to disclosures to a provider for treatment (164.502(b)(2)). Give that tool its own policy, and check that the recipient is treating this patient.

How do you declare the policy and its detector?

In the Arcjet Console, create an application policy with your tool's label and two SERVER inputs: a string for the outgoing text and a boolean for whether it leaves your organization. Then add a sensitive information detector that runs in Arcjet Cloud on the text and denies the PHI types, SSN, EMAIL, and PHONE_NUMBER.

A policy is the set of rules that Arcjet evaluates when your code calls guard() with a matching label, and a security team can publish changes to it without an application deployment. A detector is one of Arcjet's built-in checks, run on a declared input, and the policy's rules read its result.

For messages.send, declare message as a string and external as a boolean, both evaluated by Arcjet Cloud. That makes them SERVER inputs: the SDK sends the raw values to Arcjet. Arcjet detects the PHI types only in the server-side detector. The SDK-local detector, which runs in your process, recognizes core types such as EMAIL and SSN but not MRNs or member IDs.

Under Detectors, add a Sensitive information detector with the ID phi. Set Runs in to Arcjet Cloud, choose message as its input, and under Entity types choose Deny only these. Add the following types:

• MEDICAL_RECORD_NUMBER
• HEALTH_PLAN_BENEFICIARY_NUMBER
• LICENSE_PLATE_NUMBER
• VEHICLE_IDENTIFICATION_NUMBER
• DEVICE_IDENTIFIER
• SSN
• EMAIL
• PHONE_NUMBER

A detector with a deny list acts only on the types that it names. The first five come from version 2 of Arcjet's sensitive information taxonomy, the list of entity types that the detector recognizes. A new detector uses the current version, but a detector in a policy that you published earlier stays on its original version until you move it forward.

Arcjet recognizes identifiers with a fixed format, such as a vehicle identification number or a MAC address, from their structure. An MRN or a member ID looks like any short code, so the detector relies on the text around it, such as a label like MRN or Member ID next to the value. For more about how detection works, see PHI detection for AI agents.

What Rego denies PHI in an outbound message?

The policy needs two rules. The first denies the call when the detector finds any of the five PHI types, wherever the message goes. The second denies the call when the detector finds any listed type and external is true, which adds Social Security numbers, email addresses, and phone numbers to what can't leave your organization.

Rego is the policy language of the Open Policy Agent project, and Arcjet policies use it to combine detector results with input values. In the policy editor, click Take over as Rego and enter the following policy:

1
package arcjet.guard
2

3
import rego.v1
4

5
# The identifier types that taxonomy version 2 adds for PHI.
6
phi_types := {
7
  "MEDICAL_RECORD_NUMBER",
8
  "HEALTH_PLAN_BENEFICIARY_NUMBER",
9
  "LICENSE_PLATE_NUMBER",
10
  "VEHICLE_IDENTIFICATION_NUMBER",
11
  "DEVICE_IDENTIFIER",
12
}
13

14
# Deny a message that carries a PHI identifier.
15
deny contains "phi-in-message" if {
16
  some entity in input.signals.sensitive_info.phi.entities
17
  entity in phi_types
18
}
19

20
# Deny a message that carries any listed identifier when it leaves your
21
# organization.
22
deny contains "identifiers-to-external" if {
23
  input.signals.sensitive_info.phi.detected
24
  input.values.external
25
}

The split is deliberate. A note to a colleague can carry a callback number. An MRN in a ticket or a chat post is a record identifier copied into a system that isn't the record. If a BAA covers your team chat and staff reference MRNs there, drop the first rule from the chat.post policy.

A live Rego rule needs a stored test before the Console saves the policy. A test input can include detector results under signals, so the following input expects the identifiers-to-external denial without running the detector:

1
{
2
  "values": { "external": true },
3
  "signals": {
4
    "sensitive_info": { "phi": { "detected": true, "entities": ["EMAIL"] } }
5
  }
6
}

Add the near miss as well: the same input with external set to false expects no denial.

Where does the guard call go in the tool?

Put the call inside the tool's handler, after you validate the model's arguments and before the line that sends anything. The call passes the label, the actor from your authenticated session, the text to screen, and the external flag that your code computed.

The following handler sends a message for the clinic's agent:

1
import { launchArcjet, policyInput } from "@arcjet/guard";
2

3
const arcjet = launchArcjet({ key: process.env.ARCJET_KEY! });
4

5
// Domains your organization runs. Every other recipient is external.
6
const internalDomains = new Set(["clinic.example"]);
7

8
const REWRITE =
9
  "Not sent: the message contains a patient identifier, such as an MRN, " +
10
  "member ID, SSN, email address, or phone number. Rewrite it without " +
11
  "identifiers, or ask the user to send it from the EHR.";
12
const CHECK_FAILED =
13
  "Not sent: the outbound check didn't complete. Don't retry. " +
14
  "Tell the user that the message wasn't sent.";
15

16
export async function sendMessage(
17
  args: { to: string; subject: string; body: string },
18
  ctx: { userId: string; runId: string },
19
) {
20
  const domain = args.to.split("@").pop()?.toLowerCase() ?? "";
21
  const isExternal = !internalDomains.has(domain);
22
  // Only the text the model wrote. The address would always match EMAIL.
23
  const message = `${args.subject}\n\n${args.body}`;
24

25
  const decision = await arcjet.guard({
26
    label: "messages.send",
27
    actor: ctx.userId,
28
    correlationId: ctx.runId,
29
    inputs: {
30
      message: policyInput.server.string(message),
31
      external: policyInput.server.boolean(isExternal),
32
    },
33
  });
34

35
  if (decision.hasFailedOpen()) {
36
    log.warn({ decisionId: decision.id }, "messages.send check failed");
37
    return { sent: false, error: CHECK_FAILED };
38
  }
39

40
  if (decision.conclusion === "DENY") {
41
    log.info({ decisionId: decision.id }, "messages.send denied");
42
    return { sent: false, error: REWRITE };
43
  }
44

45
  // The signature carries the clinic's phone number, so add it after the
46
  // check rather than asking the model to write it.
47
  await mailer.send({ ...args, body: args.body + clinicSignature });
48
  return { sent: true };
49
}

Screen only the text that the model wrote. Put the recipient address in message and every external send matches EMAIL and trips the second rule.

Write one guard() call per tool with a fixed label, so tickets.create and chat.post each get their own. Framework wrappers such as guardTool take the same inputs and, by default, refuse to run the tool on a denial or a failed check.

What do you return to the model when the call is denied?

Return a short refusal that tells the model what to change and repeats nothing from the message. The model reads the tool result as context, so a refusal that quotes the MRN puts the identifier back into the conversation, the trace, and every log that records tool results.

Name the kinds of identifier to remove, so the model has a retry that can succeed: it drops the MRN and member ID, keeps the claim number, and the ticket goes through. When the check didn't complete, say so and tell the model to stop, because rewriting can't fix a timeout and the model can otherwise loop. In your own logs, record the decision ID and leave the arguments out, and turn off argument capture in your error tracker for these tools.

Why does fail-open handling matter on this tool?

The direct guard() client fails open. When a timeout, a network error, or an incomplete evaluation prevents a full check, it returns ALLOW with an error result. A tool that checks only for DENY sends the message unscreened whenever Arcjet can't answer, which is the outcome that the tool exists to prevent. On this tool, a refused message waits for the user to send it again, while a sent message stays in the vendor's system.

The server-side detector has time limits that long chart summaries run into. A value up to 2 KB that can't be screened within 2 seconds fails a live rule closed. A longer value, up to about 100 KB, can take up to 3 seconds, and if that scan doesn't finish, Arcjet allows the value unscreened, so detected: false on a long value is inconclusive. For the full limits, see the policy contract.

One failure doesn't show up as failed open. If the label in your code matches no published policy, the decision is ALLOW with hasFailedOpen() false and a policyEvaluation status of NOT_CONFIGURED. Assert on that status in an integration test that calls Arcjet with your published policy, so a renamed label can't switch the check off unnoticed. The SDK test client can't do this, because it returns a failed-open ALLOW for every call.

How do you roll out the policy without blocking real messages?

Save the policy with both rules in dry run, send realistic traffic through the tool, and review the decisions in the Arcjet Console. Fix the false positives, add tests for the rules you trust, switch them to live, and save again.

A dry run rule records a denial without blocking the call. Expect false positives from staff contact details: a model-written sign-off with a nurse's direct line trips the external rule on every outside message, which is why the example adds the signature in code.

Dry run records leaks rather than stopping them. Before production, run the agent in staging against synthetic charts in your electronic health record (EHR) system's identifier formats, labeled and unlabeled, to keep the production dry run short.

What does Arcjet store, and do you need a BAA?

Arcjet stores server policy inputs with each Guard decision for 90 days, except values that the server-side sensitive information detector flags. Arcjet removes a flagged value and keeps only the entity types and positions. Because the text you send can be PHI, you need a signed BAA with Arcjet before you send it.

For this tool, a message that the detector flags isn't kept, but a message it passes is kept with the external value, even though it can still hold an identifier the detector missed. Backups can hold data for longer than the storage period.

Arcjet supports HIPAA compliance and offers a HIPAA-compliant environment. Arcjet enters into a BAA with Enterprise customers whose use of Arcjet involves PHI, and both parties must sign it before any PHI is processed through Arcjet. Using Arcjet for PHI without a signed BAA may leave your organization out of compliance, for the same reason that the helpdesk vendor needs one. This is technical guidance, not legal advice.

If your organization can't let a vendor keep customer content, ask about zero data retention (ZDR) at support@arcjet.com. With ZDR, Arcjet keeps the decision structure, such as the conclusion, the rules that fired, and the shape of each input, but not the input values. Some records still carry values: request decisions from protect() keep request metadata, such as the IP address, path, headers, and query, Guard custom rules keep their inputs, and session analysis keeps session data, including a screened tool-result excerpt, while a session is active and for up to 24 hours after. Regional processing controls are part of the Enterprise plan, and Arcjet's SOC 2 Type 2 report is in the Arcjet Trust Center.

The decisions also give you a record of every send the tool attempted. The audit controls standard in 45 CFR 164.312(b) asks for mechanisms that record and examine activity in systems that contain or use electronic PHI. For more information, see how to audit and log AI agent activity for compliance.

What doesn't this policy catch?

The policy blocks a send and doesn't clean one. It can miss identifiers that have no label or that the model paraphrased, it doesn't cover every HIPAA identifier class, and it can't tell whether the recipient may see this patient's information.

Arcjet detects identifiers. It doesn't redact them or de-identify the message, so a denied call stays denied until the model or the user rewrites the text. Detection doesn't make the agent HIPAA compliant on its own either.

An MRN with no label looks like any other number, and the detector can miss it. A model that writes "his record number ends in 8827" or spells the digits out can still identify the patient without the pattern or label that the detector looks for. Dates related to a person and ages over 89, biometric identifiers, full-face photographs, and the catch-all class of other unique identifying codes are identifier classes under the Safe Harbor method in 45 CFR 164.514(b)(2), and Arcjet doesn't detect them. For the full list, see the HHS de-identification guidance. This deny list also leaves out names, which Arcjet does detect. Adding GIVEN_NAME and SURNAME affects only the external rule, so try them in dry run, where staff names show up as false positives.

The policy answers whether the text carries an identifier. It doesn't decide whether this user, through this agent, may share this patient's record with this recipient, or whether the agent needed to open the chart at all. Those are authorization decisions that belong in the tool and the data layer. For more information, see how to enforce least privilege for AI agent tool calls and how to stop AI agents accessing data they should not.

Learn more: Detect protected health information · Policy contract · Author and publish policies · Privacy