How do I prevent an AI agent from taking irreversible actions?

How do I prevent an AI agent from taking irreversible actions?

The actions you can't undo from your app are the ones that move money, delete a record, send a message someone already read, or change production. Put those on a deny-by-default list. A person can hold the few that still need to run. A token bucket stops the fiftieth approved refund. Confirm at the Stripe call, not in the system prompt.

Human-in-the-loop is a hold after a policy allow. It isn't the policy. For more information about that split, see human approval is not a security policy and human approval gates for agent actions.

What counts as irreversible

An action is irreversible when reversing it needs another organization, a backup you may not have, or a person who already saw the message. The following table classifies the common cases:

Reads, drafts, and staging writes are reversible. Don't put them on the same list. A list that includes lookupOrder trains reviewers to click through. Keep the gated set small enough that a person reads it.

Detection is how you classify: a static allowlist of tool names plus rules on arguments (amount_cents > 10_000, environment === "production", to not in allowedRecipients). Enforcement is block or queue. Don't wait for a model to "notice" that a transfer is serious.

Deny by default on that set

The tool handler is the last reversible point. A framework allowlist of tool names answers "may this agent ever see issue_refund?" It doesn't answer "this user, this invoice, this amount, on this request." For more information about that mistake on Claude hosts, see canUseTool is not a policy gate.

1
const IRREVERSIBLE = new Set([
2
  "issue_refund",
3
  "delete_customer",
4
  "send_email",
5
  "set_prod_flag",
6
]);
7

8
export async function runTool(
9
  name: string,
10
  args: Record<string, unknown>,
11
  user: { id: string; role: string },
12
) {
13
  if (IRREVERSIBLE.has(name) && user.role !== "finance-approver") {
14
    return queueForHuman({ name, args, userId: user.id });
15
  }
16
  return tools[name](args, user);
17
}

queueForHuman stores the intent and doesn't call Stripe. Store the arguments you validated, not the raw model output, and store them server-side: the resumed call has to read the amount from that record rather than from whatever the agent sends when it retries. The hold is yours to build. For more information about the policy decision, hold, and resolution, see human approval gates.

Rate-limit how often an allow can fire

A correctly approved refund, 50 times, is still an incident. Put a token bucket on the irreversible tool, keyed on the user or the account, at the handler. An HTTP limit on the chat route counts workflow starts, not refunds.

1
if (!(await refundBucket.take(user.id, 1))) {
2
  throw new Error("Refund frequency exceeded");
3
}

For more information about keying a budget the same way, see enforce token and spend budgets.

Dry-run and approve at the API call

For transfers and sends, require an approval the model can't satisfy by talking:

1
async function issueRefund(args: {
2
  invoiceId: string;
3
  amountCents: number;
4
  approvalId?: string;
5
}) {
6
  const approval = args.approvalId
7
    ? await approvals.find(args.approvalId)
8
    : undefined;
9

10
  if (!approval || approval.status !== "approved") {
11
    return {
12
      preview: true,
13
      invoiceId: args.invoiceId,
14
      amountCents: args.amountCents,
15
      hint: "This refund needs an approval before it runs",
16
    };
17
  }
18

19
  // The approval record supplies the amount. If the model can change the
20
  // number between the review and the call, the review covered nothing.
21
  return stripe.refunds.create(
22
    {
23
      charge: chargeIdFor(approval.invoiceId),
24
      amount: approval.amountCents,
25
    },
26
    { idempotencyKey: `refund:${approval.id}` },
27
  );
28
}

The approval is a row you look up, not a boolean the model passes. A confirm: true argument is satisfied by a model that guesses the field name.

The idempotency key matters more than it looks. A held action gets retried: the queue redelivers, the reviewer double-clicks, the worker restarts after the Stripe call but before the local write. Keying on the approval ID means those all collapse into one refund, and Stripe replays the original response instead of moving money twice. Without it, "resume the approved action" is a duplicate-payment bug waiting for a bad deploy.

Staging can set a dry-run environment that logs the Stripe payload and returns an HTTP 204 status code.

What a correlation ID doesn't do

A correlation ID lets you reconstruct "read customer, then refund" after the fact. It doesn't deny the refund because of the earlier read. Sequence-aware deny isn't something this document implements. For more information about the sequence problem, see anatomy of an agent incident and runtime controls on enterprise systems.

A sandbox that isolates code doesn't authorize issue_refund. For more information about that gap, see a sandbox is not a tool policy.

One in-process deny on the transfer tool

The following Guard sits in issue_refund before Stripe. It is a per-user frequency cap. Classification (IRREVERSIBLE, the human queue) stays in your code. guard() takes no Request. It has no bot primitive.

1
import { launchArcjet, tokenBucket } from "@arcjet/guard";
2

3
const arcjet = launchArcjet({ key: process.env.ARCJET_KEY! });
4

5
const refundFrequency = tokenBucket({
6
  bucket: "issue-refund",
7
  refillRate: 5,
8
  intervalSeconds: 3_600,
9
  maxTokens: 5,
10
});
11

12
const decision = await arcjet.guard({
13
  label: "tools.issue-refund",
14
  actor: session.userId,
15
  correlationId: workflowRunId,
16
  rules: [refundFrequency({ key: session.userId, requested: 1 })],
17
});
18

19
if (decision.conclusion === "DENY" || decision.hasFailedOpen()) {
20
  // A direct guard() call returns ALLOW when it couldn't finish evaluating.
21
  // On a refund, treat that as a deny.
22
  throw new Error("Refund denied");
23
}
24

25
await queueForHuman({
26
  name: "issue_refund",
27
  args: { invoiceId, amountCents },
28
  userId: session.userId,
29
});

Scan free-text memos if the refund tool accepts them. Don't scan the invoice id. For more information about outbound SaaS sends (GitHub, public email APIs), see guardrails on agents that call external APIs.