Producing Compliance Evidence for AI Agent Activity

Auditors do not want your system prompt.

13 min read
In short: An auditor asks four things: that a control exists at the point of risk, that it was evaluated before enforcement, that it operates with reasons, and that someone owns it. What is usually missing is a record of the decisions your controls made.

What compliance evidence do you need for AI agent activity?

Auditors don't want your system prompt. They want evidence that a control existed at the point of risk, that it was evaluated before it was enforced, that it operates and produces decisions with reasons, and that someone owns it and changes it deliberately.

A record of what your agent did covers part of one of those. What is usually missing is a record of the decisions your controls made on each action. For how to produce that record, field by field, see how to audit and log AI agent activity for compliance.

What an auditor is actually asking for

"How do you govern your AI agents?" isn't a question about model behavior. It decomposes into the four asks in the following table.

The askWhat satisfies itWhat does not
A control exists at the point of risk

The check is in the code path of the consequential action, shipped in the same change as the tool

A policy document, or a prompt instructing the model to behave
It was evaluated before it was enforced

Dry-run decisions recorded against real traffic, before the rule started blocking

An assertion that the threshold was chosen carefully
It operates

Decisions with allow, deny, or error conclusions and reasons, retrievable per action and per identity

Application logs of what the agent did, which show outcomes but not the control's decisions

Someone owns it

A named owner and a documented purpose on each AI system, a review date that is current, and rules in reviewed code whose thresholds change through a named process

A spreadsheet of tools with an owner column nobody has updated, or configuration whose history nobody can account for

The evidence a runtime decision produces

Each Arcjet decision records an allow, deny, or error conclusion, along with the rule that fired and the reason. DRY_RUN is a rule mode, not a fourth conclusion: a dry-run denial is recorded without changing the conclusion to deny. Guards add a label naming the action, such as tools.issue-refund, and a metadata map that you populate with the context that matters. If you pass a correlationId, then Arcjet ties the decision to the workflow run that produced it. That identifier is for reconstruction; it doesn't change the allow or deny.

That maps onto the four asks directly. The control exists because the check is in the code path, in the same pull request as the tool. It was evaluated first because dry-run results are recorded too, which makes "we measured before enforcing" demonstrable rather than asserted. It operates because decisions carry reasons and are retrievable per action and per identity. Ownership holds at two levels. Rules live in reviewed code, and Guard policies (a labeled-action system, plus coding-agent policies selected by moment) let a security team change expression rules and detectors without a deployment, and a second approver signs off before a policy change is published. Token-bucket stays in application code. Above the rules, each AI system in your Arcjet inventory carries a named owner, a purpose, and a review date. An AI system is an application or agent together with the model providers and tools it uses. The owner must be a member of your team, so when they leave the team, the system reads as having no owner instead of naming someone who can no longer answer for it.

Keep metadata to server-controlled values: authenticated user and team identifiers, enumerated types, status codes, amounts. Never include user-authored free text. Downstream consumers treat metadata as untrusted, and it is the part of a record most likely to leak something.

Retrieve the evidence

Use the tools that match the enforcement path.

Console filters by time and conclusion. Request decisions search by host, path, or request ID. Guard decisions have their own activity view.

CLI. For protect() on HTTP routes: arcjet requests list, arcjet requests details --request-id <id>, and arcjet requests explain --request-id <id>. For guard() on tools, MCP, and jobs: arcjet guards list, arcjet guards details, and arcjet guards watch, which streams live guard activity. arcjet analyze dry-run-impact is the pre-enforcement picture for request remote rules.

MCP server. Request path: list-requests, get-request-details, explain-decision. Guard path: list-guards, get-guard. get-dry-run-impact remains the request-rule promotion check.

How do decision records become framework evidence?

Decision records become framework evidence when something maps them to the requirements an auditor reads. The Arcjet evidence page does that mapping. It takes signals Arcjet computes from its own records, such as the share of AI systems with an owner or the uses of a prohibited provider since the decision, and maps them to NIST AI RMF 1.0, ISO/IEC 42001, the EU AI Act, and any framework you define.

The counting rule matters more than the mapping. The page lists every requirement in a framework and marks each one as supported or not covered. It reports evidence for some number of requirements out of the total, never a pass mark, and the requirements it does not cover stay on the page. Most of those are organizational process, such as training and policy documents, that a runtime product has no way to observe. An auditor reading the export sees both lists, which is what keeps it credible. For how the mapping reads against one framework, see the NIST AI RMF guide and the EU AI Act checklist.

Three properties of the page decide how you use it:

  • It rolls up the team. The page covers every standard project in your team, so one export answers for the whole estate rather than one application.
  • Every value drills down. A share or a count links to the records behind it: the AI systems without an owner, or the people who used a prohibited provider.
  • The export is the record. Arcjet does not recompute the page for a past date, because provider observations age out and a recomputed past would be a guess. Download it as JSON or CSV at each audit point and keep the file, or push it to Vanta or Drata. See exporting AI governance evidence.

Be precise about the limits

Arcjet retains decision history in the Arcjet Console according to your plan's log retention (longest on Enterprise). Long-term decision retention is opt-in: turn it on when a framework sets a log-retention period, such as the EU AI Act's six-month minimum for logs of high-risk systems, and Arcjet keeps decision records beyond the default 30 days. For detection and alerting, or for a retention period you control end to end, export the decisions to your SIEM or log store. On the Enterprise plan, Arcjet sends decisions there through a log drain.

Don't assume the default Console record covers a long retention obligation on its own. Either turn on long-term retention, or keep the copy in your SIEM or storage bucket under your own retention policy. If your obligation requires seven years of retained evidence, the copy you hold meets it. For how to pick the period, see AI system log retention. The Console is the live operational view. Anyone describing the Console alone as an audit trail in the archival sense is overclaiming, and an auditor who probes it finds the gap.

Export gives you the durable copy with the decision identifier, rule, and reason on every record. Where you also act on a decision in your own code, emit your own event carrying the same decision identifier, so the two records join.

The platform half

Evidence about your controls helps only if the platform enforcing them is itself in scope.

Arcjet has completed a SOC 2 Type 2 examination covering Security, Availability, and Confidentiality with an unqualified opinion, with the report available through the Trust Center. Arcjet also supports HIPAA compliance and enters into a business associate agreement (BAA) with Enterprise customers whose use of Arcjet involves PHI. Both parties must sign the BAA before any PHI is processed through Arcjet.

Because SDK-local sensitive-information inspection keeps the raw request body in your process, your assessment doesn't have to account for protected data leaving your environment for a third party to check it on that path. The server-side sensitive-information detector (coding-agent hooks and other no-SDK policies) is a different claim. That removes a transfer question from the analysis rather than answering it. For detail about which controls run locally and which call out, see keeping security inspection local.

What does HIPAA's audit controls standard ask of an AI agent?

The HIPAA Security Rule's audit controls standard, 45 CFR 164.312(b), requires mechanisms that record and examine activity in information systems that contain or use electronic protected health information (ePHI). An agent that reads a chart or sends a message about a patient is that activity, so its tool calls belong in the record.

The four asks in the earlier table still apply, with one difference: the record itself can't become a second copy of the PHI. A log line that stores the full tool arguments of a call that carried a medical record number has moved the identifier into your logging pipeline, and every system downstream of it now holds PHI. Arcjet's server-side sensitive information detector handles this for the decision record: when it flags a value, Arcjet removes the value before it stores the decision and keeps only the entity types and their positions. The record shows that a call carrying a medical record number was denied, without containing the number.

Retention is a separate question. HIPAA requires you to keep the documentation that the Security Rule calls for, such as policies and assessments, for six years under 45 CFR 164.316(b)(2). Many organizations apply a similar period to audit records. Decide the period with your compliance team, and keep the archival copy in your SIEM or storage bucket. By default, Arcjet stores Guard decisions for 30 days, and your plan's log retention sets how much of that window you can view. Opt-in long-term retention keeps them longer. If any of the values that you send to Arcjet carry PHI, a signed business associate agreement (BAA) must be in place first. For the wider requirement-by-requirement view, see HIPAA compliance for AI agents.

Checklist

  • Put the control in the code path of each consequential action, not in a document.
  • Run in dry run first, so the pre-enforcement measurement is itself recorded.
  • Label every check with the action it protects, using a stable slug.
  • Populate metadata with server-controlled values only.
  • Tag decisions with the workflow run through a correlationId.
  • Export decisions to your SIEM or log store, and emit your own event carrying the decision identifier where your code acts on a decision.
  • Know your plan's Console retention window (longest on Enterprise). Turn on long-term retention when a framework sets a period, or keep the archival copy in the system you export to.
  • Keep threshold changes deliberate and attributable.
  • Give each AI system a named owner and purpose, and review it before the 180-day review date passes.
  • Export the evidence page at each audit point and keep the file, or push it to your GRC tool.

Where Arcjet fits

Compliance evidence for agents is split between two kinds of tooling. Governance and posture platforms such as Cranium and Refractal describe your AI estate and its controls. Observability platforms record what the agent did.

Arcjet contributes a narrower and often missing piece: the record of what your enforcement controls decided, per action, with the rule and reason, including dry-run decisions from before enforcement was switched on. On top of that record, Arcjet keeps the governance layer: an inventory of AI systems built from what Discovery, the SDK and coding-agent hooks observe, an owner and purpose on each, provider decisions, and an evidence page that maps all of it to NIST AI RMF, ISO/IEC 42001 and the EU AI Act. The evidence comes from the control itself rather than from a questionnaire about the control. For the wider picture, see what is AI governance.

Console retention follows your plan (longest on Enterprise), and long-term retention is opt-in. On the Enterprise plan, a log drain exports decisions to your SIEM or log store, which is where detection rules, alerting, and long-term retention belong. A SOC 2 Type 2 examination covers the platform itself, with the report in the Trust Center.

Learn more: Security and compliance · Architecture

Frequently asked questions

What compliance evidence do I need for AI agent activity?

Evidence that a control existed at the point of risk, that it was evaluated before it was enforced, that it operates and produces decisions with reasons, and that someone owns it and changes it deliberately. A record of what the agent did covers only part of the third.

Can Arcjet keep decision records for longer than 30 days?

Yes, if you opt in. By default, Console history follows your plan's log retention, which is longest on Enterprise. Long-term decision retention keeps records for the period a framework names, such as the EU AI Act's six-month minimum for high-risk system logs. You can also export decisions to your SIEM or log store (Enterprise plan) and keep your own copy.

Can I send Arcjet decisions to my SIEM?

Yes. On the Enterprise plan, Arcjet exports decisions to your SIEM or log store, so you can build detection rules and alerts on agent activity in the tools your security team already uses.

Can I describe Arcjet decision records as an audit trail?

The Console view alone is not an archival audit trail; it covers your plan's retention window. Exported to your SIEM or storage bucket under your own retention policy, the same records, each carrying its decision identifier, rule, and reason, form the durable copy.

How do I retrieve Guard decisions versus request decisions?

Use Guard tools for tool, MCP, and job decisions: CLI arcjet guards list, details, and watch, and MCP list-guards and get-guard. Use request tools for HTTP request decisions: arcjet requests list and MCP list-requests, get-request-details, and explain-decision.

How do I demonstrate a control was measured before enforcing?

Run it in DRY_RUN mode first. DRY_RUN is a rule mode, not a third conclusion: dry-run denials are recorded without changing the allow or deny. That makes the pre-enforcement measurement demonstrable rather than asserted.

Does the Arcjet evidence page say I am compliant?

No. It lists every requirement in NIST AI RMF 1.0, ISO/IEC 42001, the EU AI Act, or your own framework, and marks each one as supported or not covered by Arcjet evidence. Requirements that are organizational process, such as training and policy documents, stay listed as not covered.

What does HIPAA require you to log for an AI agent?

The audit controls standard, 45 CFR 164.312(b), requires mechanisms that record and examine activity in systems that contain or use electronic PHI. For an agent, that means each tool call that touches PHI and the decision your control made on it, recorded without copying the PHI itself into the log.

AI runtime security in your code

Protect your AI agent workflows with Arcjet

Capture events record what the application did after a decision, which is the half of the evidence most teams are missing.