AI Agent Security Architecture: In-Code vs Proxy vs Gateway vs Security Agent

What is the right architecture for AI agent security?

The right AI agent security architecture places enforcement at every boundary where a different class of risk becomes visible. A proxy protects network and HTTP ingress. An AI gateway inspects traffic between an application and a model provider. In-code controls enforce policy inside tool handlers and business operations. A security agent or supervisor can evaluate workflow history and sequence-level risk.

These architectures are not interchangeable. A component can only stop an action it can observe and intercept. For most production agents, use a layered design: proxy controls at public entrypoints, gateway controls around model traffic, deterministic in-code checks before consequential actions, and supervisory reasoning only where cross-step context changes the decision.

Map the boundaries in an agent workflow

An agentic application crosses more boundaries than a conventional request-response service. A single task can begin with an HTTP request or queue message, call a model several times, retrieve untrusted content, invoke internal tools, and create an external side effect.

1
User or bot
2
    |
3
    v
4
Proxy / WAF ---- network, HTTP, bot, and coarse rate controls
5
    |
6
    v
7
Application ---- identity, session, workflow, and business context
8
    |       \
9
    |        +--> In-code tool guard --> API, database, payment, email
10
    |
11
    +--> AI gateway --> model provider
12
    |
13
    +--> Security agent / supervisor observes workflow state

The diagram is not a required call order. A queue worker may start directly inside the application and never pass through a proxy. A tool may call an internal API without going through the AI gateway. A supervisor may review a proposed action but still need an in-code policy enforcement point to stop it.

Before choosing controls, inventory each workflow's entrypoints, model calls, untrusted data sources, tools, identities, budgets, approvals, and side effects. “Protect the agent” is too broad. “Stop an unapproved refund before the payment API executes it” identifies an enforceable boundary.

How should security architectures be compared?

Evaluate each placement against the facts it can observe and the action it can stop:

In-code AI agent security

In-code security runs inside the application, tool handler, queue consumer, or workflow step. The developer passes trusted application context and untrusted inputs to a policy check immediately before the protected operation.

This placement has the richest context. It can know the authenticated user, on-behalf-of agent identity, tenant, route, tool, target object, transaction amount, data classification, recent workflow steps, and accumulated cost. It can also protect non-HTTP execution paths that a network appliance cannot see.

What in-code controls can enforce

• Object-level and action-level authorization
• Tool argument validation and capability restrictions
• Prompt-injection checks on fetched pages, documents, and tool output
• Sensitive-data checks before model, log, or third-party transmission
• Per-user, per-agent, and per-organization budgets
• Approval gates for payments, deletions, messages, and other side effects
• Sequence-aware policy using workflow and correlation context
• Queue workers, scheduled jobs, and multi-agent pipelines

Placement changes what can still be prevented. A check immediately before issueRefund() can deny the refund, require approval, or reduce the requested amount while the application still understands the operation. By the time an outbound request reaches a proxy or gateway, the application has already selected the action and serialized its context. That network boundary may be the last chance to stop transport, but it is too late to reverse local state changes, emitted queue messages, or data already read into model context.

Arcjet's introduction to security inside the agent loop describes this shift from request boundaries to function and tool boundaries. Untrusted input can enter through a queue, retrieved page, or tool response, so enforcement has to follow the code rather than assume every operation has an HTTP front door.

In-code trade-offs

Coverage depends on correct integration. A new tool or alternate code path can omit the check. SDK upgrades and policy behavior must remain consistent across languages and services. Synchronous remote decisions can add latency or create availability dependencies.

Reduce these risks with shared wrappers around consequential operations, centrally governed policy, code-review checks, integration tests that prove denied actions never execute, and telemetry that identifies unprotected tools. Configure deterministic local validation separately from remote analysis, and define explicit fail-open or fail-closed behavior per operation.

Proxy and WAF architecture

A proxy or web application firewall sits between a network client and an HTTP application. It can terminate TLS, normalize requests, enforce network policy, detect bots, limit traffic, validate coarse request properties, and block known web attacks before traffic consumes application resources.

What proxies can enforce

• DDoS mitigation and connection controls
• IP, ASN, geography, and network reputation
• Bot and browser-signal enforcement at public entrypoints
• Coarse per-route or per-credential rate limits
• HTTP method, header, body-size, and content-type restrictions
• Known injection and web exploit signatures
• Central coverage for services behind the same ingress

These controls remain important for agent endpoints. An attacker does not need to compromise model reasoning if they can exhaust the endpoint, automate account creation, or flood an expensive inference route.

Proxy blind spots

A proxy sees requests, not application meaning. It may not know the authenticated tenant after session resolution, whether an invoice belongs to that tenant, which tool the model calls later, what a queue worker processes, or whether a payment follows a bank-detail change in the same run.

Internal tools and background jobs may never cross the proxy. Even when a tool uses HTTP, traffic may take a different service-to-service path. Encrypting application payloads or placing prompts inside provider-specific formats can further reduce inspection quality.

A proxy is also a shared point of failure. Redundant instances can keep the network path available, but applications commonly see a denied, timed-out, or unavailable proxy as a connection failure or generic HTTP error. They cannot safely distinguish a policy denial from an outage unless the proxy provides a stable, authenticated error contract. Retries can then repeat an action whose outcome is uncertain.

An in-code check can return a typed result such as deny, approval_required, or policy_unavailable. The application can respond by removing write tools, continuing in read-only mode, asking the user for approval, or returning a specific error without starting the side effect. A remote in-code policy service can still fail, so each action needs an explicit fallback and local validation for rules that must remain available.

Use a proxy to reject hostile or excessive ingress early, but do not treat successful passage as authorization for downstream agent actions.

AI gateway architecture

An AI gateway sits between application code and one or more model providers. It standardizes model APIs and can inspect prompts, model responses, embeddings, and sometimes proposed tool calls. It is a useful central point for provider routing, usage accounting, caching, observability, and model-specific policy.

What AI gateways can enforce

• Provider credentials and model allowlists
• Model routing, fallback, and residency policy
• Token and monetary accounting at model-call boundaries
• Prompt and response logging with controlled redaction
• Prompt-injection or sensitive-data scanning before inference
• Output moderation and schema validation
• Central limits on model, tenant, or API-key usage

An AI gateway has deeper model visibility than a general HTTP proxy. It can parse provider-specific request structures and apply consistent controls when multiple applications use the same models.

Gateway blind spots

The gateway sees what is sent to and returned from the model. It may not know whether a proposed tool call actually executes, whether application code modifies its arguments, which database row is accessed, or whether an internal API call is permitted. A tool can run in a background job after the model call has completed, and some deterministic tools may not involve a model call at all.

A gateway also has only the context the application sends. Forwarding all session and business data to improve decisions can create new privacy and coupling risks. If applications can call providers directly, gateway coverage is bypassable unless egress and credential controls enforce the route.

Use an AI gateway to govern model traffic, not as the sole policy enforcement point for business side effects.

Security agent or supervisor architecture

A security agent, guardrail model, or supervisor evaluates another agent's inputs, plans, tool calls, or trajectory. It may run before each step, review only consequential actions, or analyze workflow history asynchronously.

This architecture can reason about semantic and sequence-level risk that static rules cannot enumerate. A supervisor can notice that an untrusted email changed bank details and that a payment to the new account immediately followed, even though each call is individually permitted. Arcjet's anatomy of an agent incident shows how risk can emerge from the order of otherwise permitted actions.

What supervisory agents can evaluate

• Interpreting ambiguous natural-language tasks
• Comparing a proposed action with the user's stated goal
• Reviewing workflow history and cross-step relationships
• Assigning risk and requesting human approval
• Explaining why an unusual sequence deserves investigation
• Asynchronous incident triage and policy recommendations

Security-agent risks

A model-based supervisor is probabilistic and can be wrong. It can receive the same poisoned context as the protected agent, become a second prompt-injection target, or repeat the same unsafe assumptions. Running another model adds latency and cost, and a timeout creates an ambiguous failure state.

A supervisor is advisory unless it controls an enforceable gate. If it returns “deny” after the tool has already executed, it is observability rather than prevention. Keep the final side-effect boundary deterministic: validate the supervisor's structured output, constrain its authority, and let application policy decide how a risk result affects execution.

Do not give the security agent broader credentials than the agent it monitors. Prefer read-only access to normalized workflow evidence, keep untrusted tool text separate from trusted policy guidance, and test whether injected strings can cross that boundary.

Architecture comparison

In-code security has the most application context and the shortest path to a side effect. Proxies and AI gateways provide broader centralized coverage at narrower technical boundaries. Security agents add semantic judgment, but they need a deterministic enforcement point to make that judgment preventive.

Which architecture handles each AI agent threat?

No single architecture covers every AI agent threat. Assign the primary control to the boundary where the harmful operation can still be stopped, then use earlier layers to reduce hostile traffic and uncertain inputs.

A layered reference architecture

A strong production design assigns one responsibility to each layer and keeps the final action gate inside the application:

1. Reject hostile ingress early. The proxy applies network controls, bot detection, request limits, and basic HTTP validation.
2. Establish trusted identity. The application resolves the controlling user, logical agent, workload, tenant, and delegated authority from trusted credentials.
3. Protect model traffic. The AI gateway enforces provider, model, token, data, and prompt policy around inference.
4. Treat model output as untrusted. Application code validates structured output and never grants authority because the model requested it.
5. Guard every consequential tool. In-code policy validates arguments, authorizes the exact object and action, applies budgets, and checks approvals immediately before execution.
6. Correlate the workflow. Stable run and decision IDs connect inputs, model calls, tool results, and actions without logging secrets or full sensitive payloads.
7. Add supervision selectively. A security agent reviews ambiguous or high-risk sequences and returns structured risk evidence to the deterministic gate.
8. Verify the outcome. Record whether the side effect occurred and reconcile external systems when execution is uncertain.

This follows the “floor before ceiling” model in the two speeds of AI agent runtime security. Establish dependable enforcement and evidence before relying on increasingly dynamic reasoning.

Design failure behavior per action

Security components fail through timeouts, network partitions, malformed output, stale policy, or provider outages. A single global fail-open setting is too coarse.

Fail closed for irreversible or high-impact operations such as payments, permission changes, external sends, destructive writes, and sensitive-data export. Fail open may be acceptable for low-risk reads or non-sensitive public content when availability matters more than the missed check. A third option is fail restricted: allow the workflow to continue with read-only tools, reduced budgets, redacted data, or mandatory approval.

Set a bounded decision timeout and make the fallback explicit in code. Distinguish “policy allowed” from “control unavailable” in telemetry. Never convert a timeout into an allow decision without recording that enforcement degraded.

For model-based supervisors, invalid or unparseable output should not be interpreted as approval. Validate a narrow response schema and default high-impact actions to deny or approval-required.

Preserve trust boundaries in security telemetry

Central visibility is useful only if telemetry does not create another injection or data-loss path. Keep trusted policy conclusions separate from untrusted prompts, request metadata, documents, and tool output. Arcjet's pattern for defending MCP tool outputs from prompt injection is that trusted guidance must never interpolate attacker-controlled text.

Record normalized identifiers, policy versions, reason codes, risk categories, budgets, correlation IDs, and outcomes. Store raw evidence under clearly labeled untrusted fields with strict access and retention. Do not log bearer tokens, complete prompts, secrets, or full sensitive documents simply because a security agent might want more context later.

How to choose an architecture

• Choose proxy controls when the threat exists before application code: bots, volumetric abuse, malformed HTTP, network reputation, and coarse ingress limits.
• Choose an AI gateway when the policy concerns model providers, prompts, responses, tokens, routing, residency, or centralized inference accounting.
• Choose in-code controls when the decision depends on identity, tenant, object, tool arguments, budgets, approvals, business rules, or non-HTTP execution.
• Choose a security agent when natural-language intent or workflow sequence materially changes risk and deterministic policy cannot express the full judgment.
• Combine layers when a threat crosses boundaries. Sensitive-data protection, for example, benefits from in-code minimization before dispatch and gateway scanning as defense in depth.

Do not start by purchasing one layer and assuming it covers the rest. Start with a consequential workflow, identify the last reversible point before harm, place a deterministic enforcement check there, and add earlier controls that reduce load and uncertainty.

AI agent security architecture checklist

Where should AI agent security controls be placed?

1. Map HTTP, queue, scheduler, model, retrieval, tool, and side-effect boundaries.
2. Mark which components can observe and synchronously stop each consequential action.
3. Put network and bot controls at public ingress.
4. Route model traffic through approved providers and enforce egress so the gateway cannot be bypassed.
5. Keep user, agent, workload, tenant, and delegated identities distinct.
6. Validate all model-generated tool names and arguments as untrusted input.
7. Add an in-code policy check immediately before every consequential side effect.
8. Apply prompt-injection and sensitive-data checks where untrusted content enters or re-enters model context.
9. Enforce budgets at request, model-call, workflow, tool, user, and organization levels as appropriate.
10. Connect supervisory reasoning to a deterministic action gate; never treat free-form model output as authorization.

How should protected actions fail safely?

11. Define fail-open, fail-closed, or fail-restricted behavior for every protected action.
12. Correlate decisions and outcomes without placing secrets or raw untrusted content in trusted telemetry fields.

How should an agent security architecture be tested?

13. Test direct-provider bypass, alternate tool paths, background jobs, timeouts, malformed supervisor output, and partial execution.
14. Review coverage whenever the agent gains a model, data source, tool, permission, or execution path.