AI agent bot management

Detect, classify, and apply policy to AI agents that hit logins, checkouts, scrapers, and APIs.

21 min read
In short: AI agent bot management is detecting, classifying, and applying policy to the AI agents that hit your logins, checkouts, forms, and APIs as clients, not supervising the agents your own engineers run. Classify each client into four kinds (AI agent, AI crawler, automation, other) with an evidence level, and treat delegated agents, which present a verified Personal Agent Protocol session, as a separate policy tier.

What is AI agent bot management?

AI agent bot management is the practice of detecting and classifying AI agents on your customer-facing interfaces (logins, checkouts, forms, product pages), and then enforcing a business policy: allow, restrict to a path, or block. The agents in question are clients of your site, not the agents that your engineers deploy.

The objects of AI agent bot management are crawlers from AI platforms, agentic browsers, shopping agents, and "do this for me" assistants that hold a user's cookies or credentials. The practice is not toxicity scanning, not supervising internal agents, and not a CAPTCHA product.

AI agent bot management is a management problem rather than a "block bots" problem because some of this traffic is desired: your catalog in an assistant's recommendations, or a customer sending an agent to reorder. Blind blocking is a business decision with revenue effects, not a default security posture to inherit from a CDN rule.

How is this different from Guardian Agents?

Guardian Agents supervise the agents that your organization runs, whereas AI agent bot management classifies the agents that call your site. It is a descendant of bot management and fraud controls on public interfaces, not an oversight product for internal workers. Mixing the terms leads to buying the wrong control.

Gartner's February 2026 Market Guide for Guardian Agents (G00836388, 25 February 2026) named the oversight category for buyers who need automated supervision of agentic systems that they operate. AI agent bot management points the other way: inbound HTTP on surfaces that you publish. A Guardian Agent watches your copilot as it calls tools. AI agent bot management decides what happens when someone else's agent posts to /login.

Guardian AgentsAI agent bot management
DirectionOutward from your estate: watch workers that you operateInward: clients that call your HTTP
Typical surfaceTools, data, other agentsLogin, checkout, forms, product, search
FailureYour agent overshares or pays the wrong vendor

Credential stuffing, scraping, inventory grab, or you block a useful shopping agent

BuyerAI leader / security / riskSecurity + commerce + marketing
Adjacent productTRiSM, SPM, supervisorsBot management, fraud, in-app detection

For more information about the oversight category, see Guardian Agents.

Why isn't "bot versus human" enough?

Classic bot management sorted traffic into human and bot, then blocked the bot. AI agents break that binary. They can hold a real user's session, arrive from a handful of well-known platforms, and perform a wide range of actions whose intent you cannot read from a TLS fingerprint.

You already know human versus bot. What classic bot management doesn't give you is the type and intent of AI-agent traffic. Agents from the same platform look alike, so provenance is not intent. A shopping assistant and a credential-stuffing script can share a User-Agent family, a small set of egress IPs, and a similar TLS profile.

Credential stuffing, scraping, and inventory grabbing still exist; AI agents can perform them. Those patterns are API abuse: valid operations used for a harmful outcome. The OWASP API Security Top 10 (2023) treats unrestricted access to sensitive business flows and unrestricted resource consumption as distinct risks from injection.

The false binary also harms commerce. Blocking every client in an AI category can drop you out of assistant recommendations. Allowing every client that looks like a platform agent can open login to account takeover.

If an agent uses the account password or the user's cookies, the site cannot tell whether the account holder sent it, and a User-Agent string does not close that gap. The Personal Agent Protocol (Poppy) closes part of it. An agent that follows it signs in through your OAuth server with scopes the person approves, and signs each request with a session that proves which agent sent it and that it acts for one of its users. It doesn't identify the user, and it doesn't prove that the user approved a specific action, so login and purchase policy still belong to your own authorization code.

What are the three jobs: visibility, intent/risk, and allow/scope/block?

You need three capabilities, in order. First, see that an AI agent is present and which kind. Then estimate what it is trying to do and how risky that is on this route. Then allow it, confine it to a path or data set, or block it. Skipping to "block all AI" is a policy, not a capability.

Visibility in AI agent bot management means distinguishing AI-platform crawlers (training, index, or answer-engine fetchers) from agentic browsers and from task agents completing a purchase. Combine a known-bot list and managed categories with a policy for unknown automation, and measure what share of login and checkout traffic comes from agents.

Intent and risk in AI agent bot management treat the same User-Agent as capable of price research, credential testing, or a legitimate reorder. Useful signals include route, authentication state, velocity, outcome (many holds, few purchases), and whether the session looks delegated. Risk is contextual: a documented crawler on /blog is not the same crawler on /login.

Allow, scope, or block is the enforcement job: full access, a confined path or data set, or a deny. The following table gives the route-by-route version, where stuffing, authenticated scraping, and inventory hoarding belong in the block column.

JobProduct / catalogLogin / accountCheckout
VisibilityWhich AI crawlers request public SKUs this week?Is this an agent holding a session, or stuffing?Agent-attributed carts versus completed orders
Intent / riskRecommendation fetch versus exhaustive scrapeCredential testing versus delegated loginResearch versus inventory grab versus reorder
Allow / scope / blockAllow verified AI crawlers on public catalog; deny account APIsBlock agents on login by default; step-up if you ever allow

Allow named agents on an existing session; deny unknown automation on new-account checkout

What is agentic commerce doing to your login and checkout?

Agentic commerce is an AI system participating in discovery, comparison, or purchase on a merchant site. It might arrive as a crawler feeding recommendations, as an agent completing checkout, or as a platform that never shows the user your page. You need a policy for each pattern; you will not get one by blocking "all bots."

Recommendation fetch. AI platforms scrape or fetch product data so their users see your SKU in an answer or a shopping suggestion. The shopper might never load your page. Blocking the wrong crawler can make you disappear from that channel, a marketing problem, the same way blocking Googlebot is an SEO problem.

Delegated checkout. A customer asks an assistant to buy. The assistant's agent reaches your checkout with a credential, token, or guest session. Commerce cares about conversion and fraud on the same request. A burst of holds that never pay is the inventory-grab pattern from API abuse.

Customer-owned agents. A personal agent acts for one person. If it runs in that person's browser or reuses their cookies, it carries the most identity ambiguity: the site sees a session that looks like the account holder, with nothing that shows the account holder sent the agent. If it follows the Personal Agent Protocol, as Meta's Muse does, it presents a signed session and an OAuth token instead, and Arcjet reports it as a delegated agent.

Mismanagement of agentic commerce cuts both ways. Block the wrong crawler and you lose discovery. Allow the wrong agent on login and you take on account-takeover risk. Web Bot Auth is an IETF draft, and the Personal Agent Protocol published its 1.0 specification on November 14, 2026. You can handle agents that identify themselves through either one by name. The rest still need logging, classification, and a route-by-route policy. For more information about the checkout flow step by step, see agentic commerce bot protection.

Why do edge bot products see less application intent than a request handler?

Edge bot products sit on the TLS connection and the HTTP request, so they see volumetric filters, IP reputation, and client signals. They do not see that this request is a logged-in checkout for tenant X with a limited SKU. A check in the request handler sees that application intent.

Edge bot vendors (for example Akamai or Cloudflare; see Cloudflare vs Arcjet) see connection and request signals. Edge products do not see your application's user, tenant, and workflow state unless you forward it. Edge products are strong at DDoS, obvious automation, known crawlers, and JavaScript challenges, and weak at object-level meaning: per-subscription policy, identity-aware rate limits, and an inspectable deny in the handler.

A check in the request handler sees the logged-in user and tenant, so it can run bot detection beside a token bucket keyed by userId, with different rules on /login versus /checkout. For more information about who authors that rule, see application-native versus remote security policies. For more information about proxy versus in-code placement, see AI agent security architecture. Edge bot products and in-app checks are complementary: keep the edge for volumetric attack and add in-app controls for intent.

LayerWhat it sees wellWhat it typically misses
Edge bot product (for example Akamai or Cloudflare)

TLS, IP, client fingerprints, known crawlers, JS challenges, volumetric filters

Tenant, logged-in user, cart contents, prior MFA failures, per-route business policy unless you forward them

Request handler

Route, userId, tenant, workflow state, composed rate limits, typed deny in the handler

Network-edge DDoS and traffic you never send to the origin

How do you classify AI agents without making CAPTCHA the product?

Classify AI agents into four kinds – AI agent, AI crawler, automation, and other – and record how strongly each identity was proven, using known-bot identities and categories, verification (is this really Google?), and advanced bot signals. Use CAPTCHA only as a graduated challenge when risk is high and a human is present. For more information about why a puzzle is the wrong illustration of this control, see CAPTCHAs vs Arcjet.

When Arcjet Cloud detects a bot, the bot rule's reason includes one primary bot identity. Its kind field puts the client in one of four kinds:

KindWhat it isTypical policy
AI_AGENT

An AI product that fetches a page or acts on demand for a person who is waiting on the answer. In CATEGORY:AI_AGENT.

Allow on public routes with a per-agent rate limit
AI_CRAWLER

An AI bot that collects pages ahead of time, for model training or an AI search index. In CATEGORY:AI but not CATEGORY:AI_AGENT.

A marketing decision: allow or deny route by route
AUTOMATION

Browser automation that advanced signals detected, with no named bot

Deny on forms, login, and checkout
OTHER

Any other detected bot, such as a search engine, a monitor, or a tool

Decide by bot ID or category

The identity also has an evidence field, from weakest to strongest: UNVERIFIED (a User-Agent claim or IP reputation), FINGERPRINT (advanced signals detected automation), IP (the operator's published ranges or reverse DNS), and SIGNATURE (a Web Bot Auth signature or a Personal Agent Protocol session). spoofed is true when IP or reverse DNS verification showed that a claim is false. A kind without its evidence is a guess: AI_AGENT with UNVERIFIED evidence is only what the client said about itself. For more information about the evidence levels, see AI agent evidence levels.

Known bots and categories let you allow search-engine crawlers on public pages and decide explicitly, route by route, what to do with AI-purpose crawlers: CATEGORY:SEARCH_ENGINE versus CATEGORY:AI. Verification exists because clients pretend to be Googlebot. It checks the client against the operator's published IP space, using reverse DNS and IP ownership, which are the industry-standard checks. For more information about verification, see identifying bots. Unknown automation still exists. A strict allowlist blocks unidentified bots; a permissive list lets them through. That is a choice, not a default to inherit from a CDN dashboard.

Advanced bot signals – browser telemetry collected by a client-side WebAssembly module, combined with server-side signals such as IP reputation and verification – are evidence, not identity. Advanced signals raise or lower confidence that the client is automated; they do not tell you the human intended the purchase. The 3 June 2026 announcement is the product story: the browser gives signals, the application gives intent.

Graduated responses, as in the API abuse article, run in order: log, tighter quota, delay, step-up, then block. CAPTCHA is one step-up, with known failures (accessibility, farms, broken legitimate automation) spelled out in CAPTCHAs vs Arcjet. The illustration for AI agent bot management is category plus Advanced Signals plus per-route policy, not a puzzle widget.

Why are delegated agents a separate policy tier?

Delegated agents are a separate tier because they are the only automated clients that prove they act for one of your users. A delegated agent presents a verified Personal Agent Protocol session, signed by a key that the agent publishes on its own domain, and signs in through your OAuth server with scopes the person approved. Arcjet reports it with kind AI_AGENT, SIGNATURE evidence, the agent's domain as operator, and delegated set to true.

That gives four tiers for AI agent traffic, each with a different ceiling:

  • Delegated agent (delegated is true). It can act within the scopes and limits that your authorization server granted, such as reading order history or adding to a cart. Purchases above a spend limit and account changes wait for the person.
  • Verified agent (SIGNATURE or IP evidence, no delegation). You know the operator, not the user. Allow it on public routes with a per-agent rate limit, and require the person's own session for anything that writes.
  • Claimed agent (UNVERIFIED evidence). Treat the name as a claim. Give it a tight per-IP limit on public routes and deny it on login and checkout.
  • Automation without a name (AUTOMATION, FINGERPRINT evidence). Deny it on forms, login, and checkout.

A delegated session still isn't authorization on its own. It doesn't identify the user: the user's sub is a pseudonymous ID that's different at each business, and Arcjet doesn't log or store it. It also doesn't prove that the person approved this specific action. Arcjet checks the session against the request, and your authorization server checks the access token and its scopes, so the tier decides what the agent may attempt and your code decides what it may do.

Arcjet verifies sessions only for agents vetted in the open well-known-bots catalog, starting with Meta's Muse (META_MUSE). A session from an agent outside the catalog, or one that fails verification, doesn't change the identity. The following JavaScript reads the tier from a decision:

import { isAiAgent, isSpoofedBot } from "@arcjet/inspect";
if (decision.results.some(isSpoofedBot)) {
// A forged bot claim: deny the request before reading the tier
}
for (const result of decision.results) {
if (isAiAgent(result) && result.reason.isBot()) {
const agent = result.reason.identity;
if (agent?.delegated) {
// A verified Personal Agent Protocol session: check the OAuth token
// and its scopes with your authorization server
}
}
}

For more information about the protocol, see what is the Personal Agent Protocol. For a checkout flow with a policy at each step, see agentic commerce bot protection.

Who has to agree: security, commerce, and marketing?

AI agent bot management policy requires agreement from security, commerce, and marketing: security wants to stop stuffing and fraud, commerce wants delegated checkout to complete, and marketing wants the brand to appear in assistant answers. Those three teams can disagree on CATEGORY:AI, so write the policy down before you flip a rule to live enforcement.

AI agent bot management ownership splits in practice as follows:

  • Security owns login, account-takeover, and data-exfiltration routes.
  • Commerce owns checkout, inventory, and promotions.
  • Marketing / SEO owns the public catalog, the rules for robots and AI crawlers, and answer-engine presence.

Dry-run first. Measure how much traffic would have been denied on product pages versus login. Do not let one team's default ("block all bots") silently apply to /. Blocking Googlebot-like AI crawlers can hide you from recommendations. That is a marketing decision with a security veto on sensitive forms, not the other way around.

Agent traffic on your site is a special case of API abuse when the agent automates a valid operation for a harmful outcome. Agent traffic is a different problem from AI runtime security, which protects your model and tools. You might need both in the same company: bot management on the storefront, runtime security on the copilot.

What is API abuse? covers stuffing, scraping, hoarding, and cost exhaustion, the operations AI agents can run at higher volume. Rate limiting still applies: identity-aware token buckets constrain agents the same way they constrain scripts. API security best practices remain the floor for authentication and object-level authorization.

AI agent runtime security protects your model, tools, and agent loop. If you expose a chat or copilot, inbound agents can run up your GPU bill; that is bots plus budgets on the chat route, not a reason to merge this page with those guides. This article is the AI-agent-as-client slice on customer-facing HTTP.

How does in-app bot management work?

In-app bot management runs bot detection inside the route, with allow/deny lists of known bots and categories, plus verification and advanced bot signals. It runs alongside rate limits and the rest of the request rules. Arcjet's illustration is CATEGORY:AI and Advanced Signals, not a CAPTCHA widget, and not a Guardian Agent.

Bot detection is an HTTP control in the request handler. Call protect() there (quick start, reference). Tool-call checks have no HTTP request and no bot detection, so they are outside the scope of this page.

CATEGORY:AI is the managed category for clients that scrape or fetch for AI and LLM purposes, not for search indexing. The identifiers live on the public bot list (open source in arcjet/well-known-bots). Allow or deny CATEGORY:AI route by route: often allow on public docs and product pages, deny on /login. CATEGORY:SEARCH_ENGINE is separate.

You can allow a named AI crawler inside a denied category. The AI scrapers post walks through keeping Perplexity while denying the rest of CATEGORY:AI. Verification checks that a client claiming to be a known bot actually comes from that operator's IP space.

Advanced bot signals, announced 3 June 2026, run a WebAssembly collector in the browser, store an aj_signals cookie, and evaluate it server-side inside detectBot, with no CAPTCHA. A missing cookie is itself a signal; filter rules can deny POST requests that never loaded the script.

Compose Shield, detectBot, and a token bucket on login and checkout (or signup protection as the bundled form of bots, email, and rate limit); use a different allowlist on /blog. Start in DRY_RUN. HTTP request checks can fail open when Arcjet's cloud cannot be reached; your handler can treat an errored decision as a deny if you prefer fail-closed. Bot, Shield, filter, and fixed-window or sliding-window rate-limit rules change in the Arcjet dashboard or through MCP and take effect immediately. Token-bucket and email rules stay in code. If you are comparing to a platform bot product, Vercel BotID vs Arcjet is the closest how-to contrast.

Frequently asked questions

Should I block all AI agents?

No. Decide per route: product pages and docs may want verified AI crawlers; login probably should not. Measure in dry-run before enforcing.

Is this the same as Guardian Agents?

No. Guardian Agents watch agents you run; AI agent bot management classifies agents that call your site.

What are the four kinds of bot in an Arcjet bot identity?

AI_AGENT is an AI product acting on demand for a person who is waiting. AI_CRAWLER collects pages ahead of time for training or an AI index. AUTOMATION is browser automation that advanced signals detected with no named bot. OTHER is any other detected bot. Each identity also carries an evidence level: UNVERIFIED, FINGERPRINT, IP, or SIGNATURE.

What is a delegated agent?

A delegated agent is an AI agent that presents a verified Personal Agent Protocol session, which proves which agent sent the request and that it acts for one of its users. Arcjet reports it with SIGNATURE evidence and delegated set to true. It doesn't identify the user or prove that the user approved a specific action, so your authorization server still checks the token and scopes.

Will CAPTCHA stop AI agents?

CAPTCHA stops some browsers some of the time. It fails accessibility, can be outsourced, and does not encode per-route policy. Treat it as an occasional step-up, not the product. Advanced Signals plus category and per-route policy are the alternative.

Can Cloudflare or Akamai do this alone?

Edge bot products can do a lot at the connection: volumetric filters, known crawlers, JS challenges. Edge products cannot apply tenant- and workflow-aware policy unless you build that and forward the context. Use the edge for volumetric attack and a check in the request handler for intent.

How do I tell a shopping agent from a scraper?

You often cannot from User-Agent alone. Read the bot identity's kind and evidence: a signed or IP-verified AI agent is the operator it names, and a delegated agent acts for one of its users. For the rest, combine Advanced Signals, route, auth state, velocity, and outcomes such as holds versus purchases. Start in dry-run.

Does this protect my own copilot?

Only as inbound abuse on that copilot's HTTP route: bots burning your model bill. Tool and action safety is AI agent runtime security, not AI agent bot management.

Application security in your code

Protect your application with Arcjet

Get allow/deny by AI-agent category, inspectable decisions, and advanced client signals in your request handlers.