AI agent bot management

What is AI agent bot management?

AI agent bot management is the practice of detecting and classifying AI agents on your customer-facing interfaces (logins, checkouts, forms, product pages), and then enforcing a business policy: allow, restrict to a path, or block. The agents in question are clients of your site, not the agents that your engineers deploy.

The objects of AI agent bot management are crawlers from AI platforms, agentic browsers, shopping agents, and "do this for me" assistants that hold a user's cookies or credentials. The practice is not toxicity scanning, not supervising internal agents, and not a CAPTCHA product.

AI agent bot management is a management problem rather than a "block bots" problem because some of this traffic is desired: your catalog in an assistant's recommendations, or a customer sending an agent to reorder. Blind blocking is a business decision with revenue effects, not a default security posture to inherit from a CDN rule.

How is this different from Guardian Agents?

Guardian Agents supervise the agents that your organization runs, whereas AI agent bot management classifies the agents that call your site. It is a descendant of bot management and fraud controls on public interfaces, not an oversight product for internal workers. Mixing the terms leads to buying the wrong control.

Gartner's February 2026 Market Guide for Guardian Agents (G00836388, 25 February 2026) named the oversight category for buyers who need automated supervision of agentic systems that they operate. AI agent bot management points the other way: inbound HTTP on surfaces that you publish. A Guardian Agent watches your copilot as it calls tools. AI agent bot management decides what happens when someone else's agent posts to /login.

For more information about the oversight category, see Guardian Agents.

Why "bot versus human" is not enough

Classic bot management sorted traffic into human and bot, then blocked the bot. AI agents break that binary. They can hold a real user's session, arrive from a handful of well-known platforms, and perform a wide range of actions whose intent you cannot read from a TLS fingerprint.

You already know human versus bot. What classic bot management doesn't give you is the type and intent of AI-agent traffic. Agents from the same platform look alike, so provenance is not intent. A shopping assistant and a credential-stuffing script can share a User-Agent family, a small set of egress IPs, and a similar TLS profile.

Credential stuffing, scraping, and inventory grabbing still exist; AI agents can perform them. Those patterns are API abuse: valid operations used for a harmful outcome. The OWASP API Security Top 10 (2023) treats unrestricted access to sensitive business flows and unrestricted resource consumption as distinct risks from injection.

The false binary also harms commerce. Blocking every client in an AI category can drop you out of assistant recommendations. Allowing every client that looks like a platform agent can open login to account takeover.

Identity-binding of a human to the agent that presents the user's cookies or password is still immature. If an agent uses the account password, the site cannot tell whether the account holder sent it. A User-Agent string does not close that gap. Protocols that bind user intent to agents are emerging and unsettled. Watch them; do not bet login policy on one specification.

What are the three jobs: visibility, intent/risk, and allow/scope/block?

You need three capabilities, in order. First, see that an AI agent is present and which kind. Then estimate what it is trying to do and how risky that is on this route. Then allow it, confine it to a path or data set, or block it. Skipping to "block all AI" is a policy, not a capability.

Visibility in AI agent bot management means distinguishing AI-platform crawlers (training, index, or answer-engine fetchers) from agentic browsers and from task agents completing a purchase. Combine a known-bot list and managed categories with a policy for unknown automation, and measure what share of login and checkout traffic comes from agents.

Intent and risk in AI agent bot management treat the same User-Agent as capable of price research, credential testing, or a legitimate reorder. Useful signals include route, authentication state, velocity, outcome (many holds, few purchases), and whether the session looks delegated. Risk is contextual: a documented crawler on /blog is not the same crawler on /login.

Allow, scope, or block is the enforcement job: full access, a confined path or data set, or a deny. The following table gives the route-by-route version, where stuffing, authenticated scraping, and inventory hoarding belong in the block column.

What is agentic commerce doing to your login and checkout?

Agentic commerce is an AI system participating in discovery, comparison, or purchase on a merchant site. It might arrive as a crawler feeding recommendations, as an agent completing checkout, or as a platform that never shows the user your page. You need a policy for each pattern; you will not get one by blocking "all bots."

Recommendation fetch. AI platforms scrape or fetch product data so their users see your SKU in an answer or a shopping suggestion. The shopper might never load your page. Blocking the wrong crawler can make you disappear from that channel, a marketing problem, the same way blocking Googlebot is an SEO problem.

Delegated checkout. A customer asks an assistant to buy. The assistant's agent reaches your checkout with a credential, token, or guest session. Commerce cares about conversion and fraud on the same request. A burst of holds that never pay is the inventory-grab pattern from API abuse.

Customer-owned agents. A personal agent arrives with the user's cookies. Customer-owned agents are still uncommon, and they carry the most identity ambiguity. The site sees a session that looks like the account holder, without a reliable binding that the account holder sent the agent.

Mismanagement of agentic commerce cuts both ways. Block the wrong crawler and you lose discovery. Allow the wrong agent on login and you take on account-takeover risk. Standards and protocols for agent identity and commerce are in draft in several places at once. That unsettled state is not a reason to wait before logging, classifying, and writing a route-by-route policy.

Why do edge bot products see less application intent than a request handler?

Edge bot products sit on the TLS connection and the HTTP request, so they see volumetric filters, IP reputation, and client signals. They do not see that this request is a logged-in checkout for tenant X with a limited SKU. A check in the request handler sees that application intent.

Edge bot vendors (for example Akamai or Cloudflare; see Cloudflare vs Arcjet) see connection and request signals. Edge products do not see your application's user, tenant, and workflow state unless you forward it. Edge products are strong at DDoS, obvious automation, known crawlers, and JavaScript challenges, and weak at object-level meaning: per-subscription policy, identity-aware rate limits, and an inspectable deny in the handler.

A check in the request handler sees the logged-in user and tenant, so it can run bot detection beside a token bucket keyed by userId, with different rules on /login versus /checkout. For more information about who authors that rule, see application-native versus remote security policies. For more information about proxy versus in-code placement, see AI agent security architecture. Edge bot products and in-app checks are complementary: keep the edge for volumetric attack and add in-app controls for intent.

How do you classify AI agents without making CAPTCHA the product?

Classify AI agents with known-bot identities and categories, verification (is this really Google?), and advanced bot signals such as reputation, machine learning, and browser telemetry. Use CAPTCHA only as a graduated challenge when risk is high and a human is present. For more information about why a puzzle is the wrong illustration of this control, see CAPTCHAs vs Arcjet.

Known bots and categories let you allow search-engine crawlers on public pages and decide explicitly, route by route, what to do with AI-purpose crawlers: CATEGORY:SEARCH_ENGINE versus CATEGORY:AI. Verification exists because clients pretend to be Googlebot. It checks the client against the operator's published IP space, using reverse DNS and IP ownership, which are the industry-standard checks. For more information about verification, see identifying bots. Unknown automation still exists. A strict allowlist blocks unidentified bots; a permissive list lets them through. That is a choice, not a default to inherit from a CDN dashboard.

Advanced bot signals – browser telemetry collected by a client-side WebAssembly module, combined with server-side signals such as IP reputation and verification – are evidence, not identity. Advanced signals raise or lower confidence that the client is automated; they do not tell you the human intended the purchase. The 3 June 2026 announcement is the product story: the browser gives signals, the application gives intent.

Graduated responses, as in the API abuse article, run in order: log, tighter quota, delay, step-up, then block. CAPTCHA is one step-up, with known failures (accessibility, farms, broken legitimate automation) spelled out in CAPTCHAs vs Arcjet. The illustration for AI agent bot management is category plus Advanced Signals plus per-route policy, not a puzzle widget.

Who has to agree: security, commerce, and marketing?

AI agent bot management policy requires agreement from security, commerce, and marketing: security wants to stop stuffing and fraud, commerce wants delegated checkout to complete, and marketing wants the brand to appear in assistant answers. Those three teams can disagree on CATEGORY:AI, so write the policy down before you flip a rule to live enforcement.

AI agent bot management ownership splits in practice as follows:

• Security owns login, account-takeover, and data-exfiltration routes.
• Commerce owns checkout, inventory, and promotions.
• Marketing / SEO owns the public catalog, the rules for robots and AI crawlers, and answer-engine presence.

Dry-run first. Measure how much traffic would have been denied on product pages versus login. Do not let one team's default ("block all bots") silently apply to /. Blocking Googlebot-like AI crawlers can hide you from recommendations. That is a marketing decision with a security veto on sensitive forms, not the other way around.

How does this relate to API abuse and AI runtime security?

Agent traffic on your site is a special case of API abuse when the agent automates a valid operation for a harmful outcome. Agent traffic is a different problem from AI runtime security, which protects your model and tools. You might need both in the same company: bot management on the storefront, runtime security on the copilot.

What is API abuse? covers stuffing, scraping, hoarding, and cost exhaustion, the operations AI agents can run at higher volume. Rate limiting still applies: identity-aware token buckets constrain agents the same way they constrain scripts. API security best practices remain the floor for authentication and object-level authorization.

AI agent runtime security protects your model, tools, and agent loop. If you expose a chat or copilot, inbound agents can run up your GPU bill; that is bots plus budgets on the chat route, not a reason to merge this page with those guides. This article is the AI-agent-as-client slice on customer-facing HTTP.

How does in-app bot management work?

In-app bot management runs bot detection inside the route, with allow/deny lists of known bots and categories, plus verification and advanced bot signals. It runs alongside rate limits and the rest of the request rules. Arcjet's illustration is CATEGORY:AI and Advanced Signals, not a CAPTCHA widget, and not a Guardian Agent.

Bot detection is an HTTP control in the request handler. Call protect() there (quick start, reference). Tool-call checks have no HTTP request and no bot detection, so they are outside the scope of this page.

CATEGORY:AI is the managed category for clients that scrape or fetch for AI and LLM purposes, not for search indexing. The identifiers live on the public bot list (open source in arcjet/well-known-bots). Allow or deny CATEGORY:AI route by route: often allow on public docs and product pages, deny on /login. CATEGORY:SEARCH_ENGINE is separate.

You can allow a named AI crawler inside a denied category. The AI scrapers post walks through keeping Perplexity while denying the rest of CATEGORY:AI. Verification checks that a client claiming to be a known bot actually comes from that operator's IP space.

Advanced bot signals, announced 3 June 2026, run a WebAssembly collector in the browser, store an aj_signals cookie, and evaluate it server-side inside detectBot, with no CAPTCHA. A missing cookie is itself a signal; filter rules can deny POST requests that never loaded the script.

Compose Shield, detectBot, and a token bucket on login and checkout (or signup protection as the bundled form of bots, email, and rate limit); use a different allowlist on /blog. Start in DRY_RUN. HTTP request checks can fail open when Arcjet's cloud cannot be reached; your handler can treat an errored decision as a deny if you prefer fail-closed. Bot, Shield, filter, and fixed-window or sliding-window rate-limit rules change in the Arcjet dashboard or through MCP and take effect immediately. Token-bucket and email rules stay in code. If you are comparing to a platform bot product, Vercel BotID vs Arcjet is the closest how-to contrast.