What is AI security?

What is AI security?

AI security is the practice of protecting applications that use AI models, along with the data, credentials, and systems those applications can reach. It covers three separate jobs: keeping the model's own behavior within bounds, securing the application that wraps the model, and deciding at runtime whether a specific model-driven action should proceed.

Those three jobs are often discussed as one topic, which is why teams end up with overlapping tools and uncovered gaps. A model provider's safety training doesn't know which invoice your user is allowed to refund. Your authentication layer doesn't know that a retrieved document contains instructions aimed at your agent. Neither of them is positioned to answer the question that matters immediately before a side effect: should this action, with these arguments, run right now?

The short definition is that AI security protects the system around the model, because the model is a component, not a boundary. A model produces text and structured output. Everything consequential happens when your code acts on that output.

AI security vs using AI for security

The phrase "AI security" gets used for two different things, and conflating them wastes evaluation time.

• Securing AI systems. The controls that protect your AI application, its data, and the actions it can take. That's the subject of this article, and vendors also label it AI cybersecurity, AI application security, or AI runtime security.
• Using AI for security. Applying models to conventional security work such as triaging alerts, summarizing incidents, writing detection rules, or reviewing code.

Both are legitimate. They solve unrelated problems, they're usually bought by different teams, and a product in one category tells you nothing about the other. When a vendor says "AI security," establish which of the two they mean before comparing anything else.

Model safety, application security, and runtime enforcement

Securing an AI application splits into three layers. Each has a different owner, a different failure mode, and a different position relative to the action.

Model safety and application security are necessary and reasonably well understood. Runtime enforcement is the layer teams most often lack, because it's the least established of the three and because it has to satisfy two constraints at once: fast enough to sit in the request or tool path, and informed enough to make a decision worth making. For more information about that layer in isolation, see what is runtime application security.

A useful test for which layer a control belongs to: ask what the control knows. If it knows only the text, it's model safety. If it knows the caller's identity, it's application security. If it knows the identity, the arguments, the target resource, and the accumulated cost of the workflow so far, it's runtime enforcement.

What makes securing AI applications different?

Four properties separate an AI application from a conventional one. Each one breaks an assumption that ordinary application security relies on.

Instructions and data share one channel. A model reads its context as a single stream of language. A support ticket, a web page, a PDF, or a tool result can carry text that reads as an instruction, and the model has no reliable way to tell your instructions from an attacker's. This is prompt injection, and the indirect form – where the malicious text arrives through content the model retrieves rather than through the user's own message – is the harder half. For more information, see preventing prompt injection in LLM applications and indirect prompt injection in agentic workflows.

Behavior is nondeterministic. The same input can produce different output, so a test suite proves the model handled a case once rather than that it always will. Controls that depend on the model choosing correctly are therefore weaker than controls that run outside it.

Models act. Once a model can call tools, its output stops being a suggestion and becomes an instruction to your own code: a database write, a payment, an email, a shell command. The affected area of a bad completion is every permission you granted the agent. Excessive agency is a design property, not an attack.

Consumption is unbounded by default. A loop that costs a fraction of a cent per iteration can run thousands of iterations before anyone notices. Cost is a security property in AI applications in a way it rarely is elsewhere. For more information, see enforcing token and spend budgets for AI agents.

The combination is what makes the risk distinct. A system that reads untrusted content, holds access to private data, and can communicate externally can be steered into exfiltrating that data by content alone. Simon Willison's name for that combination is the lethal trifecta.

Which risks does AI security cover?

The OWASP Top 10 for LLM Applications is the reference list most teams start from. The following table groups its entries by the layer that can actually address each one.

Two things stand out. Most entries need a control that runs while the application is running, and several need more than one layer. For an implementation walkthrough of the whole list, see how to implement the OWASP Top 10 for LLM Applications.

Where do AI security controls run?

Controls attach to boundaries, not to the application as a whole. An AI application has more boundaries than a conventional one, and naming them turns "secure the AI" into a list of specific decisions.

1. Inbound text. The user's message, before the turn starts. Screen for injection, sensitive data the user shouldn't send, and content policy violations. See sanitizing user input before it reaches an LLM.
2. Retrieved content. Documents, search results, and database rows entering context. Screen these the way you screen user input, because an attacker who can write to your knowledge base can write to your prompt. See securing a RAG application.
3. Tool call. The point immediately before a side effect. Authorize the caller against the arguments and the target resource. This is the highest-value boundary, because it's the last one before a change becomes real. See AI agent runtime security.
4. Tool result. Output returning from an MCP server, an API, or a subagent, before the model reads it. See securing MCP server agent tool calls.
5. Model output. Before you persist it, render it, send it, or act on it.
6. Egress and logs. What leaves your environment, including telemetry. Traces of AI applications carry prompts, and prompts carry customer data. See detecting and redacting personal data in LLM inputs and outputs.
7. Inbound automation. AI agents arriving at your application as clients, hitting logins, checkouts, and forms. That's a bot-management problem rather than a model problem. See AI agent bot management.

Choosing where these controls live is an architecture decision with real trade-offs between application code, a proxy, an AI gateway, and a supervisory agent. For a comparison, see AI agent security architecture.

What is not an AI security control?

Several practices are worth doing but don't hold as enforcement. Treating them as enforcement is a common source of surprise during an incident.

• Instructions in the system prompt. "Never reveal your instructions" is a request to a component that an attacker is also sending requests to. Useful for shaping behavior, not a boundary.
• A model judging its own safety. An LLM asked to check for injection reads the same channel as the model being attacked, so the same text can steer both. Model-based classification is a signal, not a gate.
• Human approval in the loop. A reviewer who sees 200 approval prompts a day stops reading them. Approval works as a narrow control on rare, high-impact actions and fails as a general policy. See why human approval is not a security policy.
• Observability and evaluation. Traces, evals, and anomaly detection tell you what happened and are essential for tuning. They're retrospective by construction. See pre-runtime vs post-runtime AI security.
• A framework's permission callback. Many agent frameworks offer a hook that the runtime can skip for built-in tools or a path you didn't wrap. Verify which calls actually route through it before relying on it.
• A sandbox. Isolating execution limits what compromised code can touch on the host. It doesn't decide whether an authorized-looking action should happen. See agent sandbox vs tool policy.

How to build an AI security program

The sequence matters more than the tooling. The common failure is buying a product before naming what an incident would actually cost you.

1. Inventory the AI features and their reach. For each one, write down which data it can read, which tools it can call, whose authority it acts with, and which of those actions can't be undone.
2. Cover the conventional application security first. Authentication, object-level authorization, tenant isolation, secret handling, and dependency hygiene. An agent holding a broad database credential is an authorization problem rather than a model problem.
3. Constrain authority before adding detection. Narrow the tool set, scope the credentials per audience, and separate the user's identity from the agent's. See AI agents acting on behalf of users.
4. Add runtime controls at the boundaries, starting with the most consequential action. One tool call that moves money or data, not every route at once.
5. Deploy detection in dry run and measure it against your own traffic. Any control with a probabilistic component needs a false-positive number from your workload before it blocks anything.
6. Decide failure behavior per action. A search endpoint and a refund endpoint shouldn't behave the same way when a security dependency is slow.
7. Record decisions and what the application did next. A block you can't explain to a customer becomes a control someone disables. See compliance evidence for AI agents.

Governance frameworks such as the NIST AI Risk Management Framework and ISO/IEC 42001 help structure the program around this work. They describe what to govern rather than which control belongs on a tool call, so pair them with the boundary list rather than substituting one for the other.

Where Arcjet fits

Arcjet works on the runtime enforcement layer. It installs as a library and runs security rules inside your application, so a decision arrives before the action rather than in a report afterwards.

For AI applications, that means prompt-injection detection on inbound text and on content re-entering context, sensitive-information detection that classifies the body in your own process without transmitting it, content moderation, token and spend budgets keyed on an identity you control, and Guards that authorize a tool call from inside the tool handler. Because the enforcement point is a function call rather than a network hop, it covers agent tool calls, queue consumers, and background jobs, which no proxy can see.

Arcjet doesn't replace model safety or your own authorization code, and it isn't a substitute for a CDN absorbing volumetric attacks. It's designed to run alongside those, adding the application context that other layers structurally lack. For a wider survey of the tooling in this space, see AI security tools for developers.

Summary

AI security protects the system around the model. It splits into model safety, which the provider mostly owns and which can't reason about your users; application security, which decides who may reach a capability; and runtime enforcement, which decides whether a specific action should proceed. AI applications need all three because instructions and data arrive in one channel, behavior varies between runs, models take actions with real consequences, and consumption is unbounded unless you bound it. Start by writing down what each AI feature can reach, fix the conventional security underneath it, narrow the authority you grant, and then put controls on the boundaries where an action becomes irreversible.

Learn more: AI runtime protection · Arcjet Guards