Observed vs survey-based AI inventory: which stays accurate?

A register says why an AI system exists. Discovery says whether it's used. Build the list from observation and ask people for the judgment.

11 min read
In short: A survey-based AI inventory records what people report, and an observed inventory records what devices and agents do. Surveys capture purpose, ownership, and risk, but miss unreported tools and go stale. Observation finds actual use and states its coverage, but can't record judgment or reach unmanaged devices. Build the list from observation, and survey owners for purpose, ownership, and risk tier.

Which stays accurate, an observed or a survey-based AI inventory?

A survey-based AI inventory records what people report. An observed inventory records what devices and agents actually do. Surveys capture intent and context that observation can't. Observation captures adoption that surveys miss, and reports its own coverage. Use observation as the base and surveys for the fields that need human judgment: owner, purpose, and risk tier.

An AI inventory is the list of AI systems your organization builds and uses. NIST AI RMF 1.0 calls for one in GOVERN 1.6, and an EU AI Act deployer can't meet its duties for systems it hasn't listed. The frameworks don't say how to build it. Most organizations start with a questionnaire, and the following sections explain where that works, where it fails, and how observation fills the gap.

Arcjet publishes this comparison and sells an observed inventory, which is one of the two approaches on this page. For the governance model behind it, see what is AI governance. For the inventory procedure step by step, see AI inventory.

What is the difference between an AI register and AI discovery?

An AI register is a record of AI systems that people maintain, usually built from intake forms, questionnaires, and interviews. AI discovery is the automated finding of AI use from technical sources, such as device telemetry, network connections, identity grants, and agent reports. A register is a survey-based inventory, and discovery produces an observed one.

Both describe AI systems. An AI system is an application or agent together with the model providers and tools it uses. The register describes each system as its owner understands it: what it's for, who is responsible, and how risky it is. Discovery describes each system as it behaves: which providers it reaches, which tools it calls, and who uses it. The two records disagree more often than teams expect, and the disagreement is the useful part.

How do observed and survey-based inventories compare?

Observed and survey-based inventories fail in opposite places. A survey knows why a system exists but not whether it's still used, and observation knows what's used but not why. The following table compares them.

QuestionSurvey-based inventoryObserved inventory
Where entries come fromIntake forms, questionnaires, and interviews

Device observation, agent reports, coding-agent hooks, and connectors

What it captures wellPurpose, owner, business context, intended users, and risk

Which providers, tools, and MCP servers are used, by whom, and how often

What it misses

Tools nobody reported, changes since the last survey, and personal use

Purpose and ownership, and anything outside its sources' reach

How it stays currentSomeone reruns the survey or files a changeIt updates as activity arrives
How it reports its own gapsIt can't; non-response looks the same as no AI use

It can state coverage, such as the share of enrolled devices reporting

Coding agents and MCP serversOnly if engineers list them, and rarely the MCP serversFound from hook events and connections to model providers
Effort to buildA form and follow-up with every teamDeploying agents or hooks, then confirming what they find
What its evidence provesThat someone declared the systemThat the system was observed, when, and by which source

Why do survey-based AI inventories go stale?

Survey-based AI inventories go stale because AI adoption changes faster than the survey cycle, and nothing tells the register when it does. A register is accurate on the day the last answer arrives, and then it ages. The following changes don't reach it by themselves:

  • New tools. A developer installs a coding agent, or a team signs up for an AI service with a company card. Neither event files an intake form.
  • New tools inside existing systems. An agent gains an MCP server or a second model provider through a configuration change. The register still lists the system, but with the old scope.
  • Personal use. People rarely report a personal ChatGPT or Claude account used for work, because they don't think of it as a system the company runs.
  • Retirements. A system that's no longer used stays in the register until someone removes it.

A survey also can't tell you what it missed. A team that didn't answer looks the same as a team with no AI use. That's the strongest argument for observation: an observed inventory can report how much of the organization it covers, so a gap is visible rather than silent. For the shadow AI side of the same problem, see what is shadow AI.

What does an observed AI inventory miss?

An observed AI inventory misses anything its sources can't reach, and it can't record judgment. The following list names the gaps that matter for governance:

  • Purpose, owner, and risk tier. Observation shows that an agent called a database MCP server. It doesn't show whether that's what the agent is for, who answers for it, or how much harm it could do. People supply these fields.
  • Unmanaged devices. An endpoint agent reaches only the devices it's installed on. Contractors' laptops and personal phones are outside it.
  • Systems that don't report. An agent that doesn't use the platform's SDK or hooks, or a coding agent without hooks installed, sends nothing.
  • Account type. A connection to a provider's hostname doesn't show whether the account is personal or the company's.
  • Local models. A model running on the device makes no outbound connection to observe.
  • Content. Connection-level observation records that an app reached a provider, not what the person sent.

Each source has its own blind spots. For a comparison of endpoint, network, browser, identity, and device-management sources, see endpoint vs network shadow AI discovery.

Which sources can build an observed AI inventory?

An observed AI inventory can draw on several sources, and each finds a different part of your AI use. The following table lists the main ones.

SourceWhat it findsWhat it can't find
Endpoint network observation

Each AI provider reached from a device, with the app and the person

Unmanaged devices, account type, local models
Coding-agent hooksEach hooked coding agent, its tool calls, MCP servers, and modelsAgents and machines without the hook
Agent SDK reports

Each agent you build that reports to the platform, with its decisions

Agents that don't use the SDK
Identity and SaaS grantsAI tools signed in with a work accountPersonal sign-ups, API keys, and actual use
MDM and software inventoryInstalled AI apps and configured MCP serversBrowser use and whether a tool was used
Cloud and machine learning connectorsModels and AI services deployed in your cloud accountsAI use on laptops and in third-party SaaS

No single source covers everything. Choose the sources that reach where your AI use happens, and record which ones feed the inventory so its coverage is stated.

How do you combine observation and surveys in one inventory?

To combine the two, let observation decide which AI systems exist, and use people for the fields only people can fill. That reverses the usual order, where a survey creates the list and discovery checks it. The following steps describe the combined workflow:

  1. Build the list from observation. Deploy the sources that reach your AI use, and let them populate the inventory with providers, agents, and coding agents.
  2. State the coverage. Record which sources feed the inventory and how complete each is, such as the share of enrolled devices that report.
  3. Group observations into AI systems. Combine each agent with the providers and MCP servers it uses, and have a person confirm the grouping.
  4. Survey for judgment, one system at a time. Ask each system's likely owner for its purpose, intended users, data classes, and risk tier. The question is about a system that observation already found, so the answer attaches to a real record.
  5. Ask about what observation can't reach. Run a short survey for AI use outside the sources: unmanaged devices, vendors' embedded AI features, and local models.
  6. Review on a schedule and on change. Review each system at a fixed interval, and sooner when observation shows a new provider or tool. For the routine, see AI system review.

The register stays a register, with owners and purposes written by people. It stops being the only source of what exists.

Where does Arcjet's approach fit, and where doesn't it?

Arcjet builds an observed inventory. Discovery finds the AI providers people reach from managed macOS and Windows devices, with the app and the person, and reports the share of enrolled devices that are healthy. The SDK and coding-agent hooks for Claude Code, GitHub Copilot, Cursor, OpenAI Codex, and Muse Code report the agents you build and run. Arcjet suggests AI systems that group each agent with the providers and MCP servers it uses, and a person confirms each one.

The judgment fields sit on that observed record. Each AI system carries an owner, a purpose, a risk tier, an EU AI Act role, and a review date, with reviews due every 180 days. An owner must be a member of the team, and an owner who leaves reads as no owner automatically. Only team administrators edit these fields, and each change is written to the WorkOS audit log. Purpose drift compares the declared tools and data classes with what Arcjet observes and notifies the owner. The evidence page maps the inventory to NIST AI RMF 1.0, ISO/IEC 42001, and the EU AI Act.

Arcjet is the wrong choice in the following situations:

  • Your AI use is mostly on unmanaged devices. Arcjet's Discovery agent reaches managed devices only, so start with an identity inventory or a browser tool that installs on unmanaged devices.
  • You need to know which account was used. Arcjet observes connections, not account type. Use the vendors' tenant controls.
  • Your inventory has to cover AI deployed in cloud machine learning platforms. Use a platform with connectors to those accounts, and keep Arcjet for devices and agents.

For how this inventory fits a wider governance program, see AI governance platforms compared and what is AI governance.

Frequently asked questions

What is an AI inventory?

An AI inventory is the list of AI systems an organization builds and uses, where an AI system is an application or agent together with the model providers and tools it uses. Frameworks ask for one, including NIST AI RMF 1.0 in GOVERN 1.6, but leave the method to you.

Can automated discovery replace an AI register?

No. Discovery finds which providers, agents, and tools are used, but it can't record why a system exists, who answers for it, or how risky it is. Use discovery to build the list and keep it current, and keep a register's human-written fields, such as owner, purpose, and risk tier, on each discovered system.

How often should an AI inventory survey run?

If the survey is the only source, more often than most organizations manage, because AI tools arrive without intake forms. With observation as the base, survey each AI system's owner when the system is first found and at each review, and run a short survey for use that observation can't reach, such as unmanaged devices and local models.

Does an observed AI inventory find coding agents?

It does when its sources reach them. Coding-agent hooks report each hooked agent's tool calls, MCP servers, and models, and endpoint observation sees the agent's connections to model providers. Agents on machines without hooks or an endpoint agent stay invisible, so record which sources feed the inventory.

How do you measure how complete an AI inventory is?

State the coverage of each source that feeds it. For endpoint observation, that's the share of enrolled devices that are healthy and reporting; for hooks, the agents and machines where they're installed. A survey can't report coverage in the same way, because a team that didn't answer looks the same as a team with no AI use.

AI runtime security in your code

Protect your AI agent workflows with Arcjet

Arcjet runs inside your application, where it can use runtime context to enforce agent actions and budgets, detect prompt injection, and protect sensitive information before a workflow acts.