Directory

AI agents

Agents act on a person's behalf. They browse, fill forms, and increasingly buy. That makes them a different problem from crawlers, and most of them cannot be identified the way a crawler can.

9

agents tracked, across 8 operators

4

can be cryptographically verified today

5

publish no user agent at all

Signing directories were checked by direct fetch on 26 July 2026, validating the JWKS content type rather than a 200 status, because several hosts return 200 with an ordinary HTML page at that path.

Why most of this table is empty

The crawler model was a declared token plus a published IP list. That model has broken down for agents. Claude for Chrome, Copilot Actions and Perplexity's Comet all drive the user's own browser, with the user's address, cookies and session. There is nothing for your server to match on, because there is nothing to match. Any directory that prints a user agent string for those products invented it.

The replacement is request signing. An agent signs its requests and publishes its public keys at a well known URL, so your origin can verify the traffic instead of trusting a string anyone can copy. That is what the verified column below measures.

User-directed agent

Visits and acts on a page on behalf of a person who is waiting for the result.

Google-Agent

Google

Verifiable
User agent
Google-Agent
Respects robots.txt
No, documented to ignore it

Used by agents hosted on Google infrastructure to navigate the web and perform actions on a user's request.

Details and how to verify

ChatGPT-User

OpenAI

Verifiable
User agent
ChatGPT-User
Respects robots.txt
Operator says it may not apply

Fetches a page in response to a specific user request inside ChatGPT.

Details and how to verify

Claude for Chrome

Anthropic

Not verifiable
User agent
None published
Respects robots.txt
Not documented

Runs inside the user's own Chrome profile and session, acting on the page in front of them.

Details and how to verify

Copilot Actions

Microsoft

Not verifiable
User agent
None published
Respects robots.txt
Not documented

Drives the user's own Edge browser with the user's cookies, selecting, scrolling and typing the way the person would.

Details and how to verify

Assistant fetch

Retrieves a page in real time to answer a question a user just asked.

Claude-User

Anthropic

IP range only
User agent
Claude-User
Respects robots.txt
Yes, uniformly

Retrieves a page because a user asked Claude about it. Distinct from ClaudeBot, which crawls for training.

Details and how to verify

Agent infrastructure

Hosted browser platforms that other people's agents run on.

Browserbase

Browserbase

Verifiable
User agent
None published
Respects robots.txt
Not applicable

Hosted browser infrastructure that other people's agents run on. Many different end users drive the same platform.

Details and how to verify

Transacting agent

Completes checkout or another transaction on behalf of a user.

Looking for crawlers rather than agents? The AI crawlers directory covers the bots that fetch pages for training and indexing.