What this does
Searchable’s LLM Analytics dashboard shows you, in near real-time, which AI assistants — ChatGPT, Claude, Perplexity, Google’s AI Overviews, and the rest — are crawling your site, what pages they’re hitting, and which of those crawls turn into referred users. To do that, Searchable needs to see your inbound HTTP requests. The setup below points your hosting platform’s request logs at our ingest endpoint. For log and edge integrations, we classify the AI bots and drop everything else. The section on what gets sent, further down this page, summarises what is stored.Setup takes about 5 minutes. There are no agents to install and no code to change in your
application.
Pick your platform
Vercel
One-click Log Drain. Works on Pro and Enterprise.
Cloudflare Worker
Edge Worker — works on any Cloudflare plan, including Free.
Cloudflare Logpush
Native Logpush job — Enterprise plan only.
Amazon CloudFront
Standard Logging V2 via Firehose — any CloudFront distribution.
Netlify Edge Function
Edge Function — works on every Netlify plan, including Free.
Netlify Log Drain
HTTP Log Drain — Enterprise plan only.
Fastly
Real-Time Log Streaming via HTTPS endpoint.
Akamai
DataStream 2 HTTPS push — any property with DataStream 2 entitled.
Google Cloud Platform
HTTPS Load Balancer / Cloud CDN logs via Pub/Sub push.
WordPress
Must-use plugin — captures AI crawlers at the origin on any self-hosted site.
Other platforms
Custom REST API or middleware for everything else.
Which one should I use?
I host on Vercel
I host on Vercel
Use Vercel Log Drain. It’s the most direct path — Vercel streams request logs to Searchable with one entry in your project settings. No code changes.
I'm on Cloudflare (any plan)
I'm on Cloudflare (any plan)
Use the Cloudflare Worker. It works on every Cloudflare plan
including Free, and the Free tier easily covers ~100k requests/day. The Worker forwards AI-bot
traffic to Searchable from the edge — no latency added to real users.
I'm on Cloudflare Enterprise
I'm on Cloudflare Enterprise
You can use either path. Cloudflare Logpush is the most “native” option —
Cloudflare batches and ships logs directly with no Worker involved. If you’d rather avoid the
Enterprise-only Logpush feature, the Worker path works on
Enterprise too.
I'm on Amazon CloudFront
I'm on Amazon CloudFront
Use Amazon CloudFront → Firehose. CloudFront’s free Standard Logging V2
feature ships records to Amazon Data Firehose, which forwards them to Searchable. Works on any
CloudFront distribution — you only pay for the Firehose ingestion.
I host on Netlify
I host on Netlify
Use the Netlify Edge Function on any plan (including Free), or
Netlify Log Drain if you’re on the Enterprise plan. The Edge Function
path runs at Netlify’s edge with no added latency for real users.
I host on Fastly
I host on Fastly
Use Fastly Real-Time Log Streaming. Configure an HTTPS Logging endpoint
inside the Fastly UI — no code changes — and Fastly streams request logs to Searchable.
I'm on Akamai
I'm on Akamai
Use Akamai DataStream 2. Create a DataStream 2 stream in Akamai
Control Center, pick the property fronting your domain, and point it at Searchable’s HTTPS
endpoint. Requires the DataStream 2 product on your Akamai contract — your account team can
confirm if it’s enabled.
I host on Google Cloud Platform
I host on Google Cloud Platform
Use Google Cloud → Pub/Sub push. Cloud Logging captures every HTTPS Load
Balancer or Cloud CDN request; a Pub/Sub topic with a push subscription forwards those logs to
Searchable.
I run WordPress
I run WordPress
Use the WordPress plugin — a must-use plugin that forwards AI-crawler
requests from the origin, with no theme changes and no JavaScript. One caveat: if your site is
behind Cloudflare, or runs a full-page cache that serves HTML without booting PHP, prefer the
Cloudflare Worker — it sits in front of the cache and sees every
request.
I'm somewhere else (other platforms, custom)
I'm somewhere else (other platforms, custom)
Use the custom integration. Searchable accepts a small REST POST from your application middleware or any platform that can ship structured request logs.
What gets sent to Searchable
For each AI-bot request, Searchable records:- HTTP method, path, and host
- User agent
- Referer
- Country code (geo-IP, no precise location)
- Client IP address (see below)
- Response status and bytes
- Timestamp
User-Agent / Referer.
Query strings. The Cloudflare Worker drops the query string from the request URL before sending. Every other integration sends the URL as your platform or your code provides it, which can include the query string. The path Searchable stores never includes it, but the full URL is stored as sent, and some integrations (Akamai DataStream 2, Google Cloud, the REST API and application middleware) also store the query as parsed parameters.
Client IP addresses. The Cloudflare Worker anonymises the client IP address before sending it by default (the last IPv4 octet is zeroed; an IPv6 address keeps its first 48 bits). Log integrations (Cloudflare Logpush, CloudFront, Akamai DataStream 2, Vercel and the other log drains) and the Netlify Edge Function send the client IP address as your platform reports it, and it is stored with the event. The REST API stores the IP address you send. The WordPress plugin zeroes the last IPv4 octet (or IPv6 group) on your server before sending, so the full IP never leaves your site, and can be set to omit the IP entirely.
Requests that aren’t AI bots. Log drains deliver every request your platform logs, and by default the Cloudflare Worker forwards every request after a cold start until its bot registry has loaded, and requests with no user agent after that (its guide shows how to turn both off). Searchable discards requests that don’t match its bot registry on arrival and does not store their path or IP address, with one exception: requests to UCP (Universal Commerce Protocol) endpoints (/.well-known/ucp, /ucp, /api/ucp/* and any others Searchable recognises) are kept whatever their user agent. Custom integrations are different: Searchable stores every event you send through the REST API or your application middleware, human requests included, so filter before sending if you only want AI-bot requests stored. Each platform guide says exactly what leaves your infrastructure.
Verifying the connection
After completing setup, return to LLM Analytics → Setup in your Searchable dashboard. The status strip at the bottom updates as events arrive:- Waiting for first event — your platform hasn’t sent anything yet. AI bots typically hit any indexed site within a few hours.
- Connected — events are flowing. The strip shows the count from the last 24 hours.
Next steps
Add a domain
LLM Analytics is scoped to your project’s primary domain.
Connect Search Console
Layer in keyword data so you can correlate AI crawls with search demand.