> ## Documentation Index
> Fetch the complete documentation index at: https://docs.searchable.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Connect WordPress to Searchable

> Install the Searchable must-use plugin on your WordPress site to forward AI-crawler requests to LLM Analytics — no theme changes, no JavaScript.

## What this is

A small **must-use plugin** that runs inside WordPress, inspects the user agent of every request that reaches PHP, and forwards the ones that match Searchable's canonical AI-crawler list to LLM Analytics.

It feeds the same server-side pipeline as the Vercel, Cloudflare, and Netlify connectors — same classifier, same dashboards. The difference is where it runs: in your application, not at a CDN edge.

> [!NOTE]
>
> Must-use plugins load automatically and can't be deactivated from the admin UI, so the forwarder
> survives plugin and theme updates.

## First, check this is the right path

> [!WARNING]
>
> **If your site sits behind Cloudflare, use the [Cloudflare Worker](https://docs.searchable.com/setup/cloudflare-worker.md)
> instead.** It runs at the edge, so it sees every AI-crawler request — including the ones a
> full-page cache serves without ever booting PHP. It works on every Cloudflare plan including Free,
> and it's less to maintain.

Use the WordPress plugin when your site **isn't** behind a CDN we support natively, or when you specifically want the forwarder in the application layer.

The same caveat applies to any full-page cache. See [Troubleshooting](#troubleshooting) below.

## Before you start

> [!TIP]
>
> Self-hosted WordPress (this does not work on WordPress.com's hosted plans, which don't allow
> `mu-plugins` or `wp-config.php` edits)

> [!TIP]
>
> SFTP, SSH, or a file manager that can write to `wp-content/` and `wp-config.php`

> [!TIP]
>
> Your project's domain confirmed in **LLM Analytics → Setup → Confirm your domain**

LLM Analytics is available on every plan — no upgrade required.

## Step 1: Open the WordPress connector

**1. Go to the Setup tab**

In Searchable, open **LLM Analytics** (left nav, under Analytics) → **Setup**, then pick **WordPress** under "Connect your crawler source".

**2. Generate a tracker API key**

Click **Generate key**. Searchable mints a key with the `log_events` permission, bound to this project in the key list, and shows it **once** — copy it now. Keys start with `sk_live_…`, so the list tells them apart by a short **Key** id (for example `Key 0a1b2c3d`); note it down next to the key. Ingest accepts any active key in the workspace, so a key bound to another project still works if a site there uses it.

> [!WARNING]
>
> The key is a live write credential. Treat it like a password. If you lose it, revoke it (the change takes effect within a minute) and regenerate from the WordPress connector panel in **Setup**, on the project the key is bound to.

**3. Copy the wp-config snippet**

The panel shows a `define()` block with your project's site token already filled in. Copy it verbatim — you'll paste it in Step 3.

## Step 2: Install the plugin

**1. Download the plugin file**

Click **Download plugin** in the WordPress panel. It downloads `searchable-ai-traffic.php`, served from your project's tracker host.

> [!TIP]
>
> Download from the button rather than reusing a copy from a colleague. The artifact is refreshed in place on every release, so the button always gives you the current plugin — and the download is pinned to the tracker host your project actually posts to.

**2. Upload it to wp-content/mu-plugins/**

Create `wp-content/mu-plugins/` if it doesn't already exist, then copy `searchable-ai-traffic.php` **directly** into it.

> [!WARNING]
>
> WordPress does not load must-use plugins from subdirectories. The file has to sit at `wp-content/mu-plugins/searchable-ai-traffic.php`, not `wp-content/mu-plugins/searchable/searchable-ai-traffic.php`.

There is nothing to activate. Must-use plugins load on every request automatically, and appear under the **Must-Use** filter on the admin Plugins screen (not in the main list, and with no activate/deactivate controls).

## Step 3: Add your credentials

Open `wp-config.php` and paste the snippet from Step 1 **above** the line that reads `/* That's all, stop editing! Happy publishing. */`.

```php wp-config.php theme={null}
define('SEARCHABLE_API_KEY', 'sk_live_YOUR_API_KEY');
define('SEARCHABLE_SITE_TOKEN', 'st_YOUR_SITE_TOKEN');
```

Replace `sk_live_YOUR_API_KEY` with the key you generated. The site token is already filled in for you in the in-app snippet.

> [!WARNING]
>
> The plugin deliberately stays inert — it sends nothing at all — while either constant still
> contains `YOUR_`. If nothing arrives in your dashboard, this is the first thing to check.

> [!NOTE]
>
> **Projects on a dedicated tracker hostname** see two extra `define()` lines in the in-app snippet
> (`SEARCHABLE_API_URL` and `SEARCHABLE_BOTS_URL`). Don't drop them — without them the plugin falls
> back to the shared host, and you'd end up downloading the plugin from one endpoint while posting
> events to another. Most projects use the shared host and won't see these lines.

### Why `wp-config.php` and not the plugin file

`wp-config.php` sits outside the plugin directory, is excluded from most deployment pipelines by default, and won't leak the key if `mu-plugins/` is ever served as plain text.

### Optional constants

| Constant | Default | What it does |
| - | - | - |
| `SEARCHABLE_SEND_IP` | `true` | Set to `false` to omit the (already anonymized) IP entirely. Geo enrichment then falls back to edge metadata. |
| `SEARCHABLE_API_URL` | `https://tracker.searchableanalytics.com/v1/events` | Ingest endpoint. Set automatically by the in-app snippet for projects on a dedicated tracker hostname. |
| `SEARCHABLE_BOTS_URL` | `https://tracker.searchableanalytics.com/v1/bots.json` | Canonical bot list. Same rule as above. |

## Step 4: Verify

Send a request to your site with a known crawler user agent:

```bash theme={null}
curl -A "GPTBot/1.0" https://yourdomain.com/
```

**Expected result:** the request appears in LLM Analytics within roughly 30 seconds.

1. Check the **Logs** tab first — it shows individual bot requests as they land.
2. On the **Setup** tab, the WordPress card flips from **Waiting for the first event from your WordPress site…** to **Connected — the plugin is sending events.**

> [!TIP]
>
> Real crawler traffic follows on its own schedule. AI bots typically hit any indexed site within a
> few hours, so don't read an empty dashboard an hour after install as a failure — the `curl` above
> is the definitive test that the wiring works.

## How it works

* **Bot list.** The plugin fetches Searchable's canonical bot list from `/v1/bots.json`, validates each pattern, and caches the compiled list in a transient for one hour. If the fetch fails it falls back to the last good list persisted in `wp_options`, so a network blip doesn't silently disarm the filter. Failures are cached for five minutes, so an outage never adds a fetch attempt to every request.
* **Matching.** On the `shutdown` hook the plugin matches the request's user agent against that list. Non-matching requests do nothing — no outbound call, no work beyond the regex pass. WP-CLI, cron, and CLI requests are skipped entirely.
* **Latency.** On PHP-FPM the response is flushed to the crawler via `fastcgi_finish_request()` *before* the outbound POST, so forwarding adds no latency. On other SAPIs the POST is blocking with a 2-second timeout, and only ever fires for matched crawler requests.
* **IP handling.** The client IP is resolved proxy-aware (`CF-Connecting-IP` → left-most `X-Forwarded-For` → `X-Real-IP` → `REMOTE_ADDR`, each validated) and **anonymized on your server before sending** — the last IPv4 octet or IPv6 group is zeroed, so the full IP never leaves the site.

## What gets sent

Only requests from user agents matching Searchable's canonical AI-crawler list are forwarded. Human traffic is never sent.

Each matched request sends:

* HTTP method, path, and full URL (the stored path has no query string; the full URL is stored as sent, including any query string)
* Response status code and server response time
* User agent and referrer
* An **anonymized** client IP — last octet zeroed, or omitted entirely with `SEARCHABLE_SEND_IP`
* A `source` tag of `wordpress`, so these events are distinguishable from edge-connector or REST API events

Cookies, request and response bodies, other headers, and full IP addresses are never sent.

## Optional: enhance the data

The plugin is a **crawler-log source**. It powers the Overview, Pages, AI Crawlers and Logs tabs.

To populate the **Visitors** tab — humans arriving on your site from AI answers — connect [Google Analytics 4](https://docs.searchable.com/integrations/google-analytics-4.md), [Search Console](https://docs.searchable.com/integrations/google-search-console.md), or Bing in step 3 of the Setup wizard. These are additive; they don't replace the crawler-log source.

## Troubleshooting

> [accordion] Nothing appears in the dashboard
>
> Work through these in order:
>
> 1. **Placeholders still in place.** The plugin no-ops while `SEARCHABLE_API_KEY` or `SEARCHABLE_SITE_TOKEN` contains `YOUR_`. Confirm both were replaced with real values.
> 2. **Wrong directory.** The file must be directly in `wp-content/mu-plugins/` — not `plugins/`, and not a subdirectory. Confirm it appears under the **Must-Use** filter on the admin Plugins screen. If that filter isn't shown at all, WordPress isn't seeing any mu-plugin — check the path over SFTP.
> 3. **Constant names.** They're case-sensitive and must be spelled exactly as in the snippet.
> 4. **Outbound HTTP blocked.** Some managed hosts restrict `wp_remote_post` to an allowlist. Ask your host to allow `tracker.searchableanalytics.com` (or your dedicated tracker hostname).

> [accordion] A page cache sits in front of WordPress
>
> Full-page caches — WP Rocket, WP Super Cache, W3 Total Cache, Varnish, Cloudflare APO, or host-level caches on WP Engine and Kinsta — serve cached HTML without booting PHP. When that happens the `shutdown` hook never fires and the crawler visit is never recorded.
>
> Two options:
>
> * **Exclude known crawler user agents from the cache**, so their requests always reach PHP.
> * **Move to an edge-level integration** — the [Cloudflare Worker](https://docs.searchable.com/setup/cloudflare-worker.md) is the usual answer, since it runs in front of the cache.
>
> If you do neither, you'll still capture the origin-level hits that pass through on cache misses and TTL expiry — an undercount, not a zero.

> [accordion] The crawler views are empty but Visitors has data
>
> You've connected Google Analytics or Search Console but no crawler-log source. The Overview,
> Pages, AI Crawlers and Logs tabs need this plugin (or a Cloudflare, Vercel, or other edge source)
> working.

> [accordion] Only some crawlers are captured
>
> Searchable's canonical bot list is regenerated **once daily** at 03:00 UTC, and the plugin caches its own copy for up to another hour. A newly added crawler can therefore take up to \~25 hours to start being captured.
>
> Deleting the `ai_traffic_bot_patterns` transient forces a refetch, but that only helps once the daily upstream publish has picked the new bot up. To see what the plugin is working from, fetch the list yourself:
>
> ```bash theme={null}
> curl https://tracker.searchableanalytics.com/v1/bots.json
> ```

> [accordion] Requests are slower after installing
>
> On PHP-FPM they shouldn't be — the response is flushed before the POST. If your host runs mod\_php or another non-FPM SAPI, matched crawler requests carry a blocking POST bounded at 2 seconds. Human traffic is unaffected either way, since non-matching requests never make an outbound call.

## Next steps

> [card] [See the data](https://docs.searchable.com/using-searchable/visibility-tracking.md)
>
> Open LLM Analytics to see which assistants are crawling your site.

> [card] [Connect Search Console](https://docs.searchable.com/integrations/google-search-console.md)
>
> Layer in keyword data so you can correlate AI crawls with search demand.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.