PrompeteerAEOBot

PrompeteerAEOBot fetches a page when someone asks us to measure it. This page states what it sends, how many requests it makes, when it honours robots.txt and when it does not, and how to opt a domain out.

We sell a product that grades other people on how they treat crawlers. This page is the same standard applied to us: what we send, how much of it, when we ask permission, and how you stop it.

The identifier

PrompeteerAEOBot/1.0 (+https://prompeteer.ai/bot)

The product token is PrompeteerAEOBot. It is a substring of every request we make, which is what RFC 9309 §2.2.1 asks of a crawler, and it is what you write in robots.txt to address us.

What a crawl costs you

At most four requests in flight at a time, and at most one audit of a domain per day. The range covers a second page we read from your sitemap; a site with no sitemap costs the lower number.

Why some of our requests carry someone else's name

When you ask us to measure a page, we fetch it several times, each time presenting a different AI crawler's user-agent token — Googlebot's, GPTBot's, PerplexityBot's and so on — because comparing what each of them receives is the measurement. We do not pretend those requests are theirs: every one of them also carries PrompeteerAEOBot/1.0 (+https://prompeteer.ai/bot), and every one of them comes from a Prompeteer address, not the vendor's. OpenAI, Anthropic, Perplexity and Google all publish IP lists so you can tell a real crawler from a stranger sending its name. If you verify crawlers properly, you refused us — correctly — and our report shows that as a block. A block there is not evidence that the real crawler is blocked.

The 11 requests that only ask whether a file exists — robots.txt, the discovery paths, your sitemap, a markdown twin — send our own identity and nothing else. They read a status code, so there was never a reason for them to claim to be anybody.

Since 15 September 2026 Cloudflare blocks Training and Agent crawlers by default on new and free-plan domains, so a block our report finds may be a default nobody at your organisation chose.

The strings we present

Vendor documentation read 2026-09-06. "Not measured" means we have not checked that vendor's documentation, which is not the same as the vendor publishing nothing.

• anonymous — The browser baseline. Every parity comparison is made against the document a person receives, so this request has to look like a browser.
• ClaudeBot — Anthropic publishes an IP list but no user-agent string, so this one is our construction and cannot be checked against the vendor.
• Claude-SearchBot — Anthropic publishes an IP list but no user-agent string, so this one is our construction and cannot be checked against the vendor.

robots.txt

The scheduled re-measure reads your robots.txt first and skips your domain entirely when it disallows PrompeteerAEOBot. A user-initiated audit does not consult robots.txt: a person asked for that one page, which is the same convention under which OpenAI's ChatGPT-User and Perplexity's Perplexity-User are exempt. We would rather say that plainly here than leave it implied.

Opting out

User-agent: PrompeteerAEOBot
Disallow: /

That stops the scheduled re-measure within a week. To stop user-initiated audits as well — including one a stranger runs on your site — write to support@prompeteer.com from an address at the domain and we will refuse it at our end.

Verifying us