Why it matters
Anyone can send a request that says it's GPTBot or Googlebot: the user agent is just text. Scrapers use the big crawlers' names to get past bot rules, which skews your AI traffic numbers and can get the real crawler blocked when you react to the fake one.
The companies that run the main crawlers publish the IP ranges they crawl from. This verifier keeps a dated copy of those lists on leej.uk, reads your log lines, works out which crawler each visit claims to be and checks the IP against that company's ranges, IPv6 included. A match is verified; a miss is spoofed. Some companies (Anthropic among them) don't publish ranges, so those visits are marked as unverifiable here, with the reverse DNS suffix to check where one exists. The rules at the end block the spoofing IPs in nginx and give a Cloudflare expression that blocks crawler user agents that don't come from a verified bot.
How to use it
- Paste lines from your access log, or an IP address and user agent per line.
- Read the verdicts: Verified, Spoofed or Can't verify here.
- Copy the block rules if you want to stop the spoofers.
- Check the whole site's AI crawler access with PingAI.
FAQ
How do I verify a crawler that doesn't publish IP ranges?
Use reverse DNS: look up the IP's host name (host 1.2.3.4), check it ends in the company's documented domain, then look that name up again to confirm it points back to the same IP. Where no domain is documented, check the network owner with whois.
How current are the IP ranges?
The date of the snapshot is shown under the input. The ranges change every few weeks, so leej.uk refreshes the copy from the companies' published files when the tools are updated.
Will blocking spoofed GPTBot visits block the real GPTBot?
No, as long as you block the IPs or use a rule that only matches requests not from verified ranges. Don't block the user agent string on its own.
Is my log sent anywhere?
No. The checks run in your browser against the saved range lists.
