Start a project

Tool 07 of 194 · AI visibility

AI Bot Log Analyser

Drop in your server or CDN logs and see which AI crawlers really visit, which pages they read, and which hits only pretend to be them.

Use the tool All developer tools Live · free
AI Bot Log Analyser at a glanceIllustration: 1, Server logs; 2, processed in your browser; 3, Which AI bots came.AI Bot Log Analyser1Server logsUser-agent: *Allow: /Sitemap: …# llms.txt3Which AI bots came2In yourbrowser
1 Server logs 2 processed on your device 3 Which AI bots came

AI Bot Log Analyser

Your access logs

Crawlers

    CrawlerCompanyPurposeHitsVerified IPWrong IPName onlyTop pageLast seen

    Hits per day

    Status codes

    Crawler2xx3xx4xx5xx

    Most crawled pages

    PathHitsBy
    Summary (CSV)
    Hits claiming a crawler name from outside its IP ranges

    A crawler visit isn't a citation: it means a company read the page, not that an AI answer used or linked it.

    Want the AI crawler view of your whole site? Run PingAI, which scores how well ChatGPT, Claude, Perplexity, Gemini and Copilot can reach and read it.

    01

    What it does

    Reads access logs in the browser, finds 35 AI and search crawlers, checks their IPs against the ranges they publish and charts hits by day.

    • Apache, nginx, IIS, Cloudflare and JSON log formats
    • 35 crawlers: training, AI search, user fetchers and search engines
    • IPs checked against OpenAI, Google, Bing, Perplexity and Apple ranges
    • Hundreds of MB streamed in a background worker, .gz too

    02

    Private by design

    Everything happens in your browser. Nothing you type, paste or upload is sent anywhere, and it keeps working offline once the page has loaded.

    03

    What you get

    A crawler leaderboard and a CSV, with a Copy and a Download button on every result.

    • AI crawlers
    • Logs
    • GPTBot
    • Security

    Why it matters

    robots.txt says what you'd like AI crawlers to do; your access logs say what they actually did. Logs show whether GPTBot is reading your guides, whether OAI-SearchBot or PerplexityBot can reach the pages you want quoted, and whether that 'ClaudeBot' hammering your site is really Anthropic or a scraper borrowing the name.

    This analyser reads the logs in your browser, in a background worker, so even large files stay on your device. It recognises 35 crawlers, groups them by purpose, and checks each hit's IP against the ranges the company publishes, where it does. Remember that a visit isn't a citation: it shows a crawler read the page, not that an AI answer used it.

    How it works

    1. InputAn access log from your server or CDN
    2. In your browserUser agents matched, IPs checked against published ranges
    3. OutputHits per crawler, verified or spoofed, and top pages

    Where it fits in a designer's workflow

    1. Plan
    2. Design
    3. Build
    4. Test
    5. Launch
    Seeing whether AI crawlers actually visit.

    By the numbers

    Crawler hits in the sample logGPTBot (OpenAI): 2 hits, 1 verified; ClaudeBot (Anthropic): 2 hits, 0 verified; Googlebot (Google): 2 hits, 1 verified; OAI-SearchBot (OpenAI): 1 hit, 1 verified; ChatGPT-User (OpenAI): 1 hit, 1 verified; PerplexityBot (Perplexity): 1 hit, 1 verifiedGPTBot (OpenAI)2 hits, 1 verifiedClaudeBot (Anthropic)2 hits, 0 verifiedGooglebot (Google)2 hits, 1 verifiedOAI-SearchBot (OpenAI)1 hit, 1 verifiedChatGPT-User (OpenAI)1 hit, 1 verifiedPerplexityBot (Perplexity)1 hit, 1 verified
    Counted by this tool from its 10-line sample log. A hit is verified when its IP is inside the ranges the company publishes; red bars include a hit claiming the name from an IP outside them, which is usually a scraper in disguise. Source: OpenAI crawlers; Google: verify Googlebot.
    Show the numbers as a table
    CrawlerCompanyPurposeHitsVerified IPWrong IPName only
    GPTBotOpenAIModel training2110
    ClaudeBotAnthropicModel training2002
    GooglebotGoogleSearch engine2110
    OAI-SearchBotOpenAIAI search1100
    ChatGPT-UserOpenAIFetch for a person1100
    PerplexityBotPerplexityAI search1100

    How to use it

    1. Open one or more log files (.log, .gz or JSON lines), or paste some lines.
    2. Read the leaderboard: verified hits came from the crawler's own IP ranges.
    3. Check the 'Wrong IP' column and the suspicious IP list for impostors.
    4. Download the summary as CSV.

    FAQ

    Which log formats work?

    Apache and nginx Common and Combined logs, IIS W3C extended logs, Cloudflare Logpush JSON and most JSON-lines logs with ip, user agent, path and status fields.

    What does 'Name only' mean?

    The company doesn't publish IP ranges (Anthropic, Meta and Amazon, for example), so the hit is matched by its user agent alone and could be spoofed.

    How current are the IP ranges?

    They're a snapshot taken on the date shown on the page, from each company's published list. Ranges change; we refresh the snapshot when we update the tool.

    Are my logs uploaded?

    No. They're read in your browser and nothing is sent anywhere.

    Work with me

    Want a site that gets these right?

    I build fast, accessible websites and web apps that pass these checks from day one. Tell me about your project and I'll reply personally with honest next steps.

    Start a project What I do

    All developer tools · Back to leej.uk