Start a project

Tool 25 of 244 · AI visibility

AI Crawler Firewall Rule Builder

Writes WAF, nginx, Apache and Workers rules to allow, block or rate-limit AI crawlers by user agent and verified IP range.

AI Crawler Firewall Rule Builder

Which crawlers, and what to do

Rules are built from the same dated AI crawler list used by the IP verifier and the directory. Per bot you can allow, block, rate-limit, or allow only from published IP ranges.

Rules

Generated rules

    Want the AI crawler view of your whole site? Run PingAI, which scores how well ChatGPT, Claude, Perplexity, Gemini and Copilot can reach and read it.

    01

    What it does

    Writes WAF, nginx, Apache and Workers rules to allow, block or rate-limit AI crawlers by user agent and verified IP range.

    • Reads the shared scripts/ai_bots.py list and the dated IP-range snapshot
    • Per bot: allow, block, rate-limit, or allow only from verified ranges (spoof protection)
    • Emitters for Cloudflare expression syntax, nginx map + geo, Apache mod_rewrite/RequireAll, Workers JS and Caddy
    • Lints conflicts (blocking a bot robots.txt still allows wastes its crawl)

    02

    Private by design

    Everything happens in your browser. Nothing you type, paste or upload is sent anywhere, and it keeps working offline once the page has loaded.

    03

    What you get

    Cloudflare, nginx, Apache, Workers and Caddy rules, with a Copy and a Download button on every result.

    • AI crawlers
    • WAF
    • nginx
    • Firewall

    Why it matters

    Writes WAF, nginx, Apache and Workers rules to allow, block or rate-limit AI crawlers by user agent and verified IP range.

    Reads the shared scripts/ai_bots.py list and the dated IP-range snapshot. Per bot: allow, block, rate-limit, or allow only from verified ranges (spoof protection). Everything runs in your browser; nothing you paste is sent unless you press Fetch.

    How to use it

    1. Paste or type the input the tool asks for.
    2. Read the summary and the table or preview.
    3. Adjust the options if you need a different result.
    4. Copy or download the output. Nothing leaves this page.

    FAQ

    Is my input uploaded?

    No. The work runs in your browser and nothing you type, paste or upload is sent anywhere.

    Will this change my site for me?

    No. It shows results and, where it writes rules or files, you copy them and apply them yourself.

    How does this relate to PingAI?

    PingAI tests whether firewalls block AI crawlers; this writes the rules. Link: 'Check the result with PingAI.'

    Work with me

    Want a site that gets these right?

    I build fast, accessible websites and web apps that pass these checks from day one. Tell me about your project and I'll reply personally with honest next steps.

    All developer tools · Back to leej.uk