User agent
LeejTools/1.0 (+https://leej.uk/tools/bot)
Requests come from Cloudflare's network (a Cloudflare Worker), so there's no fixed IP range to allow-list. Match on the user agent instead.
When it fetches
Only when a visitor presses a Fetch, Trace or Check button on one of these tools, after a human check (Cloudflare Turnstile):
- Indexability Checker
- llms.txt Validator
- llms-full.txt Builder
- XML Sitemap Validator
- IndexNow Submission Builder
- Indexing Submission Tracker
- Redirect Chain Tracer
- Canonical Conflict Finder
- Hreflang Cluster Validator
- Schema.org Entity Graph
- Rendered vs Raw HTML Diff
- Snippet & AI Preview Control Tester
- robots.txt & llms.txt Change Watch
- AI Crawler robots.txt Tester
There's no scheduled crawling, no link following beyond what the tool needs (for example the alternates of one hreflang cluster, or the child sitemaps of one index), and no AI model reads what it fetches.
What it reads and keeps
- GET requests only, over https on port 443. Plain http, other ports, IP addresses and private or internal hosts are refused, and every redirect is checked again.
- Pages up to 512 KB, robots.txt up to 500 KB, sitemaps up to 2 MB (8 MB unzipped). Each request gives up after 6 seconds.
- At most 10 fetches a minute for any one site across all visitors, and daily limits per visitor and overall.
- The result goes straight back to the visitor's browser. leej.uk keeps only daily counts (which tool, which kind of file, what class of status code), plus short-lived hashed rate-limit keys deleted each day. It doesn't store the addresses or what came back.
How to opt out
Before fetching a page, LeejTools reads your robots.txt and follows any group addressed to it. To stop it fetching pages on your site, add:
User-agent: LeejTools Disallow: /
Like other fetchers that act for a person rather than crawl, it doesn't apply rules written for every crawler (User-agent: *). Files that exist to be read by tools, such as robots.txt itself, sitemaps, llms.txt and an IndexNow key file, are read when a visitor asks for them. Questions or problems: hello@leej.uk.
Pasting instead
Every tool that can fetch also accepts pasted HTML, headers or files. Pasted content is processed in the visitor's browser and never reaches leej.uk.