Start a project

Tool 29 of 244 · Indexing

X-Robots-Tag Header Rule Builder

Writes server rules that send X-Robots-Tag for PDFs, feeds, search pages and staging hosts, with a tester for paths.

X-Robots-Tag Header Rule Builder

Rules

    Server rules

    Generated rules

      01

      What it does

      Writes server rules that send X-Robots-Tag for PDFs, feeds, search pages and staging hosts, with a tester for paths.

      • Rule model: match (extension, path glob, host) to directives (noindex, nofollow, none, noarchive, nosnippet, max-snippet, unavailable_aft…
      • Emitters per platform with their glob/regex dialects
      • Path tester runs the same matcher
      • Warns when robots.txt also blocks those paths (the header would never be seen)

      02

      Private by design

      Everything happens in your browser. Nothing you type, paste or upload is sent anywhere, and it keeps working offline once the page has loaded.

      03

      What you get

      Server rules and a path test, with a Copy and a Download button on every result.

      • X-Robots-Tag
      • noindex
      • PDF

      Why it matters

      Writes server rules that send X-Robots-Tag for PDFs, feeds, search pages and staging hosts, with a tester for paths.

      Rule model: match (extension, path glob, host) to directives (noindex, nofollow, none, noarchive, nosnippet, max-snippet, unavailable_after, per-bot prefix). Emitters per platform with their glob/regex dialects. Everything runs in your browser; nothing you paste is sent unless you press Fetch.

      How to use it

      1. Paste or type the input the tool asks for.
      2. Read the summary and the table or preview.
      3. Adjust the options if you need a different result.
      4. Copy or download the output. Nothing leaves this page.

      FAQ

      Is my input uploaded?

      No. The work runs in your browser and nothing you type, paste or upload is sent anywhere.

      Will this change my site for me?

      No. It shows results and, where it writes rules or files, you copy them and apply them yourself.

      Can I rely on this as official guidance?

      It follows public documentation and well-known algorithms, and tells you where it is a subset or a simulation. Confirm critical changes in the official tool or a staging environment.

      Work with me

      Want a site that gets these right?

      I build fast, accessible websites and web apps that pass these checks from day one. Tell me about your project and I'll reply personally with honest next steps.

      All developer tools · Back to leej.uk