Why it matters
Filters are great for shoppers and dangerous for crawling. Twelve categories, forty brands and a few multi-select colours and sizes multiply into millions or billions of URLs, and if every one is linked, search engines spend their time on near-duplicates instead of your products.
This planner counts exactly how many URLs your facets create, then splits them by plan: pages worth indexing (usually one or two facets people search for, such as brand or colour), pages crawlers can follow but shouldn't index, and parameters to keep out with robots.txt. It shows the effect of fixing the parameter order, writes the robots.txt rules and gives example URLs for each case, including pagination.
How to use it
- List your facets: name, URL parameter (or path), number of values, multi-select and plan.
- Set how many values a shopper can pick, and how many facets an indexed page may combine.
- Read the counts and the plan per facet.
- Copy the robots.txt rules and apply the link and meta robots advice.
FAQ
Should I block filters with robots.txt or use noindex?
Use noindex, follow for filter pages crawlers may visit but shouldn't index, and robots.txt for parameters that only create duplicates, like sort order or price sliders. A page blocked by robots.txt can't be seen, so a noindex on it is never read.
Why does parameter order matter?
?colour=red&size=9 and ?size=9&colour=red are different URLs to a crawler. If your site can produce both, every combination is multiplied by every possible order. Build links in one fixed order and canonicalise the rest.
How should I handle pagination?
Make each page self-canonical and indexable with plain links between them, don't canonicalise page 2 to page 1, and return 404 for pages past the end. Google no longer uses rel=next and rel=prev.
Which facets should be indexable?
Those with search demand and enough products, such as brand or colour on a category. Keep combinations shallow: one or two facets is usual.
