Technical SEO checklist for a new website

I run a pass like this when I ship a site, including leej.uk, and I use the same “ready or it does not ship” bar that I use in the iOS app MVP checklist and the App Store launch process. The list below is for a new public site, including the AI crawlers that will fetch it whether you planned for them or not.

Staging first: do not leak the preview

The most expensive technical SEO mistake you can make before launch is to let the preview be indexed as if it were the site.

Treat every staging, preview or review host as a secret until you are ready:

• Prefer access control (a password, a VPN, or a network allow-list) over robots rules. robots.txt is not a lock.
• Add Disallow: / on the staging host, and a site-wide noindex as defence in depth, never as the only control.
• Do not list the preview in a production sitemap, in Search Console as a property you forget about, or in a public llms.txt.
• Use a separate host or a basic-auth gate, not example.com/staging, if people will still share links.
• Before you point the real domain at the new build, search for the staging hostname and for distinctive strings from the preview. If anything is already indexed, use noindex, removals, and a clear robots file, and do not launch on the same URLs that leaked.

Production should not inherit the staging Disallow: /. That sounds obvious, and it is still a classic go-live failure: the new site is perfect, and Googlebot is politely staying away. Put a launch-blocker in the pipeline that fails if the production robots.txt still disallows the whole site, or if every page still carries noindex.

HTTPS, host, slash and a single URL shape

Pick one public shape for every URL, and make every other shape a permanent redirect to it:

• HTTPS only. HTTP should redirect in one hop, not via a mixed chain of www and slashes.
• One host, either www or the bare domain, not both as equals.
• One trailing-slash policy, applied consistently to directories and to files.
• Lower-case paths if you can, or at least redirects from the accidental mixed-case bookmarks.
• No leftover .html and clean-URL twins, no index.html next to /, no tracking-parameter variants that you intend to index.

Set the canonical tag on each page to that chosen URL, and make the sitemap, internal links and any hreflang annotations use it too. The Indexability Checker will catch a surprising number of these, as will a simple crawl that lists final status codes.

If you are replacing an old site on the same domain, this is also when redirects from old paths must exist. A new information architecture with no map from the old URLs is a migration, and it needs the website migration SEO checklist, not only a launch party.

robots.txt, sitemap and indexability

On the production host, the day you go live:

• robots.txt should allow the pages and the assets you want crawled, including CSS and JavaScript a renderer might need. It should disallow only the paths you have chosen on purpose, such as internal search or previews that remain on the same host.
• The file should list the sitemap with an absolute HTTPS URL.
• The sitemap should list only live, canonical, indexable URLs, with lastmod values you can defend, as in the XML sitemap guide.
• Every important URL should return 200, with no noindex, and with a canonical that points at itself.
• Soft 404s — 200 pages that say “not found” — should not exist in the new build.

Spot-check a sample with the Indexability Checker and the HTML Page Auditor: home, a section index, an article, a legal page, and a URL you expect to stay out of the index. Then fetch robots.txt and the sitemap as a crawler would, not only as a logged-in preview.

The robots.txt and llms.txt Generator is a way to draft a consistent pair of files. Read them before they ship; a generator will not know about a forgotten /beta path.

Titles, descriptions and one H1

Each indexable page needs:

• A unique title that states the job of the page, short enough to use in a result, and consistent with the H1.
• One H1. Not zero, and not three competing ones in the hero, the article and a hidden template.
• A meta description that can stand alone, without a trailing teaser, and that does not copy the same sentence onto every URL.
• A visible first paragraph that actually answers the page’s promise.

This is writing work as much as technical work, but a launch is when template bugs show up: a site-wide suffix duplicated, a default “Home” title on inner pages, or an H1 that is only an empty logo. Audit the templates, then sample the pages. Do not rely on a single hand-built home page while the article template still says “Untitled”.

Schema, llms.txt and AI crawlers

Decide, before launch, what you want machines to trust:

• JSON-LD that matches the visible page: site, person or organisation, breadcrumbs, articles, and FAQ only where the questions are on the page. See the JSON-LD guide. Invalid or invented markup is not a “later” fix if it ships a fake rating on day one.
• llms.txt, and optionally llms-full.txt, if you want a short, accurate map for AI tools. Write it as a map, not a brochure, as in how to write llms.txt, and do not put it behind a challenge page.
• AI crawler policy. Allow or disallow named user-agents in robots.txt on purpose, rather than inheriting a default from a security plugin. The guide to controlling AI crawlers is the place to think that through. Tokens change, so review the current names; do not paste an old list and assume it still matches.
• Firewalls and bot challenges. Confirm that Googlebot, Bingbot and any fetcher you intend to allow receive the same article body a person receives, not a CAPTCHA or an empty shell.

You do not need a perfect entity graph on every URL at launch. You do need to not ship a contradiction, and you do need the first HTML response to contain the text you want quoted.

Performance, images and the first paint

A new site does not need a perfect Lighthouse score. It does need to be usable, and it should not ship obvious front-end waste that you will then have to unwind.

• Confirm HTTPS, compression, and a reasonable cache policy for static assets.
• Make sure the largest heading or hero image is in the first response, with dimensions reserved, so the layout does not jump.
• Resize and compress images; do not ship a 4,000-pixel photograph into a 320-pixel card.
• Keep third-party scripts few, and load the ones you must have in a way that does not block the first paint.
• Check the Security Headers on the live host: HTTPS is not the only header that should be present on day one, but a broken Content-Security-Policy that you copied from a blog can be a launch-blocker if it strips your own scripts or images.

The Core Web Vitals Calculator is useful as a lab sense-check. Field data will not exist until real people visit, so do not treat a missing CrUX history as a failure, and do not treat a green lab score as a promise. There is a fuller method in diagnosing Core Web Vitals if something already looks slow.

Analytics, Search Console and forms

These are easy to leave until “after we launch”, and they are part of being able to see what happens next.

• Create the Google Search Console property for the exact host you chose, verify it, and submit the sitemap on go-live day. Do the same in Bing Webmaster Tools if Bing matters to you.
• If you use IndexNow, host the key file and ping the new URLs, remembering that this does not notify Google; the IndexNow guide covers the limits.
• Put analytics or a privacy-respecting measurement tool on the production domain only, with a cookie or privacy policy that matches what you actually collect. Staging data should not pollute the production property.
• Test every form: contact, sign-up, checkout if you have it, and the failure states. A launch that cannot receive a message is a business failure, not only an SEO one. Confirm the success page is noindex if it should be.
• Check that legal pages (/privacy, terms, cookies if you need them) are linked, indexable if you want them found, and dated honestly. This is not legal advice; it is a reminder not to ship a site with a lorem policy, or with none.

Cost and scope still matter at this point, just as they do when you think about what an iOS app really costs. A smaller site that is consistent will be easier to keep searchable than a large one with half of this list deferred.

The go-live pass and the week after

On the morning you point the domain, or the moment the site becomes public, walk this list once, on the public host, in a logged-out browser and with curl:

• Staging is closed, passworded, or noindex and disallowed, and production is not still disallowed or noindex site-wide.
• HTTP, the unused host, and slash variants redirect to the one canonical shape, in as few hops as you can manage.
• robots.txt allows what it should, lists the sitemap, and can be fetched over HTTPS.
• The sitemap is well formed, lists only indexable URLs, and is submitted in Search Console.
• A sample of important pages has a unique title, one H1, a description, a self-canonical, and a complete first HTML response.
• JSON-LD, if present, parses and matches the page; llms.txt, if present, is accurate and allowed.
• Firewall rules are not serving a challenge to the crawlers you intend to welcome.
• The contact form works, and analytics and Search Console are attached to the right host.
• Security headers and the basic performance checks are good enough that you would be willing to send a stranger to the site.

In the week after:

• Watch Search Console for coverage surprises: unexpected noindex, 404s, or a sitemap that did not process.
• Fetch a few URLs as Googlebot or with a raw client if a page “looks fine” but is reported as empty or redirected.
• Ping IndexNow for pages you added after the first batch, and keep the sitemap in step.
• Fix anything you skipped, and write down the items that should become part of every later release, not a one-off panic.

Launch is a day. Indexing is a queue. The checklist is there so the queue starts with the site you meant to publish.

Key takeaways

• Close or lock staging so the preview cannot be indexed, and make sure production does not inherit a site-wide Disallow or noindex.
• Choose one HTTPS URL shape, and make redirects, canonicals, sitemaps and internal links agree with it.
• Go live with an accurate sitemap, a deliberate robots policy, complete first HTML, and titles and H1s that match the pages.
• Decide your schema, llms.txt and AI crawler rules before launch, and do not put a firewall challenge in front of content you want fetched.
• Verify Search Console, forms and a short live-host checklist on the morning you switch, then watch coverage in the week after.

If you want a second pair of eyes on a site the week before it goes public, tell me about it.

Frequently asked questions

Written by Lee Johnson. Published 10 November 2026.