The technical SEO checklist

Before anyone can find your page, a search engine has to crawl it (download it), render it (run it like a browser would, if it needs JavaScript) and index it (store it so it can be shown for a search). Ranking only starts after that. If any step fails, the best writing on your site earns nothing, because nobody is shown it.

This module covers those basics in the order you should check them, for a site built with an AI coding tool such as Lovable, Bolt, v0, Cursor or Replit. Those tools produce good-looking sites fast, but they can skip the parts no visitor sees, like the HTML that crawlers read. Everything below follows Google Search Central documentation, listed in the sources.

#How Google reads a page, and where AI-built sites go wrong

Googlebot first fetches the HTML your server sends. If the page is a JavaScript app, Google queues it, runs the JavaScript in a headless Chromium browser later, and indexes what appears. Google's own advice is that server-side rendering or pre-rendering "is still a great idea because it makes your website faster for users and crawlers, and not all bots can run JavaScript."

Other crawlers are less patient. In a December 2024 analysis of its network traffic, Vercel found that none of the major AI crawlers it measured rendered JavaScript, including OpenAI's, Anthropic's ClaudeBot and PerplexityBot. If your pricing table only exists after JavaScript runs, they see an empty page.

Many AI-built sites are single-page apps of exactly this kind: one index.html containing <div id="root"></div> and a JavaScript bundle. Every URL returns the same HTML, the same title and no text. You can check yours in ten seconds:

curl -s https://yourapp.com/pricing | grep -i -E "<title>|<h1"

If every page shows the same title and no <h1>, crawlers that don't run JavaScript see nothing. "View page source" in your browser shows the same raw HTML; the Inspect panel shows the page after JavaScript ran, so it can fool you.

The fix is to send real HTML. Frameworks such as Next.js, Astro, Nuxt and SvelteKit can render pages on the server or at build time. Or pre-render just the public pages (home, pricing, docs, blog) and keep client-only rendering for the logged-in app. Two smaller rules from Google's JavaScript guide: use real paths through the History API instead of #/pricing fragments, and make links real <a href="..."> elements, because Google only follows <a> tags with an href.

#Set up Search Console before anything else

Google Search Console is Google's free dashboard for site owners. It tells you, page by page, whether Google indexed your site and why not. Set it up on launch day.

1. Add your site as a Domain property (it covers http, https, www and every subdomain) and verify it with the DNS record Search Console gives you.
2. Submit your sitemap in the Sitemaps report (see below).
3. Paste a URL into URL Inspection. It shows whether the page is on Google and which canonical Google chose. "Test live URL" shows the HTML Google rendered, and "Request indexing" asks Google to crawl the page.
4. A few days later, open the Pages report. Every URL Google knows about is listed as indexed or not indexed, with a reason.

Each reason points to a fix. URL marked 'noindex' and URL blocked by robots.txt are instructions you gave, on purpose or not. Soft 404 means the page returned status 200 but looks like an error or an empty page. Duplicate without user-selected canonical means Google found copies and you never said which is the original. Crawled - currently not indexed means Google fetched the page and chose not to index it for now. Google calls that "a common message for many sites"; a better page with more internal links to it is often the fix.

Then add the site to Bing Webmaster Tools, which can import and verify it straight from Search Console. Bing's index also serves Microsoft Copilot, and Bing supports IndexNow, a protocol for telling search engines a URL changed. Google does not take part in IndexNow. For a rough check at any time, search Google for site:yourapp.com: if results appear, at least part of the site is indexed.

#Tell crawlers what to fetch and which URL is the real one

#robots.txt

robots.txt is a plain text file at the root of your domain (https://yourapp.com/robots.txt) that tells crawlers which URLs they may fetch. Google is explicit that it "is not a mechanism for keeping a web page out of Google": a blocked URL can still be indexed, without its content, if other pages link to it. Use it to keep crawlers out of areas that don't belong in search, like the logged-in app, the API and checkout.

# https://yourapp.com/robots.txt
User-agent: *
Disallow: /app/
Disallow: /api/
Disallow: /checkout/

Sitemap: https://yourapp.com/sitemap.xml

Check for two mistakes. One is a staging rule shipped to production: Disallow: / under User-agent: * blocks the whole site. The other involves named groups. Under the robots.txt standard (RFC 9309), which Google follows, a crawler obeys only the most specific group that names it and ignores the * group. If you add a group for one bot, repeat the Disallow lines it should still respect. Which AI crawlers to allow, and what blocking each one does, is covered in Get cited by AI assistants.

#XML sitemap

A sitemap lists the URLs you want indexed. Google finds most pages through links, so a sitemap guarantees nothing, but it helps a new site get discovered and says when each page last changed. Google's rules: use full absolute URLs, list only the canonical version of each page, stay under 50,000 URLs or 50 MB per file, and keep <lastmod> honest, because Google only uses it when it is "consistently and verifiably" accurate. Google ignores <priority> and <changefreq>, so leave them out.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://yourapp.com/</loc>
    <lastmod>2026-10-28</lastmod>
  </url>
  <url>
    <loc>https://yourapp.com/pricing</loc>
    <lastmod>2026-10-14</lastmod>
  </url>
</urlset>

Most frameworks have a plugin that builds it. Reference it in robots.txt and submit it in both webmaster tools.

#Canonical URLs

The same page often answers at several addresses: with and without www, with a trailing slash, with ?utm_source= tracking, or on your host's preview domain as well as your own. A canonical URL is the one address you want search engines to show. Declare it with a tag in the <head> of every page, pointing at the page itself:

<link rel="canonical" href="https://yourapp.com/pricing">

Use absolute URLs. Google treats both a canonical tag and a permanent redirect as strong signals, so the best fix for a preview domain is to redirect it to your own. Google can still pick a different canonical when your signals disagree, for example when the sitemap lists www URLs and the tags don't. Don't use robots.txt or noindex to choose between duplicates.

#Status codes and redirects

Google reads the status code first. 200 means "here is the page." 301 and 308 are permanent redirects, which tell Google to show the new URL; 302 and 307 are temporary, so the old URL can stay in results. Google doesn't index URLs that return 404 or 410. Server errors (5xx) slow Google's crawling, and pages that keep failing are eventually dropped.

The classic single-page-app bug is the soft 404: a mistyped URL like /pricng shows a friendly "not found" screen with status 200, so Google may index piles of empty pages. Return a real 404 from the server, or add <meta name="robots" content="noindex"> to the not-found view.

Redirect straight to the final URL. Googlebot follows up to 10 hops, but Google advises "redirecting to the final destination directly", and every hop adds delay for people. A 404 for a page that is really gone is correct. Redirect it only when links or visitors still arrive and a close replacement exists.

#noindex

To keep a page out of Google (a thank-you page, internal search results), add <meta name="robots" content="noindex"> or send an X-Robots-Tag: noindex header. The page must stay crawlable, because if robots.txt blocks it Google never sees the tag. Before launch, search your codebase for noindex: a leftover from staging is a common reason a new site never appears.

#Help Google understand each page

#Titles and meta descriptions

Every page needs its own <title>. Google uses it for the clickable headline in results, the title link. Its guidance: descriptive and concise, unique to the page, no keyword stuffing or boilerplate, the brand added briefly at the start or end. There is no hard character limit; Google truncates titles to fit the screen.

The meta description is the summary under the title. Google says snippets are "primarily created from the page content itself" and uses your description when it fits the page better. Write one per page anyway, one or two sentences on what the page offers and to whom. Long ones get cut.

#Headings

Give each page one <h1> that says what the page is, then <h2> and <h3> for sections. Google says heading order doesn't matter to Search and there is no ideal number, so this is mostly for readers and screen readers. One clear H1 per page is still a good convention.

#Internal links

Google finds pages mainly through links from pages it already knows. A page nothing links to (an orphan page) is hard to find even when it's in the sitemap. Link every public page from somewhere sensible, with anchor text that says what's on the other side: "pricing for teams", not "click here". Breadcrumbs on deep pages show people and crawlers where a page sits.

#Image alt text

Google indexes images in <img> elements, not CSS backgrounds, and calls alt text "the most important attribute" for describing an image. Say what's in it, briefly: alt="Dashboard showing 1,240 invoices processed this week", not alt="dashboard invoices OCR invoice OCR".

#Structured data

Structured data is a block of machine-readable facts about the page in the schema.org vocabulary. Google recommends the JSON-LD format inside a <script> tag. It can make a page eligible for rich results (extra elements in the listing) but never guarantees them, and Google's rule is to mark up only what is visible on the page.

Take a hypothetical invoice-scanning product called Acme. On its home page, an Organization block gives its name, logo and official profiles:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "Organization",
  "name": "Acme",
  "url": "https://yourapp.com/",
  "logo": "https://yourapp.com/logo.png",
  "sameAs": [
    "https://github.com/acme/acme",
    "https://www.linkedin.com/company/acme"
  ]
}
</script>

On a blog post, an Article block names the author and the dates:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "Article",
  "headline": "How accurate is OCR on handwritten invoices? We tested 500",
  "datePublished": "2026-10-02T09:00:00+00:00",
  "dateModified": "2026-10-20T15:30:00+00:00",
  "author": [{
    "@type": "Person",
    "name": "Dana Reyes",
    "url": "https://yourapp.com/team/dana-reyes"
  }]
}
</script>

Product, SoftwareApplication and BreadcrumbList fit where a page really is a product or sits deep in a hierarchy. Some features are gone: Google stopped showing HowTo rich results in 2023 and FAQ rich results on May 7, 2026, and breadcrumbs have shown on desktop only since January 2025. FAQPage markup that matches a visible FAQ does no harm, but expect nothing from it in Google. Test markup with the Rich Results Test and the Schema Markup Validator.

#Speed, mobile and HTTPS

Core Web Vitals are three measurements of real visits that Google's ranking systems use:

Anything in between "needs improvement." Each metric is measured at the 75th percentile of page loads, separately for mobile and desktop, so three visits in four should meet the target. INP replaced the older FID metric in March 2024. Search Console's Core Web Vitals report and PageSpeed Insights show your real-visit data once Chrome has collected enough.

Keep speed in proportion. Google says good scores don't guarantee top rankings and that it "always seeks to show the most relevant content, even if the page experience is sub-par." On AI-built sites the fixes are usually the same: resize and compress images (Google Images supports WebP and AVIF), give every image and embed a width and height so nothing jumps, and ship less JavaScript on public pages.

Google indexes the mobile version of your site (mobile-first indexing), so the phone layout must carry the same text, links and structured data as desktop. Responsive design, which Google recommends, does that by default. Avoid pop-ups that cover the content on arrival. Serve everything over HTTPS and 301-redirect http:// to https://.

#Common problems and the launch checklist

The first twelve items can usually be done in a day. The last four are about whether the site deserves to rank, and they take ongoing work.

That last item is where SEO meets promotion. Google names links and references from other prominent websites as one factor in judging quality. Placing honest, useful work on sites your audience already reads is what agents on promotedby.ai are paid for.

#Try it