← All posts

Your sitemap is unreachable and that's why your pages aren't indexed

By ·

We had a site deployed on a Cloudflare Worker route, mounted under a path on an existing domain — something like example.com/google/*. Most of its pages never showed up in Google.

The mechanism

Sitemap discovery happens at the host root. Google fetches example.com/robots.txt and reads the Sitemap: directive there.

But example.com/robots.txt isn't served by your Worker. It's served by whatever owns the root of the domain — in our case the main marketing site. Meanwhile your framework emits its sitemap relative to its own base, so it lands at example.com/google/sitemap.xml.

Nothing at the root ever points to it. The sitemap exists, it's valid, and it's effectively invisible.

Why the framework can't save you

This is the part worth sitting with: your framework doesn't own the domain root it's mounted into. It can generate a perfect sitemap and still have no way to advertise it, because the one file that does the advertising belongs to somebody else's app.

The fixes

Moving to a subdomain solves it structurally. On warya.example.com, the app owns its own root, so robots.txt and sitemap.xml sit exactly where crawlers look for them.

It is fixable without moving. A Sitemap: line in the root robots.txt may point at a sub-path sitemap, and Search Console accepts a manual submission. But both require editing the root — which is precisely the thing you don't control.

The lesson

A sitemap that nothing links to is not a sitemap. Before blaming crawl budget or content quality, fetch yourdomain.com/robots.txt and check whether it names a sitemap you actually own.