Sitemaps

XML sitemap fixtures covering parsing, the 50,000-URL protocol limit, gzip handling, malformed XML, sitemap index traversal, extension namespaces, and sitemap-only discovery.

Fixture sitemaps in this group are test subjects served under /tests/sitemaps/fixtures/. They are deliberately distinct from the site’s own generated /sitemap.xml and /sitemaps/<scenario>.xml crawl-discovery outputs: no route in this group ever appears in those generated sitemaps, so every discovery-source assertion stays controlled.

Families:

• Baseline and discovery: a schema-valid single-URL sitemap referenced from the generated robots.txt, a sitemap deliberately absent from robots.txt, a robots.txt Sitemap: line that returns 404, and a page discoverable only through a sitemap (zero HTML anchors anywhere reference it).
• Limits and encoding: streamed sitemaps at exactly 50,000 and 50,001 URLs, a gzipped .xml.gz sitemap served as application/gzip with no Content-Encoding header, and an unclosed urlset with a bare ampersand that strict XML parsers must reject.
• Referenced-URL status: single-URL sitemaps whose targets return 404, 301, and 410, plus targets whose canonical points elsewhere or that carry a meta robots noindex.
• Sitemap index: a sitemapindex referencing two child sitemaps, each listing one page reachable only through index traversal.
• Extension sitemaps: image 1.1, video 1.1, and news 0.9 Google namespaces plus reciprocal en/de xhtml:link hreflang alternates. The news publication date is frozen at 2026-01-01, so staleness warnings from date-sensitive crawlers are expected; the signal is parse-without-error.

The limit pair and the malformed sitemap are risky-manifest, direct-entry only; the rest are safe within the fixture domain. Targets and index children referenced only by fixture sitemaps live in the canonical manifest only.

Crawler-specific caveats: gzipped sitemap detection may rely on the .xml.gz extension or the gzip magic bytes rather than the content type; whether the 50,001st URL is processed or truncated is recorded as the crawler’s observed policy, not asserted.

Fixtures in this category

• Valid single-URL sitemap — valid-single-url (safe)
• Sitemap absent from robots.txt — not-referenced-in-robots (safe)
• Shared 200 sitemap target — sitemap-target-valid-200 (safe)
• Gzipped sitemap file — gzip-sitemap (safe)
• Sitemap listing a 404 URL — url-404 (safe)
• Sitemap listing a redirecting URL — url-301 (safe)
• Sitemap listing a 410 URL — url-410 (safe)
• Sitemap listing a non-canonical URL — non-canonical-url (safe)
• Sitemap listing a noindex URL — noindex-url (safe)
• Canonical sitemap target — sitemap-target-canonical (safe)
• Sitemap index — sitemap-index (safe)
• Image extension sitemap — image-sitemap (safe)
• Video extension sitemap — video-sitemap (safe)
• News extension sitemap — news-sitemap (safe)
• Hreflang sitemap — hreflang-sitemap (safe)

Back to the fixture index