URLs and normalisation

URL format, encoding, duplicate-parameter, tracking-parameter, internationalisation, host, and path normalisation fixtures.

Fixtures for URL-shape behaviour: how a crawler normalises, deduplicates, and records URLs that differ only in shape, encoding, parameters, or host form.

Families:

• Trailing slash: a slashed/slashless pair served as two distinct 200 responses with their own markers, so slash normalisation is observable. The app pins Astro’s trailingSlash: 'never'; the slashed variant is served by an exact-path middleware handler.
• Default documents: literal index.html, index.htm, and default.html files in the static directory so the URLs resolve verbatim.
• Path shapes: double/triple slashes, a percent-encoded space, a double-encoded %2520, and a path-case pair. The slash and double-encoding fixtures require the deployment path to pass raw URLs through unmangled and declare the raw-url-passthrough capability.
• Fragments and queries: a stable fragment source page, duplicate query keys, and a query-order pair served by one endpoint registered under two query-string variants.
• Length: a page linking to a deterministically generated ~1,800-character URL, kept risky and direct-entry only.
• Parameters: session-ID and UTM tracking parameter echoes.
• International paths: Hebrew, Japanese, Polish, and Arabic percent-encoded path segments.
• Hosts: punycode IDN and hostname-case fixtures owned by supporting profiles, plus a trailing-dot FQDN variant of a primary-host page.

Fixtures in this category

• Trailing slash — with slash — urls-trailing-slash-with (safe)
• Trailing slash — without slash — urls-trailing-slash-without (safe)
• Default document — index.html — urls-index-html (safe)
• Default document — index.htm — urls-index-htm (safe)
• Default document — default.html — urls-default-html (safe)
• Double slash in path — urls-double-slash (safe)
• Triple slash in path — urls-triple-slash (safe)
• Percent-encoded space in path — urls-space-encoded (safe)
• Double-encoded percent sign in path — urls-double-encoded (safe)
• Path case sensitivity — urls-path-case (safe)
• Fragment link source page — urls-fragment-source (safe)
• Duplicate query keys — urls-duplicate-query-keys (safe)
• Query parameter order — a then b — urls-query-order-ab (safe)
• Query parameter order — b then a — urls-query-order-ba (safe)
• Session ID parameter — urls-session-id (safe)
• UTM tracking parameters — urls-utm-parameters (safe)
• International URL — Hebrew — urls-utf8-hebrew (safe)
• International URL — Japanese — urls-utf8-japanese (safe)
• International URL — Polish — urls-utf8-polish (safe)
• International URL — Arabic — urls-utf8-arabic (safe)
• Punycode IDN host — urls-punycode-idn (safe)
• Hostname case normalisation — urls-host-case (safe)
• Trailing-dot FQDN host — urls-trailing-dot-host (safe)

Back to the fixture index