Crawler behaviour

How a crawler's observations are compared against fixture expectations.

Each registry entry carries an expectation object — the API between this site and a crawler’s test runner. The crawler crawls the route; the test runner diffs what the crawler recorded against the expectation fields.

Signals fall into families:

• First-response signals: expectedStatus, expectedHeaders, expectedContentType — apply to every fixture.
• Redirect-flow signals: expectedLocation, expectedFinalUrl, expectedFinalStatus, with a mandatory maxFollowCount bound on chains and loops.
• Body markers: stable grep-able strings of the form sb-fixture:<route-id>:<state> evaluated against the raw response body.
• Rendered markers: the same convention evaluated against the rendered DOM, for fixtures whose behaviour only appears after JavaScript runs.
• Directives: robots meta, X-Robots-Tag, canonicals, hreflang, rel next/prev, refresh — each declaring whether it should be read from the raw or rendered document.
• Links: expected outlinks with rel attributes and discoverability flags.

Raw/rendered marker pairs prove what JavaScript changed. Markers are part of a fixture’s frozen observable behaviour: once a route ID ships, its behaviour never changes — a behaviour change means a new route ID.

Back to the fixture index