crawlview
See what search engines and AI crawlers actually store for your pages — not what your browser shows you.
Your browser runs JavaScript, so the page in DevTools is a finished page. Most crawlers do not run it. What they keep can be an empty container, and nothing on screen tells you.
crawlview requests a page once as each crawler, compares what comes back, and says where the browser and the machines disagree. One command, no account, nothing to install.
One command
npx crawlview https://example.com --renderNeeds Node 20 or newer. The browser comparison uses a Chrome already on the machine — nothing is downloaded behind your back.
A page that renders in the browser and nowhere else
AGENT STATUS HTML TEXT TITLE DESC H1 CANON LD
Googlebot 200 870 B 0 w ok ok 0 ok 0
bingbot 200 870 B 0 w ok ok 0 ok 0
GPTBot 200 870 B 0 w ok ok 0 ok 0
ClaudeBot 200 870 B 0 w ok ok 0 ok 0
PerplexityBot 200 870 B 0 w ok ok 0 ok 0
browser (rendered) 200 15 KB 2,408 w ok ok 1 ok 1
x problems
2,408 words render in the browser; every crawler stores none of it.
Crawlers see no internal links at all.
Structured data is injected by JavaScript.The table is the report. The sentence under it is the diagnosis — which is the part a crawl of raw HTML cannot give you.
Nine groups of checks
Divergence
What each crawler stores against what a browser renders: text, structured data, and internal links that only exist after JavaScript runs — which is how a whole site becomes undiscoverable from its own homepage.
robots.txt, per crawler
Whether each crawler is allowed this exact URL and which rule decided it, including Google-Extended and Applebot-Extended: tokens that gate AI answers but never fetch anything, so no other method can observe them.
Indexing directives
X-Robots-Tag headers, which are invisible in the page source and outrank the meta tag. A stray noindex in a header survives for months because nobody thinks to look there.
Structured data
Required properties, and claims the page itself never makes. Markup may only describe what a visitor can see; a price or phone number that exists solely in the JSON-LD is what manual actions are issued for.
Declared language against real language
A translated route that renders the default copy declares one language and serves another. The tell is identical word counts across translations, and the site looks perfect until somebody reads it.
hreflang
Every alternate is fetched and checked for a link back. A set where one member does not reciprocate is discarded whole — the other translations lose their links too.
AI answer readiness
Whether crawlers that never run JavaScript receive anything, how much text survives content extraction, whether headings produce usable sections, and whether the page can be cited at all.
Site-wide
Sitemap entries that 404, redirect, are noindex or disallowed; pages that canonicalise elsewhere; orphans linked from nowhere; duplicate titles; routes that are secretly the same page.
Soft 404s
A URL that cannot exist answering 200 — what a single-page app does to every misspelled link on the internet that points at your site.
Fail the build, not the quarter
A rendering regression is usually found weeks later, in Search Console, after the traffic has already gone. It is a build failure that nobody arranged to catch.
crawlview exits non-zero when a crawler sees less than a set share of the browser's text, and a committed snapshot turns that into a comparison against the last known-good run. A published GitHub Action wires it into a pull request.
Ran it on your site and want the problems gone?