Do AI Crawlers Run JavaScript? Auditing and Fixing Missing Citations for CSR/SPA Content

Do AI Crawlers Run JavaScript? Auditing and Fixing Missing Citations for CSR/SPA Content

Do AI crawlers run JavaScript? The answer is short, most of them do not. When Vercel and MERJ analyzed crawler request logs across nextjs.org and the rest of their network, not one bot from OpenAI, Anthropic, Perplexity, Meta, or ByteDance rendered JavaScript^[1]. Plenty of teams never learn this. They open Google Search Console, see a green indexed status, and decide the page is healthy, because Google does render. This page covers where that asymmetry starts, how to tell in three minutes whether your site sits in the gap, and what to fix in what order.

30-second definitions

• Client-side rendering (CSR) builds the visible page by running JavaScript in the browser.
• Server-side rendering (SSR) finishes the HTML on the server at request time and sends it down.
• Static rendering (SSG) finishes the HTML at build time and stores it as a file.
• Hydration takes the finished HTML the server sent and layers JavaScript behavior on top of it in the browser.
• A rendering blind spot is content the browser shows but the raw HTML omits, which puts it out of reach for any crawler that skips JavaScript.

JavaScript execution by crawler

Five of those eight rows say no. If a site leaves its body content to client-side rendering, those five crawlers receive none of the page's substance.

The evidence, in numbers

Vercel and MERJ split the crawler requests hitting nextjs.org and the rest of their network by content type. Two findings stand out. The major AI crawlers do not render JavaScript, and yet they still pull down JavaScript files at a meaningful rate^[1]. The shares below are measured on nextjs.org, and the ChatGPT family combines GPTBot, OAI-SearchBot, and ChatGPT-User. They fetch the files without running them, so a JS request in your server log is no proof that a bot rendered your app.

Volume is worth noting too. Over the same one-month window, GPTBot made 569 million requests and the Claude family made 370 million, against 4.5 billion for Googlebot (Vercel, MERJ, 2024)^[1]. AI crawler traffic has passed the point where anyone can wave it off, and every one of those requests reads raw HTML and moves on.

The other side of the story sits in official documentation. Google describes processing JavaScript apps in three stages, crawling, rendering, and indexing, with rendering handled by an evergreen version of Chromium (Google Search Central, 2026)^[2]. Applebot states that it can render website content inside a browser, and warns that blocking JS and CSS in robots.txt prevents proper rendering (Apple, 2026)^[7]. Bing wrote that Bingbot can generally render JavaScript but has trouble doing it at scale for every page on every site (Bing Webmaster Blog, 2018)^[8]. OpenAI's documentation separates the roles of GPTBot (model training), OAI-SearchBot (ChatGPT search surfacing), and ChatGPT-User (user-triggered visits), and says nothing about rendering (OpenAI crawler documentation, accessed September 2026)^[6].

Why the page survives in Google and vanishes from AI answers

The difference comes down to how many stages each pipeline runs. Googlebot queues a page that returned HTTP 200 for rendering, and when its turn arrives, headless Chromium executes the JavaScript and the resulting HTML goes to the index^[2]. Google attaches the caveat that the wait can run from seconds to considerably longer, but execution does happen.

The AI crawlers with no observed rendering have no second stage (Vercel, MERJ, 2024)^[1]. Request, response, parse, and that is the whole sequence. That produces the asymmetry teams keep running into, a page that ranks well in Google search never shows up in the source list behind a ChatGPT or Perplexity answer. Months disappear into blaming content quality or prompt wording. The actual cause is one line of HTML.

The effect spills back onto Google as well. To appear as a supporting link in AI Overviews and AI Mode, Google requires that a page be indexed and eligible to appear in Google Search with a snippet (Google Search Central, 2025)^[5]. Failed or delayed rendering delays indexing, and delayed indexing pushes back eligibility on AI surfaces.

Structured data is the sharpest trap. Google reads JavaScript-generated structured data from the DOM after rendering and supports injecting JSON-LD through a tag manager (Google Search Central, 2025)^[4]. The Rich Results Test passes. For a crawler that skips JavaScript, that JSON-LD does not exist. The markup design covered in structured data schemas for AEO is worthless for AI citation if the injection point is the client.

The elements that land in the blind spot most often

Execution, from audit to fix

Step 1: Audit the Raw HTML (3 Minutes)

The Elements tab in your browser devtools shows a DOM that has already finished rendering, which makes it useless for this audit. You need the HTML the server sent first.

# 1. Is the key body sentence in the raw HTML?
curl -sL "https://example.com/product/123" | grep -c "key phrase"

# 2. Is JSON-LD in the raw HTML?
curl -sL "https://example.com/product/123" | grep -o "application/ld+json" | wc -l

# 3. Do internal links exist as real anchors?
curl -sL "https://example.com/category/shoes" | grep -o '<a [^>]*href="[^"]*"' | wc -l

# 4. Does a crawler UA get a different response? (check each vendor's docs for exact UA strings)
curl -sL -A "GPTBot" "https://example.com/product/123" | wc -c

If checks 1 and 2 return 0, that page is blank to ChatGPT, Claude, and Perplexity. Looking fine in a browser proves nothing.

Step 2: The Diagnostic Checklist

Checks 4 and 5 run through the URL Inspection tool and Rich Results Test that Google's documentation points to^[2]. Check 6 belongs with the per-bot allow policies covered in the AI crawler management guide.

Step 3: Fix Priority

Google has already settled the direction. It calls dynamic rendering a workaround rather than a recommended solution, one that adds complexity and resource demands, and points instead to server-side rendering, static rendering, and hydration (Google Search Central, 2025)^[3]. Bolting on a prerender service will put out the immediate fire, but treating it as the final architecture leaves you with maintenance costs and a standing risk that the two versions drift apart.

The rule is simple. Anything quotable comes from the server, and only the interactions nobody will ever quote go to the browser. Vercel and MERJ landed in the same place: server-render body content, product data, documentation, metadata, and navigation, and keep client-side rendering for non-essential dynamic elements^[1].

To run the raw HTML audit with tooling instead, the free GEO audit tool comparison maps what the free tiers actually answer. Once the fix ships, track citations separately. A raw HTML audit answers whether a page can be read, not whether it gets cited. Teams track per-engine citations with AI visibility monitoring tools such as Profound and Peec AI, or Korean options including BOIDA, Nextt, and LeadGenLab. The alternative is to query each engine directly and collect the source links. If rendering is fixed and citations still do not appear, the cause has moved to content structure, and why Perplexity does not cite your content plus the technical GEO and content GEO framework pick up the next step. For the concept itself, start with what GEO is.

Summary

Most AI crawlers do not execute JavaScript. GPTBot, ClaudeBot, PerplexityBot, Meta-ExternalAgent, and Bytespider all fall in that group, and while they download JS files, they never run them (Vercel, MERJ, 2024). Googlebot documents a rendering step, Applebot states it can render in a browser, and Bingbot says it cannot do so at scale. That leaves a CSR page in a split state, alive in Google search and absent from AI answers. Verify with curl rather than a browser, and fix with server-side and static rendering rather than dynamic rendering. Quotable content comes from the server, everything else can stay in the browser, and that one line is the baseline that closes the blind spot.