Claude SEO: the technical guide to getting cited by Claude

How Claude searches and picks sources#

Claude answers most questions from what it learned in training. It searches when the answer depends on something current or changeable — news, prices, a specific product or company — or when the user asks it to. In Profound’s 2026 test of 400+ prompts, Claude searched on 36.6% of them. When it does search, the results come from a different index than most people assume.

Anthropic’s subprocessor list has named Brave Search as its web-search vendor since March 2025, when Simon Willison spotted it and found a BraveSearchParams parameter in Claude’s tool definition. In May 2026 the list reportedly added TurboPuffer, a vector and full-text search database, in an undisclosed web-search role — possibly a re-ranking layer, which would explain why about a fifth of Claude’s citations sit outside Brave’s top 10.

1. 1

   Claude decides whether to search

   Stable facts, code and analysis are answered from training. Anything time-sensitive or specific triggers a search.

   Your lever: Be in the training data too: blocking ClaudeBot keeps future content out of the roughly two-thirds of answers that never search.

2. 2

   It writes its own queries — usually with a year

   Simple questions take one to three searches; comparisons ten or more. Profound found Claude added the current year to 94% of its sub-queries, against 17% for ChatGPT.

   Your lever: Put an honest year in titles and a visible “updated” date on pages where freshness matters.

3. 3

   Brave returns about ten results per search

   Every result — URL, title and page_age — is loaded into Claude’s context. Newer tool versions let Claude write code to filter results before reading them.

   Your lever: Rank in Brave’s top 10 for the queries Claude writes.

4. 4

   Claude fetches the pages it needs

   Full pages are retrieved by Claude-User as raw HTML or PDF. No JavaScript runs, and long pages are truncated to fit the context window.

   Your lever: Server-render. Put the answer in the first screen of HTML, not after a script or a click.

5. 5

   It answers with mandatory citations

   Citations are always on in web search, and each quotes the passage it relies on — up to 150 characters of cited_text. Claude doesn’t sell placement: Anthropic’s February 2026 pledge rules out sponsored links and advertiser influence.

   Your lever: Write self-contained sentences that carry a fact in under 150 characters.

Anthropic's three crawlers — and the one you can't see#

Anthropic documents three bots, each with its own job. All three honor robots.txt, including the non-standard Crawl-delay, and respect CAPTCHAs. That last part is unusual: OpenAI, Perplexity and Google all say their user-triggered fetchers may ignore robots.txt. Claude’s doesn’t.

A robots.txt that opts out of training while keeping Claude’s search and live fetches looks like this. Named groups override a User-agent: * block, and the rules must be repeated on every subdomain:

# Claude search index and live fetches: allow
User-agent: Claude-SearchBot
Allow: /
 
User-agent: Claude-User
Allow: /
 
# Model training: your call
User-agent: ClaudeBot
Disallow: /
 
# Brave (Claude's search index) won't crawl what Googlebot can't
User-agent: Googlebot
Allow: /

Check your CDN and WAF next: “block AI bots” toggles override robots.txt. Anthropic publishes its crawler IP ranges at claude.com/crawling/bots.json for verification — though it notes that allowing or blocking by IP alone is unreliable. The old anthropic-ai and Claude-Web tokens are retired; leaving them in robots.txt is harmless.

Getting into Brave Search#

Because Claude’s candidate pages come from Brave, Brave indexing is the prerequisite that most Claude advice skips. Brave has no webmaster console and doesn’t support IndexNow, so you can’t push pages in. It discovers URLs two ways: its own crawler, and the opt-in Web Discovery Project, in which Brave browser users anonymously report pages they visit.

• Rank on Google and Bing first. Per MERJ’s analysis of the Web Discovery Project, a single opted-in user who finds your page in a Google, Bing or DuckDuckGo result can report it — otherwise a new URL needs about 20 separate visitors.
• Get linked from pages Brave already knows. Links from established sites are the crawler’s main route to you.
• Use the re-fetch form after changes. search.brave.com/submit-url asks Brave to re-crawl a page. It doesn’t guarantee indexing.
• Keep URLs clean. The discovery client drops URLs with more than one query parameter or a long query string, unless a canonical points to a clean version.
• Put canonical and noindex in the HTML head. The discovery client reads them from <head>, not from HTTP headers, and drops pages that redirect.
• Stay fast and light. Discovery fetches time out after 10 seconds and refuse pages over 2 MB.

Then check where you stand: search your target questions on search.brave.com — with the year appended, the way Claude writes them — and treat a top-10 Brave position as your Claude eligibility test.

Technical requirements#

Anthropic’s web fetch documentation is blunt: it “does not support websites dynamically rendered via JavaScript.” Vercel’s crawler study found ClaudeBot downloading JavaScript files without running them. Whatever isn’t in the HTML response doesn’t exist for Claude.

Server-rendered HTMLRequired

The answer must be in the raw HTML — SSR, static generation or pre-rendering. Client-rendered React, Vue or Angular apps show Claude an empty shell.

Crawlable by GooglebotRequired

Brave mirrors Googlebot’s robots rules, so a Googlebot block removes you from Claude’s index source.

No walls on citable contentRequired

Anthropic’s bots respect CAPTCHAs and don’t access login-gated pages. Interstitials and gates hide the answer.

Answer near the topHelps

Long pages are truncated to a token budget — about 2,500 tokens per 10 kB of page. The first screens of HTML are the ones that reliably get read.

Visible freshnessHelps

page_age is passed to the model, and Claude’s queries include the year. Show a real “updated” date and keep dateModified honest.

Markdown for agentsHelps

Claude Code sends Accept: text/markdown. Serving markdown to that header (Cloudflare’s Markdown for Agents, or your own server) gives it a clean read.

Structured dataUnconfirmed

No statement from Anthropic. Web fetch returns page text, so facts that exist only in JSON-LD may never reach the model — keep them in visible copy too.

SitemapsUnconfirmed

Not documented for Anthropic or Brave. Keep them for Google and Bing, which feed Brave’s discovery indirectly.

llms.txtUnconfirmed

Anthropic publishes its own for developers, but has never said Claude reads other sites’ files. Ahrefs found 97% of llms.txt files get zero requests.

IndexNowNo effect

Brave doesn’t support it. It still helps with Bing, and so with ChatGPT.

The quickest test is to request a page the way Claude does and check your key sentence is there. If it isn’t, neither Claude nor Brave can see it:

# Does your server (and WAF) serve Claude the real content?
curl -s -A "Claude-User" https://yoursite.com/pricing \
  | grep -c "Plans start at"
 
# 0 means the text is rendered by JavaScript or blocked

What Claude cites#

Claude’s citation patterns differ sharply from ChatGPT’s — the two share only 8% of cited domains. The studies agree on a few things: Claude leans on first-party and primary sources, likes fresh comparison content, and cites fewer domains per answer than Perplexity.

• Brave ranking

  Observed

  The strongest predictor found so far: 79.2% of cited URLs sit in Brave’s top 10 for the query Claude ran.

• First-party sources

  Observed

  Brand domains took 64% of Claude’s citations in Otterly’s June 2026 study of 379K citations in SaaS and tech. Your own pricing, docs and specs pages are citable assets.

• Primary over secondary

  Official

  Anthropic tunes Research to prefer primary sources over content farms, and evaluates it on exactly that.

• Fresh lists and comparisons

  Observed

  Listicles were 36.4% of the pages Claude cited, against 19.7% for ChatGPT — and nearly every query carries a year.

• Short, quotable facts

  Official

  Each citation quotes up to 150 characters. A sentence that states the fact on its own is easier to cite than one that leans on the paragraph around it.

• Consistency across runs

  Observed

  Claude cites a median of 3.6 domains per answer and about half change between runs (Attrifast, 2026). Measure across repeated runs, not one screenshot.

One more thing that doesn’t work: buying your way in. Anthropic’s February 2026 pledge says Claude will carry no sponsored links and its answers won’t be influenced by advertisers.

Tracking Claude traffic and crawls#

Clicks from Claude arrive with the referrer claude.ai and no UTM parameters. On 13 May 2026 GA4 added a default AI Assistant channel for referrers on Google’s list of assistants — but Google’s channel documentation names ChatGPT, Gemini, DeepSeek, Copilot and Grok, not Claude. Check where your claude.ai sessions land; if they’re still in Referral, add a custom channel above it with the regex below. Some app traffic loses its referrer entirely and lands in Direct.

# Session source — Claude only
^(.*\.)?claude\.ai$
 
# Session source — all major AI assistants
chatgpt\.com|claude\.ai|perplexity\.ai|gemini\.google\.com|copilot\.microsoft\.com

Your server logs show the other half: which pages Claude reads. Claude-User hits are the best proxy you have for live demand, because each one is a real conversation that needed your page.

# Anthropic bot hits by type
grep -oE "ClaudeBot|Claude-SearchBot|Claude-User" access.log | sort | uniq -c
 
# Pages Claude fetched for users — your live-demand list
grep "Claude-User" access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20

Verify that hits are genuine against the published IP ranges — except Claude Code, which fetches from users’ own machines. Brave’s crawler can’t be separated out by user agent at all.

Myths worth dropping#

The action checklist#

Everything above, in the order we’d do it. Tick items off as you go — your progress is saved in this browser.

Claude SEO: frequently asked questions#