On this page11 sections
The short answer
To get cited by Grok, serve your key pages as plain server-rendered HTML, keep bot challenges off them, and be present in both places Grok searches: the open web and X. xAI documents a web search tool that browses pages and an X search tool that finds posts by keyword, meaning, account and thread — but no crawler user agent or robots.txt token, so there’s nothing to allow by name. The work is crawlable first-party pages, mentions on the community sites Grok cites most, and an active X account that states your facts in text.
Key takeaways
- Grok has two live sources: web search with page browsing, and X search by keyword, meaning, account and thread. Both are cited inline.
- xAI publishes no crawler documentation. Cloudflare says Grok’s fetcher “does not self-identify at all,” and tests saw only browser-like user agents.
GrokBot,xAI-GrokandGrok-DeepSearchcome from third-party bot lists, not xAI. A robots.txt rule naming them may never match a request.- Reddit (16.3%), YouTube (15.1%) and Facebook (13.9%) lead Grok’s 50 most-cited domains in Ahrefs’ June 2026 data. x.com is 12th, at 1.4%.
- In a June 2026 test, Grok answered from a page’s raw HTML even though one of its fetch nodes ran the JavaScript. Server-render.
- grok.com’s citation links carry
noreferrer, so many Grok clicks land in Direct — where GA4’s new AI Assistant channel can’t see them.
How Grok searches the web and X#
Grok answers from training when it can and searches when a question needs something current. SpaceX’s May 2026 S-1 describes Grok Chat as offering “real-time integration of web search, X data, code execution, and multimodal analysis” on grok.com, the Grok apps, inside X and through the API. xAI — acquired by SpaceX effective 2 February 2026 and now branded SpaceXAI — gives its API the same two search tools, and their documentation is the clearest public description of what Grok can do.
The Web Search tool lets Grok “search the web in real-time and browse web pages.” The X Search tool performs “keyword search, semantic search, user search, and thread fetch on X.” What the docs leave out matters as much: xAI names no search provider, publishes no index size and documents no crawler. Where Grok’s web results come from is not public.
- 1
Grok decides whether to search
The model “decides what to do next: make a tool call, or provide a final answer,” per xAI’s tools overview, and can chain several tools per question. Stable knowledge comes from pre-training, which the Grok 4 model card says includes “publicly available Internet data.”
Your lever: Be in the training data: public, crawlable pages are the only route, since xAI gives websites no documented opt-in or opt-out token.
- 2
It searches the web and X, often in parallel
xAI’s published prompt for @grok says “parallel search should be used to find diverse viewpoints.” The chat prompt tells Grok to “search for a distribution of sources that represents all parties/stakeholders” on controversial questions, and not to “shy away from deeper and wider searches” on X.
Your lever: Be one of the viewpoints it finds: a first-party page for the facts, and posts on X that state them.
- 3
An undisclosed web index returns candidates
xAI doesn’t name the index or say how it ranks results. There’s no webmaster console, no URL submission and no IndexNow participation.
Your lever: Keep your own pages crawlable, and earn mentions where Grok’s citations already concentrate: Reddit, YouTube, review sites.
- 4
Grok opens the pages it needs
Grok 4 Fast “hops through links, ingests media (including images and videos on X),” per xAI’s September 2025 launch post. The fetches carry browser-like user agents from datacenter IPs, and in a June 2026 test the answer came from raw HTML.
Your lever: Server-render, and don’t put a bot challenge in front of content pages.
- 5
It answers with inline citations
The API inserts numbered links like
[[1]](https://x.ai/news). grok.com renders citations as small pills showing the site’s name or an X@handle, and an “N sources” pill under the answer opens the full list. Presenc AI counted an average of 4.7 sources per answer in 2026.Your lever: Write facts that stand on their own, on pages whose titles read well on a source card.
117M
monthly active users used Grok's AI features at 31 March 2026, out of about 550M across Grok and X
16.3%
of citations among Grok's 50 most-cited domains go to Reddit — YouTube 15.1%, Facebook 13.9%
1.4%
for x.com in the same ranking — 12th, although Grok is built into X
Crawlers and access: nothing to allow by name#
OpenAI, Anthropic, Google and Perplexity all publish their AI crawlers. xAI doesn’t: there’s no crawler page, no user-agent token, no IP list and no statement on robots.txt for Grok’s search or live fetches. Cloudflare wrote in September 2025 that “xAI’s bot, grok, does not self-identify at all, making it impossible for website operators to block it.” A February 2026 controlled test by StackFox found the same: asked to fetch test URLs, Grok sent 30 requests in under a second from datacenter IPs, with user agents like Chrome/139.0.0.0 and Go-http-client/1.1. “The word ‘Grok’ never appears. Neither does ‘xAI.’”
| User agent | Source | What it is | robots.txt |
|---|---|---|---|
GrokBot, xAI-Grok, Grok-DeepSearch | Third-party bot lists | Claimed training, search and DeepSearch bots — not in xAI’s documentation | Unknown; StackFox’s test never saw them |
Browser strings, e.g. Chrome/139.0.0.0 | Observed by StackFox, Feb 2026 | Live fetches when a user asks Grok to read a URL | Not documented |
Go-http-client/1.1 | Observed by StackFox, Feb 2026 | The Go HTTP library’s default, seen in the same bursts | Not documented |
GrokAgent | Cloudflare Radar’s bot directory | Grok Bot, xAI’s always-on agent product, launched 11 August 2026 | Listed as not following it |
The consequence: robots.txt can’t target Grok, and neither can a CDN’s “block AI bots” switch, because nothing in the request says it’s Grok. Nothing documents whether Grok’s fetcher reads robots.txt at all. Adding the unverified tokens does no harm, but don’t mistake them for control:
# xAI documents no user agent for Grok's search or fetches.# These names come from third-party lists and are unverified:# a rule for them may never match a real request.User-agent: GrokBotUser-agent: xAI-GrokUser-agent: Grok-DeepSearchAllow: / # Grok's browser-like fetches fall under your default groupUser-agent: *Allow: /Disallow: /account/Training: no website opt-out, one X setting
The Grok 4 model card lists “publicly available Internet data” among Grok’s pre-training sources. With no documented crawler token, site owners have no robots.txt line that xAI says it honors for training. X posts are different: X’s privacy policy, effective 15 January 2026, says X may use what it collects “and publicly available information to help train our machine learning or artificial intelligence models.” Account holders can opt out under Settings and privacy → Privacy and safety → Grok & Third-party Collaborators, which PCMag describes as covering public posts, engagements and profile data. As far as xAI documents, that setting governs training, not whether X search can find and cite your public posts.
X is Grok's second index#
Grok’s second source is X itself. Its X search runs keyword and semantic search over posts, looks up accounts and fetches whole threads, with filters for handles and date ranges and optional image and video understanding. Cited posts come back as x.com/<handle>/status/<id> URLs, per xAI’s citations docs, and grok.com shows them as @handle pills and profile photos in the sources stack.
How much of Grok’s citing goes to X depends on who’s counting. Presenc AI sampled 4,400 prompts in March–April 2026 and found about 45% of Grok’s citations were X posts: “The mix shifts toward X heavily for current-events queries and toward web for definitional or how-to queries.” Ahrefs’ Brand Radar, across 1.9 million broad US queries, put x.com at 1.4% of the citations among Grok’s top 50 domains. Both can be right — a product question gets answered from the web, a what’s-happening question from X — so plan for both.
~45%
of Grok's citations were X posts across 4,400 prompts in March–April 2026
2.8x
the citation rate of posts from verified accounts versus equally-engaged posts from unverified ones
- State the fact in the post’s text. X search matches keywords and meaning. Grok can read images and video, but a claim that lives only in a screenshot or a link preview is the harder one to find.
- Link the post to the canonical page. Grok’s system prompt says it can “analyze individual X user profiles, X posts and their links” — a post is a route to your page.
- Answer recurring questions in threads. Thread fetch reads a whole conversation in one call, so a thread that settles a question completely is a ready-made source.
- Make the account unambiguous. User search is a documented capability. Put the brand name in the handle or display name, and say what you do in the bio.
- Consider verification for the brand account. Presenc AI found verified accounts’ posts cited at 2.8x the rate of equally-engaged unverified ones.
- Post on X the day your facts change. The X tool filters by date and Grok leans on X for current events. A price change or launch that exists only on your blog is easier to miss.
Grokipedia: worth checking, not a Grok lever#
Grokipedia launched on 27 October 2025 with 885,279 articles and passed 6 million by January 2026, per its own article. Some articles are generated by Grok and others were forked from Wikipedia. Nobody edits directly: logged-in users highlight text, click Suggest Edit and submit a correction with supporting evidence, which Grok reviews before anything changes.
Its documented reach has been outside Grok. In January 2026 TechCrunch reported the Guardian’s finding that GPT-5.2 cited Grokipedia nine times across more than a dozen questions. Ahrefs’ March 2026 comparison counted 356,200 AI citations for Grokipedia against 24.9 million for Wikipedia, and found its organic traffic had fallen to half its peak. For Grok itself, xAI doesn’t document any use of Grokipedia in answers, and grokipedia.com isn’t among the 50 domains Grok cites most in Ahrefs’ June 2026 data.
- Search Grokipedia for your brand, products and founders. Generated pages can exist without anyone having asked for them.
- Correct what’s wrong, with a primary source. Link your own documentation, a filing or a dated announcement in the suggestion form.
- Don’t build a Grok strategy on it. Its documented citations are in ChatGPT and other assistants, not in Grok’s top cited domains.
Technical requirements#
Nothing about Grok’s fetcher is documented — not its user agent, rendering, timeouts or size limits. What’s known comes from tests, and it points the same way as every other engine: plain HTML, reachable without a challenge.
- Server-rendered HTMLRequired
- In Search Engine World’s June 2026 test, one Grok fetch node ran the page’s JavaScript, but the answer still quoted the raw-HTML decoy. Put the content in the HTML response — see server-side rendering.
- No challenge on content pagesRequired
- Grok’s fetches arrive as browser bursts from datacenter IPs. A JavaScript challenge, CAPTCHA or datacenter block hands them a page with none of your content.
- Tolerant rate limitsHelps
- StackFox logged 30 requests in under a second when Grok fetched its test pages. Per-second limits tuned for human visitors will cut some of them off.
- Descriptive titlesHelps
- grok.com’s source cards show the page title in bold over a three-line preview, and inline pills show the site’s display name. The title is what a reader sees before clicking.
- Links from X postsHelps
- Grok can follow the links in the X posts it reads, so posts that link your canonical URL are a second path to the page.
- Structured dataUnconfirmed
- xAI has said nothing about schema.org. Keep every fact that matters in visible text as well.
- SitemapsUnconfirmed
- Not documented for Grok. Keep one with honest
lastmodvalues for the engines that do use them. - Canonicals and redirectsUnconfirmed
- Undocumented. Keep one clean URL per page and avoid redirect chains, so whichever copy Grok finds is the right one.
- llms.txtUnconfirmed
- xAI hasn’t said Grok reads llms.txt, and no test has shown it requested.
- robots.txt rules naming GrokNo effect
- There’s no documented token to name. A rule for
GrokBotorxAI-Grokmay never match a request. - IndexNowNo effect
- xAI isn’t on IndexNow’s list of participating engines, so pings don’t reach Grok.
Test pages the way Grok’s fetcher appears to see them — a browser user agent, no JavaScript — ideally from a cloud VM, since that’s where its requests come from:
# A browser user agent and no JavaScript, like Grok's fetchescurl -s -A "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36" \ https://yoursite.com/pricing | grep -c "Plans start at" # 0 = the text needs JavaScript, or your WAF served a challenge pageWhat Grok cites#
Ahrefs’ June 2026 ranking of the 50 domains Grok cites most, built from 1.9 million US queries, is dominated by user-generated platforms: Reddit 16.3%, YouTube 15.1% and Facebook 13.9% — about 45% between them — then Instagram, Quora, Amazon, TikTok and Wikipedia. Retail and review sites fill much of the rest: eBay, Walmart, Consumer Reports, Yelp, Tripadvisor and Trustpilot.
Community platforms
ObservedReddit, YouTube and Facebook are Grok’s top three cited domains. Being discussed there is a route in that doesn’t depend on your own rankings — see Reddit SEO.
Marketplaces and reviews
ObservedAmazon (5.0%), eBay, Walmart, Consumer Reports and Trustpilot all make Grok’s top 50. Product questions get answered from listings and reviews.
X posts, by query type
ObservedAbout 45% of citations in Presenc AI’s sample, 1.4% in Ahrefs’. The share is heaviest for current events and lightest for how-to questions.
Verified accounts
ObservedPosts from verified accounts were cited at 2.8x the rate of equally-engaged unverified posts in Presenc AI’s 2026 sample.
Primary sources
Official@grok is told to “fetch primary sources for current events” and to verify search results by opening pages. Your own announcement page is the primary source for your news.
A spread of viewpoints
OfficialOn contested questions Grok is told to search for “all parties/stakeholders” and to treat media viewpoints as biased. Expect your page to be cited next to others, not alone.
Paid placement isn’t a route yet. Elon Musk told advertisers in August 2025 that X planned ads in Grok’s responses. SpaceX’s May 2026 S-1 still described advertising in the stand-alone Grok product as a possible “incremental monetization opportunity,” and said it doesn’t sell advertisers placement on the Grok API.
Measuring Grok traffic#
Expect little Grok referral traffic in your analytics — and some of the shortfall is measurement. SE Ranking’s study of 101,574 sites found Grok’s share of all website traffic peaked at 0.0019% in August 2025 before falling back toward zero. Goodie’s B2B panel saw “effectively zero attributable referrals” from Grok while Similarweb counted 904 million visits to it in January–April 2026.
0.0019%
Grok's peak share of total website traffic across 101,574 sites (August 2025)
904M
visits to Grok in January–April 2026, with effectively zero attributable referrals in a B2B GA4 panel
Part of that is how grok.com links out. Its web client, as shipped on 21 September 2026, renders citation pills and source cards with rel="noopener noreferrer nofollow" and adds no UTM tags, so the browser sends no referrer and those visits land in Direct. Links without that attribute inherit the site’s Referrer-Policy: origin-when-cross-origin and arrive as grok.com. GA4’s AI Assistant channel, added in May 2026, names Grok — Google’s channel documentation lists “ChatGPT, Gemini, Deepseek, Copilot, or Grok” — but it can only classify visits that carry a referrer. How clicks from Grok inside the X app are attributed isn’t documented.
# Session source — Grok's web client, when a referrer survives^(.*\.)?grok\.com$ # X itself: t.co is the link wrapper on X posts^(t\.co|x\.com)$Server logs are the better instrument, with one catch: you can’t filter Grok by user agent. Two things work instead. Look for its signature — bursts of requests for one URL within a second, browser user agents, no referrer — and run a controlled test: publish a URL nobody knows, ask Grok to read it, and see exactly what arrives.
# Bursts: 10+ no-referrer requests for one URL in one second (combined log format)awk '$11=="\"-\"" {print substr($4,2,20), $7}' access.log \ | sort | uniq -c | awk '$1 >= 10' | sort -rn | head -20 # Controlled test: ask Grok to summarize https://yoursite.com/grok-test-7f3a, thengrep "/grok-test-7f3a" access.logFor visibility itself, run a fixed prompt panel through Grok on a schedule. Mix current-events prompts, where X dominates, with product and how-to prompts, where the web does, and record which domains and @handles get cited. Answers vary between runs, so compare across repeats rather than trusting one screenshot. Rankbox’s free AI visibility prompt kit gives you a starting set.
Myths worth dropping#
Myth
Allowing GrokBot in robots.txt gets you into Grok.
Reality
xAI documents no such token, and a February 2026 test never saw it in a request. Grok’s fetches arrive as ordinary browsers, so what matters is that your default rules and WAF let them through.
Myth
A CDN's block-AI-bots switch keeps Grok out.
Reality
Those switches match declared bots, and Grok’s fetcher “does not self-identify at all” (Cloudflare). Only behavioral rules — challenges, datacenter blocks, rate limits — catch it, and they catch real users too.
Myth
Grok mostly cites X posts.
Reality
It depends on the question. Ahrefs’ June 2026 data put x.com at 1.4% of top-50 citations, behind Reddit, YouTube and Facebook; Presenc AI’s sample put X posts near 45%, heaviest on current events.
Myth
Grokipedia is the way into Grok.
Reality
xAI doesn’t document Grok using it, and it isn’t among the 50 domains Grok cites most. Its documented citations are in ChatGPT and other assistants.
Myth
Grok runs JavaScript, so a client-rendered site is fine.
Reality
In a June 2026 test one Grok node ran the script, yet the answer quoted the raw HTML. Treat Grok like every other engine and server-render.
Myth
GA4's AI Assistant channel shows your Grok traffic.
Reality
Only visits that carry a referrer. grok.com’s citation links use noreferrer, so those clicks arrive as Direct.
The action checklist#
Everything above, in the order we’d do it. Tick items off as you go — your progress is saved in this browser.
0 of 15 done
Grok SEO: frequently asked questions#
How do I get my website cited by Grok?
Serve key pages as server-rendered HTML without bot challenges, earn mentions on the platforms Grok cites most — Reddit, YouTube, review sites — and run an X account that states your facts in text and links to your pages. There’s no crawler to allow by name.
What search engine does Grok use?
xAI doesn’t say. Its documentation describes a web search tool that searches and browses pages and an X search tool for posts, accounts and threads, but names no provider, index or ranking method.
What is Grok's user agent?
None is documented. GrokBot, xAI-Grok and Grok-DeepSearch circulate on third-party lists but aren’t in xAI’s docs, and a February 2026 test saw only browser user agents such as Chrome/139.0.0.0 plus Go-http-client/1.1, from datacenter IPs. Cloudflare Radar lists GrokAgent for Grok Bot, xAI’s agent product.
Can I block Grok with robots.txt?
Not reliably. There’s no documented token, and xAI hasn’t said its fetcher reads robots.txt. Cloudflare says Grok’s fetches don’t self-identify, so only behavioral WAF rules catch them. For training, xAI offers websites no documented opt-out.
How much do X posts matter for Grok?
A lot for current events, less for product and how-to questions. Presenc AI found about 45% of Grok’s citations were X posts in a March–April 2026 sample; Ahrefs’ June 2026 data put x.com at 1.4% of top-50 citations across broad US queries.
Can Grok read JavaScript-rendered pages?
xAI doesn’t document it. In Search Engine World’s June 2026 test one Grok node executed the JavaScript, but the answer still came from the raw HTML. Server-render anything you want cited.
Does Grok use Grokipedia?
xAI hasn’t documented Grok citing Grokipedia, and grokipedia.com isn’t among Grok’s 50 most-cited domains in Ahrefs’ June 2026 data. ChatGPT has been found citing it, so check your brand’s page and suggest corrections.
How do I track Grok traffic in GA4?
GA4’s AI Assistant channel includes Grok, but only visits with a referrer get classified, and grok.com’s citation links use noreferrer. Add a custom channel for grok.com, watch Direct on pages Grok cites, and use server logs plus a controlled fetch test.
Can I pay to appear in Grok's answers?
Not as of SpaceX’s May 2026 S-1, which called ads in the stand-alone Grok product a possible future opportunity and said advertisers can’t buy placement on the Grok API. Elon Musk said in August 2025 that ads in Grok’s responses were planned.
Sources
- 1.Web SearchSpaceXAI Docs · docs.x.ai ↗
- 2.X SearchSpaceXAI Docs · docs.x.ai ↗
- 3.CitationsSpaceXAI Docs · docs.x.ai ↗
- 4.Grok 4 FastxAI · x.ai ↗
- 5.Grok 4 model cardxAI · data.x.ai ↗
- 6.Grok prompts: system prompts for Grok chat and the @grok bot on XxAI on GitHub · github.com ↗
- 7.Grok release notesgrok.com · grok.com ↗
- 8.Space Exploration Technologies Corp. Form S-1U.S. SEC · sec.gov ↗
- 9.To build a better Internet in the age of AI, we need responsible AI bot principlesCloudflare · blog.cloudflare.com ↗
- 10.Grok bot user agent: why you can't block xAI's crawlerStackFox · stackfox.co ↗
- 11.Do AI assistants actually render your JavaScript when grounding? We put it to the testSearch Engine World · searchengineworld.com ↗
- 12.The 50 most-cited websites in Grok (June 2026)Ahrefs · ahrefs.com ↗
- 13.Grok citation patterns 2026: how xAI Grok selects sourcesPresenc AI · presenc.ai ↗
- 14.Wikipedia vs GrokipediaAhrefs · ahrefs.com ↗
- 15.GrokipediaGrokipedia · grokipedia.com ↗
- 16.ChatGPT is pulling answers from Elon Musk's GrokipediaTechCrunch · techcrunch.com ↗
- 17.Analysis of top AI search engines: who is catching up to ChatGPT?SE Ranking · seranking.com ↗
- 18.ChatGPT's AI referral share fell from 89% to 63%Goodie · higoodie.com ↗
- 19.Default channel group (AI Assistant)Google Analytics Help · support.google.com ↗
- 20.X Privacy PolicyX · x.com ↗
