On this page12 sections
- Key takeaways
- One answer, two indexes: the web and X
- The X half: posts written for four lookups
- The web half: an undisclosed index that leans on communities
- The fetcher with no name: robots.txt, WAFs and bursts
- Why Grok's clicks show up as Direct
- Grokipedia: Grok's encyclopedia, cited mostly by other AIs
- Who runs Grok in 2026, and whether you can buy a spot
- Grok folklore, checked
- 14 moves, ranked by impact
- FAQ
- Sources
Key takeaways
- 01
Grok has two search tools: Web Search, which searches and browses pages, and X Search, which runs keyword, semantic, account and thread lookups. grok.com shows them as “Searching the web” and “Searching 𝕏” and cites both inline.
- 02
Which half wins depends on the question. Ahrefs’ June 2026 data put x.com at 1.4% of Grok’s top-50 citations; a vendor panel found X posts in about 45% of citations, weighted toward current events.
- 03
xAI documents no crawler, fetcher or robots.txt token. Tests saw Chrome-like user agents from datacenter IPs, so your WAF, not robots.txt, decides whether Grok can read a page.
- 04
grok.com’s citation chips carry
rel="noopener noreferrer nofollow"(checked 26 October 2026), so most clicks land in Direct, out of reach of GA4’s AI Assistant channel. - 05
Grok 4.7’s training data stops at May 2026. Anything newer reaches an answer only through search — which is where X, with its date filters, earns its place.
- 06
Grokipedia is live (v0.3 shipped 30 September 2026), but its documented citations come mostly from other assistants, not Grok.
One answer, two indexes: the web and X
Grok answers from its training when it can and searches when a question needs something newer. Its current flagship, Grok 4.7, has a knowledge cutoff of May 2026, per xAI’s models page, so a price change, launch or incident after that reaches an answer only through search. SpaceX’s May 2026 S-1 describes Grok Chat as offering “real-time integration of web search, X data, code execution, and multimodal analysis” on grok.com, the Grok apps, inside X and through the API.
xAI’s API exposes the same two search tools, and their documentation is the clearest public account of how Grok gathers sources. The model works in a loop: at each step it “decides what to do next” — call a tool or answer — and “continues until sufficient information is gathered,” per the tools overview. Nothing stops it calling both tools for one question, and the side-by-side below is what each one does.
| Web Search | X Search | |
|---|---|---|
| Reads | Web pages: it can “search the web in real-time and browse web pages” | Posts, accounts and threads on X |
| Lookups | Search, then open the pages it needs | x_keyword_search, x_semantic_search, x_user_search, x_thread_fetch |
| API filters | allowed_domains or excluded_domains, up to 5 | allowed_x_handles or excluded_x_handles, up to 20; from_date and to_date |
| Media | Optional image understanding and image search | Optional image and video understanding |
| On grok.com | “Searching the web”, then chips naming the site | “Searching 𝕏” or “Reading thread”, then chips that link the posts |
| Cited as | The page URL | x.com/<handle>/status/<id> |
| You control | Crawlable HTML, no challenge page, titles that read well on a source card | Handle, bio, post text, links and threads |
On grok.com both kinds of citation sit inline in the answer. The web client, as served on 26 October 2026, types every citation as web or x_post, labels each web chip with the site’s name taken from its domain — reddit.com shows as Reddit — and folds several sources behind one chip as “+N”. Through the API, the citations field lists every URL the agent encountered, and inline citations mark the ones it used; xAI’s citations docs warn that “not every URL in this list will necessarily be directly referenced in the final answer.”
117M
monthly active users used Grok's AI features at 31 March 2026, out of about 550M across Grok and X
SpaceX S-1, May 20261.4%
of citations among Grok's 50 most-cited domains went to x.com, 12th place, across 1.9M US queries
Ahrefs, Jun 2026~45%
of Grok's citations were X posts across 4,400 prompts run in March and April 2026
Presenc AI, May 2026
The two studies measure different things. Ahrefs’ Brand Radar counted every domain Grok cited across 1.9 million broad US queries. Presenc AI, which sells Grok monitoring, ran 4,400 prompts and classified each citation as an X post or a web page; it reports that the mix “shifts toward X heavily for current-events queries and toward web for definitional or how-to queries.” Read together: a question about what happened this week leans on X, and a question about what something is or how to do it leans on the web. Work out which kind your buyers ask, and weight the next two sections accordingly.
The X half: posts written for four lookups
X Search is four lookups, and each rewards something specific about how a brand posts. None of this is a ranking secret: it’s the capabilities in xAI’s X Search docs, mapped to what an account owner can change.
| Lookup | What it finds | What to post |
|---|---|---|
x_keyword_search | Posts that contain the query’s words | The fact itself in the text — product name, number, date — not only inside an image or a link card |
x_semantic_search | Posts that mean the same thing in other words | Complete sentences that answer the question a person would type into Grok |
x_user_search | Accounts, by name | The brand name in the display name or handle, and a bio that says what you do |
x_thread_fetch | A whole conversation in one call | One thread that settles a recurring question from start to finish |
Two more documented behaviours shape what gets picked. Grok’s published chat prompt says it “can analyze individual X user profiles, X posts and their links,” so a post that links your canonical page is also a route to the page. And the from_date and to_date filters let the agent restrict X results to a window, which suits the current-events questions X is cited for most: post on the day a fact changes, not a week later.
- One fact, one post. “Pro is now $19 a month, from today” can be matched by keyword and by meaning; a launch-day meme can’t.
- Link the page that proves it. The post gets the speed; the page holds the detail Grok’s web search can open and quote.
- Keep one thread per question you get every week. Pricing, availability, migration steps — one thread, kept current, is a ready-made source for
x_thread_fetch. - Keep the handle and bio boring. Searchable beats clever when a lookup runs by account name.
- Correct in public. If @grok answers wrongly about you under a post, reply with the primary source. The next search has something to find.
2.8x
the citation rate of posts from verified accounts versus equally engaged unverified posts
Presenc AI, May 20264.7
sources cited per Grok answer on average, rising to 6.3 for current-events prompts
Presenc AI, May 2026
When Grok answers in public: @grok and Explain
Grok also answers inside X itself: people tag @grok under a post, and the Explain button summarises one. xAI’s published system prompts — last updated in November 2025, so they may lag the current models — tell @grok to use search “to confirm facts and fetch primary sources for current events,” that “parallel search should be used to find diverse viewpoints,” and that it “must use the browse page to verify all points of information you get from search.” Replies are capped under 550 characters with no markdown. Explain is told to “prioritize peer-reviewed research.” Both are public posts in the thread, so a wrong answer about your brand is visible to everyone reading it.
The web half: an undisclosed index that leans on communities
xAI names no search provider for Grok’s web results, publishes no index size and offers no webmaster console or URL submission. It isn’t on IndexNow’s list of participating engines either. What is visible is the output: Ahrefs’ June 2026 ranking of the 50 domains Grok cites most, built from Brand Radar data, is led by user-generated platforms.
| Rank | Domain | Share of top-50 citations |
|---|---|---|
| 1 | reddit.com | 16.3% |
| 2 | youtube.com | 15.1% |
| 3 | facebook.com | 13.9% |
| 4 | instagram.com | 5.9% |
| 5 | quora.com | 5.5% |
| 6 | amazon.com | 5.0% |
| 7 | tiktok.com | 4.8% |
| 8 | en.wikipedia.org | 3.4% |
| 9 | ebay.com | 1.5% |
| 10 | walmart.com | 1.5% |
| 11 | pinterest.com | 1.4% |
| 12 | x.com | 1.4% |
Community platforms carry most of it
ObservedReddit, YouTube, Facebook, Instagram, Quora and TikTok together take about 61% of the top-50 citation share. Being discussed there is a route that doesn’t depend on your own pages ranking anywhere — see Reddit SEO.
Product questions go to listings
ObservedAmazon (5.0%), eBay and Walmart (1.5% each) all make the top ten. For anything people buy, a complete, accurate marketplace listing is a web page Grok already reads.
News goes to primary sources
Official@grok’s prompt tells it to “fetch primary sources for current events” and to verify search results by opening pages. For news about your company, your own dated announcement page is the primary source.
Contested topics get a spread
OfficialOn a controversial query, Grok’s chat prompt says to “search for a distribution of sources that represents all parties/stakeholders” and to “assume subjective viewpoints sourced from media are biased.” Expect to be cited next to critics; evidence holds up better than adjectives.
There's no paid route yet
OfficialSpaceX’s S-1 says it does “not currently sell or offer advertisers the ability to place ads on the Grok API,” and calls ads in the stand-alone Grok product a possible “incremental monetization opportunity.”
Your own rankings don't transfer by default
Our readWith no known index behind the web tool, a strong Google position is not a documented path into Grok. The pages that get cited are the ones its searches surface and its fetcher can open.
So the web half is two jobs: make your own pages openable by a fetcher that doesn’t announce itself (next section), and be part of the conversations on the platforms Grok already cites. Rankbox’s Reddit Presence finds the threads worth joining and drafts replies posted under your own name.
The fetcher with no name: robots.txt, WAFs and bursts
OpenAI, Anthropic, Google and Perplexity publish their AI crawlers. xAI publishes no crawler page, user-agent token, IP list or robots.txt statement for Grok’s search or its live page fetches. Cloudflare wrote in September 2025 that “xAI’s bot, grok, does not self-identify at all, making it impossible for website operators to block it.” A controlled test by StackFox on 6 February 2026 found the same: asked to read a news site, 30 requests hit the server within a second — 19 from one datacenter IP, 10 from another in a third of a second — all claiming to be an ordinary browser, desktop Chrome on both main IPs. “The word ‘Grok’ never appears. Neither does ‘xAI.’”
| User agent | Where it comes from | Status |
|---|---|---|
GrokBot, xAI-Grok, Grok-DeepSearch | Third-party bot directories | Not in xAI’s docs; StackFox’s test never saw them |
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36 | StackFox controlled test, February 2026 | Seen from both main IPs in the test, on M247 and Datacamp datacenter ranges |
Go-http-client/1.1, iPhone Safari strings | StackFox, general Grok traffic | Reported, but not from the controlled test |
The consequence: robots.txt can’t target Grok, and neither can a CDN’s “block AI bots” switch, because nothing in the request says it’s Grok. Nothing documents whether Grok’s fetcher reads robots.txt at all. Naming the unverified tokens does no harm, but don’t mistake it for control — your default group is what applies:
# xAI documents no user agent for Grok's search or page fetches.
# These names come from third-party lists and are unverified:
# a rule for them may never match a real request.
User-agent: GrokBot
User-agent: xAI-Grok
User-agent: Grok-DeepSearch
Allow: /
# Grok's browser-like fetches fall under the default group
User-agent: *
Allow: /
Disallow: /account/- Server-rendered HTMLRequired
- In Search Engine World’s June 2026 test, one node of Grok’s fetch pipeline ran the page’s script, but the answer still quoted the raw-HTML decoy: “The machinery ran the script; the answer ignored what it found.” Put the content in the HTML response — see server-side rendering.
- No challenge on content pagesRequired
- A JavaScript challenge, CAPTCHA or datacenter block hands Grok’s fetcher a page with none of your content on it.
- Rate limits that allow burstsHelps
- StackFox logged 19 requests in about a second from one IP. Per-second limits tuned for people will cut some of them off.
- Titles that work on a source cardHelps
- grok.com’s chips show only the site’s name; the card behind each one shows the page title and a short preview. The title is what a reader weighs before clicking.
- Links from your X postsHelps
- Grok can analyse the links in the posts it reads, so a post that links the canonical URL is a second path to the page.
- Structured dataUnconfirmed
- xAI says nothing about schema.org. Keep every fact that matters in visible text as well.
- llms.txtUnconfirmed
- xAI hasn’t said Grok reads llms.txt, and no published test has shown it requested.
- robots.txt rules naming GrokNo effect
- There’s no documented token to name. A rule for
GrokBotorxAI-Grokmay never match a request. - IndexNow pingsNo effect
- xAI isn’t a participating engine, so pings never reach Grok.
Test pages the way Grok’s fetcher appears to see them — the exact user agent StackFox logged, no JavaScript — ideally from a cloud VM, since that’s where its requests come from:
# The user agent StackFox logged from Grok, and no JavaScript
curl -s -A "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/139.0.0.0 Safari/537.36" \
https://yoursite.com/pricing | grep -c "Pro is \$19"
# 0 = the text needs JavaScript, or your WAF served a challenge pageTraining: no website opt-out, one X setting
The Grok 4 model card says Grok 4 is pre-trained on a recipe that “includes publicly available Internet data.” With no documented crawler token, a website has no robots.txt line xAI says it honours for training. X posts are different: X’s privacy policy, effective 15 January 2026, says X may use what it collects “and publicly available information to help train our machine learning or artificial intelligence models.” Account holders can switch that off under Settings and privacy → Privacy and safety → Grok & Third-party Collaborators, per Roboin’s 2026 walkthrough. As far as anything documents, that setting governs training, not whether X Search can find and cite your public posts.
Why Grok's clicks show up as Direct
Expect little Grok referral traffic in analytics, and know that some of the shortfall is measurement, not absence. Two facts about grok.com, both checked on 26 October 2026, explain most of it.
| Where the click happens | What reaches your site | Basis |
|---|---|---|
| A citation chip or source card on grok.com | No referrer, so the visit lands in Direct | grok.com’s web client renders these links with rel="noopener noreferrer nofollow" and adds no UTM tags |
| Any other grok.com link without that attribute | https://grok.com/ — the origin only | grok.com sends Referrer-Policy: origin-when-cross-origin |
| Grok inside x.com or the X app | Undocumented | No published test; check your own logs |
| The Grok iOS and Android apps | Undocumented | App link-outs often carry no referrer at all, so assume Direct until you see otherwise |
0.0019%
Grok's peak share of total website traffic across 101,574 sites, in August 2025, before falling back to near zero
SE Ranking, Jun 2026904M
visits to Grok in January–April 2026, against "effectively zero attributable referrals" in a B2B GA4 panel
Goodie, May 2026
GA4’s AI Assistant channel won’t rescue the Direct visits. Google’s channel definitions describe it as traffic “from sources like ChatGPT, Gemini, Deepseek, Copilot, or Grok,” and classify a session there when the medium is ai-assistant — which GA4 sets only “if the referrer matches a list of AI Assistants.” No referrer, no match. Catch the visits that do carry one with a custom channel, and treat a Direct spike on a page Grok cites as a lead worth checking:
# Session source: grok.com, when a referrer survives
^(.*\.)?grok\.com$
# X itself: t.co wraps links in X posts
^(t\.co|x\.com)$Server logs are the better instrument, with a catch: you can’t filter Grok by user agent. Two methods work instead. Look for its signature — a burst of requests for one URL within a second, a desktop Chrome user agent, a datacenter IP, no referrer — and run a controlled test: publish a URL nobody knows, ask Grok to read it, and see exactly what arrives. Rankbox’s free AI crawler log analyzer handles the named bots; the bursts below are the part only a query like this finds.
# Bursts: 10+ no-referrer requests for one URL in one second (combined log format)
awk '$11=="\"-\"" {print substr($4,2,20), $7}' access.log \
| sort | uniq -c | awk '$1 >= 10' | sort -rn | head -20
# Controlled test: ask Grok to summarise https://yoursite.com/grok-test-7f3a, then
grep "/grok-test-7f3a" access.logFor visibility itself, run a fixed prompt panel through Grok on a schedule. Mix current-events prompts, where X dominates, with how-to and product prompts, where the web does, and record which domains and @handles get cited. Answers vary between runs, so compare across repeats rather than trusting one screenshot. The free AI visibility prompt generator gives you a starting set.
Grokipedia: Grok's encyclopedia, cited mostly by other AIs
Grokipedia launched on 27 October 2025 with about 885,000 machine-generated articles, and its homepage now lists more than six million. Nobody edits it directly: readers submit suggestions through a form, and Grok is meant to review each one. That review stalled for months — Lawfare found no article updated after 24 April 2026 and 13,002 suggestions stuck “in review” as of August. Grokipedia v0.3 shipped on 30 September 2026 with a redesign; whether the backlog has cleared isn’t documented.
Its documented reach is mostly outside Grok. Ahrefs’ March 2026 comparison counted 356,200 AI citations of Grokipedia against 24.9 million for Wikipedia, with the most coming from ChatGPT and Google’s AI Mode, then Copilot and Gemini. grokipedia.com isn’t among the 50 domains Grok cites most in Ahrefs’ June 2026 data, and xAI doesn’t document Grok using it in answers.
- Search Grokipedia for your brand, products and founders. Generated pages can exist without anyone asking for them.
- Suggest corrections with a primary source attached — your documentation, a filing or a dated announcement — then check back on the suggestion’s status rather than assuming it was applied.
- Don’t build a Grok strategy on it. Its citations show up in ChatGPT and Google’s AI answers far more than in Grok’s.
Who runs Grok in 2026, and whether you can buy a spot
Grok was built by xAI, which SpaceX acquired effective 2 February 2026, according to its S-1. The AI business has carried SpaceX branding since: SpaceXAI, and then SpaceX Super Intelligence. Musk said on 4 October 2026 that SpaceXAI would be renamed SpaceXSI, Reuters reported, and the company’s X account became @SpaceXSI around 10 October. The developer docs still live at docs.x.ai under the xAI API name, and some pages already credit “SpaceXSI.” None of the renaming changed how Grok searches — but search for all three names when you look for documentation.
Paid placement isn’t a route. The S-1 says SpaceX does “not currently sell or offer advertisers the ability to place ads on the Grok API,” and describes ads in the stand-alone Grok product only as something there “might be” an opportunity for. Until that changes, a citation in Grok is earned through the two halves above.
Grok folklore, checked
01Heard
Allowing GrokBot in robots.txt gets you into Grok.
Checked
xAI documents no such token, and a February 2026 controlled test never saw it in a request. Grok’s fetches arrive as desktop Chrome, so what matters is that your default rules and your WAF let them through.
02Heard
A CDN's block-AI-bots switch keeps Grok out.
Checked
Those switches match declared bots, and Grok’s fetcher “does not self-identify at all” (Cloudflare). Only behavioural rules — challenges, datacenter blocks, rate limits — catch it, and they catch real people too.
03Heard
Grok mostly cites X posts.
Checked
It depends on the question. Ahrefs’ June 2026 data put x.com at 1.4% of top-50 citations, behind Reddit, YouTube and Facebook; Presenc AI’s panel put X posts near 45%, heaviest on current events.
04Heard
Ranking on Google means ranking in Grok.
Checked
xAI names no index provider for its web search. Google rankings may help indirectly by making you easy to find and discuss, but there’s no documented link.
05Heard
GA4's AI Assistant channel shows your Grok traffic.
Checked
Only visits that carry a referrer. grok.com’s citation chips use
noreferrer, so those clicks arrive as Direct.06Heard
Grokipedia is the way into Grok.
Checked
xAI doesn’t document Grok using it, and it isn’t among the 50 domains Grok cites most. Its documented citations are in ChatGPT and Google’s AI answers.
14 moves, ranked by impact
0 of 14 done · saved in this browser
High impact
Medium impact
Worth doing
