On this page12 sections
- In brief
- One prompt, many searches
- Read isn’t cited
- OAI-SearchBot, GPTBot, ChatGPT-User
- What OAI-SearchBot actually receives from your page
- From citation pill to pageview
- Product answers run on feeds, not on your product pages
- Agents after Atlas
- ChatGPT beliefs the 2026 data broke
- The fixes, in order
- Questions
- Sources
In brief
- 01
OAI-SearchBotis the only OpenAI crawler that decides search inclusion.GPTBotis training-only: block it and you stay in search. - 02On 8 August 2026 pages consulted per answer roughly doubled (11.6 to 22.5 in Searchable’s 266,182-prompt test), and
site:searches of brand domains jumped. - 03How far it fans out depends on the prompt: 7.61 searches per commercial prompt in Nectiv’s API test, but fan-outs in only 14.4% of 66.6M logged-out answers in Ahrefs’ data.
- 04Reading isn’t citing. Ahrefs found ChatGPT cites about half the URLs it retrieves, and retrieved Reddit pages just 1.93% of the time.
- 05Product answers come from feeds: Shopify Catalog, or a direct Agentic Commerce Protocol feed for approved merchants.
- 06ChatGPT visits land in GA4’s AI Assistant channel (since 13 May 2026) with source
chatgpt.com. Theutm_source=chatgpt.comtag is still seen on clicks, but OpenAI no longer documents it.
01
One prompt, many searches: how ChatGPT rewrites the question#
ChatGPT decides prompt by prompt whether to search the web, and users can force a search from the tools menu. When it searches, OpenAI says it “typically rewrites your query into one or more targeted queries” and, after reviewing the first results, “may send additional, more specific queries” (OpenAI Help). OpenAI’s own example turns a researcher’s question about drugs that target CCR8 into CCR8 immunotherapy drug development 2025, then follows up with CHS-114 conference 2025. Saved memories and approximate location can shape the rewrite.
Not every prompt searches. Semrush’s US clickstream panel put web search at 34.5% of ChatGPT prompts in February 2026, down from 46% in late 2024. Everything below is about that searched share, which skews toward current, commercial and local questions.
These rewritten searches are called query fan-out. For a pricing question about a product, they now tend to name the brand’s own domain, the current year and the word “official”. For the sample prompt at the top of this page, they would look something like this:
- 01
site:plotbright.com pricing - 02
Plotbright Team plan price per user 2026 - 03
Plotbright free plan limits official
What changed on 8 August 2026
ChatGPT’s retrieval changed overnight on 8 August 2026, as GPT-5.6 became the default model for Free and Go users. Four independent datasets agree on the direction and differ on the size:
| Study | Sample | What it measured |
|---|---|---|
| Searchable, Aug 2026 | 266,182 paired prompts, before and after 8 August | Pages consulted per answer 11.58 → 22.54. Inline citations per answer 0.49 → 0.82. site: share of sub-queries 0.2% → 18%. |
| Peec AI, Aug 2026 | GPT-5.5 vs GPT-5.6 (sample size not published) | Sources retrieved per prompt ~12 → ~24. Prompts needing a single fan-out 94.0% → 43.5%. site: in 23% of fan-outs, up from 0.3%. |
| Nectiv, Aug 2026 | About 4,000 commercial prompts via the API, rerun from 2025 | 7.61 fan-out queries per prompt, up from 2.17. site: in 64% of fan-outs. “Official” the second most common word. |
| Ahrefs, Oct 2026 | 66.6M free, logged-out answers, Aug–Sep 2026 | Fan-outs in 14.4% of answers, about 1.6 each. site: in 6.8% of fan-outs; 75.9% of the most-searched targets were brand sites. |
The spread comes from the prompts, not from error. Nectiv used commercial “best X” prompts on a reasoning model, which trigger brand lookups; Ahrefs sampled broad prompts in free, logged-out sessions, which search less. For a buying question in your category, plan for the Nectiv end of the range. For informational questions, expect the Ahrefs end.
02
Read isn’t cited: why most pages ChatGPT opens never get the pill#
Behind every searched answer are two lists: the pages ChatGPT consulted and the few it cites. Ahrefs’ study of 1.4 million prompts found it cites only about half of what it retrieves, 16.57 cited URLs against 16.58 uncited per prompt (Ahrefs, Apr 2026). Searchable’s August data shows the gap inside the answer text: 22.5 pages consulted, 0.82 inline citations. Getting read is the easy part.
Titles that match a rewritten query
ObservedAhrefs compared each cited page’s title with every fan-out query and kept the closest match: 0.656 cosine similarity. Against the original prompt, cited titles scored 0.602 and uncited ones 0.484. Title pages for the query ChatGPT writes, not only for the prompt.
Readable URL slugs
ObservedRetrieved URLs with natural-language slugs were cited 89.78% of the time, against 81.11% for those without (Ahrefs, Apr 2026).
The answer in the first third
ObservedKevin Indig’s analysis of 1.2 million ChatGPT results found about 44% of citations came from the first third of a page and 24.7% from the last. Definitional “X is…” sentences were about twice as likely to be cited (Growth Memo, Feb 2026).
Your own domain, since August
ObservedAfter 8 August, brand-owned pages’ citation share rose 1.80× while their reading share barely moved (0.95×), and 85.9% of 1,665 brands were cited on their own domain more often (Searchable).
Sections with substance
ObservedSE Ranking’s 129,000-domain study found sections under 50 words averaged 2.7 citations, 120–180 words 4.6, and over 180 words 5.7 (Nov 2025).
Fresh pages
ObservedPages updated in the past three months averaged 6.0 citations against 3.6 (SE Ranking), and ChatGPT’s citations averaged 458 days newer than Google’s organic results across 17 million citations (Ahrefs, Jul 2025).
Being talked about
ObservedAcross 75,000 brands, branded web mentions correlated with ChatGPT visibility at 0.664, domain rating at just 0.266 (Ahrefs, Dec 2025). Mentions beat link metrics.
Reddit: read often, cited rarely
ObservedReddit has its own retrieval type in ChatGPT, yet retrieved Reddit pages were cited 1.93% of the time (Ahrefs, Apr 2026). Its citation share then fell 86% after 8 August (Promptwatch, via Semrush).
03
OAI-SearchBot, GPTBot, ChatGPT-User: three switches, not one#
OpenAI’s crawler documentation is explicit: “Each setting is independent of the others.” The one that decides search inclusion is OAI-SearchBot. Sites that block it stay out of ChatGPT’s search answers, though they can still appear as “navigational links”. OpenAI’s help center adds that eligibility also means letting its published IP ranges through your host or CDN, and that “placement is not guaranteed.”
- Allow
OAI-SearchBotSearch inclusion
Crawls pages to surface them in ChatGPT search. The user agent ends
compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot; IPs are inopenai.com/searchbot.json. Robots changes take about 24 hours to reach search.Reads robots.txt.
- Allow
ChatGPT-UserLive fetch for a user
Visits a page when a user’s question, a custom GPT or a GPT Action needs it. Not used for automatic crawling or to decide search inclusion. IPs are in
openai.com/chatgpt-user.json.Robots.txt not guaranteed. OpenAI: because users start these actions, robots.txt rules may not apply.
- Your call
GPTBotModel training
Crawls content that may be used to train OpenAI’s foundation models. Disallowing it signals your content shouldn’t be used for training and has no effect on search. IPs are in
openai.com/gptbot.json.Reads robots.txt.
- Your call
OAI-AdsBotAd review
Visits only landing pages submitted as ChatGPT ads, to check safety and policy compliance and judge relevance. Not used for training. IPs are in
openai.com/adsbot.json.Reads robots.txt. OpenAI's advertiser guidance says advertisers must allow it.
A robots.txt that stays in ChatGPT search, opts out of training, and lets ad review through if you advertise:
# ChatGPT search: required to be citedUser-agent: OAI-SearchBotAllow: /# Model training: your call, independent of searchUser-agent: GPTBotDisallow: /# Only if you run ChatGPT adsUser-agent: OAI-AdsBotAllow: /- One crawl can serve two bots. If you allow both
OAI-SearchBotandGPTBot, OpenAI says it may use the results of just one crawl for both, to avoid duplicate crawling. - Blocking isn’t hiding. A Disallow keeps pages out of answers, not out of navigational links. OpenAI doesn’t document whether its crawlers honor
noindex, so don’t rely on it alone. - Your WAF outranks robots.txt. Bot-fight modes and “block AI” toggles stop
OAI-SearchBotbefore it reads a single rule. Allowlist the IPs in OpenAI’s JSON files and serve them no challenges.
04
What OAI-SearchBot actually receives from your page#
Access is the only inclusion rule OpenAI documents. The rest of this spec sheet comes from Microsoft’s role as a search partner and from correlation studies, graded by how sure anyone can be.
- OAI-SearchBot allowed, its IPs unchallengedRequired
- In robots.txt and at the CDN or WAF, per OpenAI’s help center. The one rule OpenAI states.
- Facts in the initial HTMLRequired
- OpenAI documents no JavaScript rendering, and Vercel’s crawler study saw OpenAI’s bots (
OAI-SearchBot,ChatGPT-User,GPTBot) fetch JavaScript files without executing them (Dec 2024). Treat client-rendered text as missing. - Indexed by BingHelps
- OpenAI links Microsoft’s privacy statement as a search provider. Verify the site in Bing Webmaster Tools and submit sitemaps. Bing’s AI Performance report covers Copilot and Bing, not ChatGPT.
- IndexNowHelps
- Microsoft recommends it to keep content fresh “across search and AI experiences.” Ping on every publish and substantive update.
- Fast first paintHelps
- SE Ranking: pages with first contentful paint under 0.4s averaged 6.7 citations, pages over 1.13s just 2.1. Correlation, but live fetches don’t wait forever.
- Visible, honest datesHelps
- Fan-outs carry the year, and recently updated pages earn more citations. Change the content when you change the date.
- Sitemaps and canonicalsUnconfirmed
- OpenAI doesn’t say whether
OAI-SearchBotreads sitemaps or honorsrel=canonical. Keep both correct for Bing. - noindexUnconfirmed
- Not documented for OpenAI’s crawlers. Use it, but don’t count on it alone to keep a page out of ChatGPT.
- FAQ schemaNo effect
- SE Ranking found pages with FAQ schema averaged 3.6 citations against 4.2 without.
- llms.txtNo effect
- OpenAI has never said ChatGPT reads it, and SE Ranking’s model got more accurate when the variable was removed.
To check what the crawler gets, request the page with its user agent and count the sentence you want cited in the raw HTML:
curl -s -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot" \ https://yoursite.com/pricing | grep -c "per user a month"# 0 = the sentence is rendered client-side, or your WAF served a challengeA WAF that verifies OpenAI’s IP ranges may answer your spoofed request differently from the real crawler, so confirm with real OAI-SearchBot hits in your logs too.
05
From citation pill to pageview: reading ChatGPT in your analytics#
In the answer, cited pages appear as inline citations that open the source (on desktop, hovering previews it) and in a Sources button that lists cited sources and other relevant links (OpenAI Help). Since 7 May 2026 ChatGPT also links brand names inline far more often: Profound saw the share of responses containing a URL jump from about 4.5% to 20–24% in a day, and OpenAI referrals to the brands it monitors rise 60–65%, with homepages taking a much bigger share (Profound).
What a ChatGPT click looks like on arrival
- Landing URL
- https://plotbright.com/pricing?utm_source=chatgpt.com
- Referrer
- https://chatgpt.com/
- GA4 source
- chatgpt.com
- GA4 channel
- AI Assistant
- The tag is undocumented now. OpenAI used to state in a publisher FAQ that ChatGPT adds
utm_source=chatgpt.comto its links. That article has since been rewritten as the ChatGPT Atlas retirement notice, so treat the tag as observed behaviour that can change, and build reports on the referrer too. - GA4 groups it for you. On 13 May 2026 Google Analytics added an AI Assistant default channel: visits with a recognised AI-assistant referrer get the medium
ai-assistantand the campaign(ai-assistant)(Google). It covers ChatGPT, Gemini, DeepSeek, Copilot and Grok, and excludes Google’s own AI Overviews and AI Mode. - Keep a backstop. A custom channel on the
chatgpt.comsource catches visits the default rules miss, such as an app that strips the referrer while the tag survives.
# Custom channel group: "ChatGPT", placed above Referral# Condition: Session source matches regex^(chatgpt\.com|chat\.openai\.com)$Keep paid and feed clicks out of the organic number: tag ChatGPT ad landing pages with your own utm_medium (such as cpc), and add utm_medium=feed to product-feed URLs, as OpenAI’s commerce docs suggest. Then add the server-side view, where ChatGPT-User hits are pages being fetched live for a conversation:
# OpenAI hits by agentgrep -oE "OAI-SearchBot|ChatGPT-User|GPTBot|OAI-AdsBot" access.log | sort | uniq -c# Pages fetched live for ChatGPT conversations, most firstgrep "ChatGPT-User" access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20User agents are easy to fake, so check hits against OpenAI’s IP files. OpenAI offers publishers no citation dashboard, so the citations themselves need a fixed panel of buyer prompts, run on a schedule, recording which domains appear in the Sources list.
06
Product answers run on feeds, not on your product pages#
When a question is about buying something, ChatGPT can show a product carousel, and selecting a product can list the merchants that sell it. OpenAI says product results “are not ads” and come from structured metadata from first- and third-party providers, and that merchants are ranked on availability, price, quality and whether they’re the maker or primary seller (OpenAI Help).
- 1Shopify stores: nothing to do. Product data flows in through Shopify Catalog.
- 2Everyone else: apply for a direct feed. OpenAI’s commerce docs say product-feed onboarding is “currently available to approved partners”. Feeds go in by API or SFTP upload, following the Agentic Commerce Protocol (ACP) specs.
- 3Checkout can stay yours. Since 24 March 2026 merchants can send buyers to their own checkout; Instant Checkout inside ChatGPT remains only for some eligible products (OpenAI).
- 4Let the crawler see your images. OpenAI’s advertiser guidance asks product-feed merchants to let
OAI-SearchBotcrawl product image URLs.
07
Agents after Atlas: cloud browser, desktop browser and WebMCP#
OpenAI’s agent surfaces changed through 2026. ChatGPT Atlas, the browser launched in October 2025, “has since been deprecated” (OpenAI). The standalone ChatGPT agent “is no longer available”, and OpenAI points users to ChatGPT Work and its cloud browser instead (OpenAI Help). What reaches your server now:
| Surface | Where it runs | How your server sees it |
|---|---|---|
| Cloud browser (ChatGPT Work, paid plans except Free and Go) | A browser on OpenAI’s computers | Requests signed with Web Bot Auth (RFC 9421), header Signature-Agent: "https://chatgpt.com" |
| Built-in browser (ChatGPT desktop app) | The user’s own Mac or Windows PC | Not documented by OpenAI |
ChatGPT-User fetches | OpenAI’s servers | Declared user agent, IPs in chatgpt-user.json |
Signature-Agent: "https://chatgpt.com"Signature-Input: sig1=(...)Signature: sig1=:...:# Public keys: https://chatgpt.com/.well-known/http-message-signatures-directoryVerify the signature rather than blocking the whole agent category. OpenAI’s allowlisting guide notes that Akamai, HUMAN and Vercel already recognise it by name. In the desktop app, websites can also offer site tools through WebMCP, which OpenAI describes as “a proposed web standard that lets websites make tools available directly to AI agents” (OpenAI Help). OpenAI publishes no developer guide for it yet, so a clean, server-rendered sign-up and checkout flow matters more today.
08
ChatGPT beliefs the 2026 data broke#
- 01
Blocking GPTBot takes you out of ChatGPT.
GPTBotis training-only. Search inclusion isOAI-SearchBot, and OpenAI keeps the two settings independent. - 02
ChatGPT just repeats Google's or Bing's top 10.
In Ahrefs’ 15,000-query test about 8% of ChatGPT’s citations ranked in either top 10 for the same prompt. An October 2026 paper found Google results and OpenAI’s search API together recovered only about a quarter of the URLs ChatGPT cited (arXiv).
- 03
Reddit threads are the shortcut into answers.
Retrieved Reddit pages were cited 1.93% of the time, and Ahrefs measured an 88% fall in Reddit citations from March to September 2026. Brand-owned pages gained.
- 04
Ads or shopping deals buy recommendations.
OpenAI: “Ads do not influence the answers ChatGPT gives you.” Ads are labelled sponsored and sit apart from the answer, and product results “are not ads” (OpenAI).
- 05
llms.txt and FAQ schema get you cited.
Neither showed a lift in SE Ranking’s ChatGPT data, and OpenAI has never said ChatGPT reads llms.txt.
Do this
The fixes, in the order to make them#
0 of 12 done
FAQ
Questions people ask about ChatGPT search#
- Does ChatGPT use Google or Bing to search?
- Neither alone. OpenAI says ChatGPT search partners with other search providers and links Microsoft’s and Shopify’s privacy policies, and it runs its own crawler,
OAI-SearchBot. Being indexed by Bing and crawlable byOAI-SearchBotcovers both paths. - Can I stay in ChatGPT search but opt out of AI training?
- Yes. Allow
OAI-SearchBotand disallowGPTBotin robots.txt. OpenAI treats the two as independent settings, and changes take about 24 hours to reach search. - Why does ChatGPT read my page but not cite it?
- It reads far more than it cites: Ahrefs found about half of retrieved URLs go uncited. Pages whose titles match the rewritten query and state the fact in their first lines are the ones that make the cut.
- What changed in ChatGPT search in August 2026?
- On 8 August, as GPT-5.6 became the default, ChatGPT began consulting about twice as many pages per answer and running far more
site:searches of brand and official domains. Brand-owned pages gained citations, and Reddit’s share fell about 86%. - How do I see ChatGPT traffic in Google Analytics 4?
- Check the AI Assistant default channel, added on 13 May 2026, and the session source
chatgpt.com. Add a custom channel onchatgpt.comas a backstop, and tag your own ads and product feeds so they don’t inflate it. - Does ChatGPT render JavaScript?
- OpenAI doesn’t document rendering, and Vercel’s crawler study found OpenAI’s bots fetch JavaScript files without running them. Put anything you want cited in the server-rendered HTML.
- How do products get into ChatGPT shopping results?
- Shopify merchants are included through Shopify Catalog. Other merchants apply for a direct product feed under the Agentic Commerce Protocol, which is open to approved partners. Product results aren’t ads, and checkout can stay on your own site.
- Is ChatGPT Atlas still worth optimizing for?
- No. OpenAI has deprecated Atlas and retired the standalone ChatGPT agent. Agent browsing now runs through ChatGPT Work’s cloud browser, which signs its requests with Web Bot Auth, and the built-in browser in the ChatGPT desktop app.
Sources
The 32 sources behind this guide#
OpenAI documentation
- 1Overview of OpenAI crawlersOpenAI · developers.openai.com
- 2Searching the web with ChatGPTOpenAI Help Center · help.openai.com
- 3Shopping with ChatGPT SearchOpenAI Help Center · help.openai.com
- 4Powering product discovery in ChatGPTOpenAI · openai.com
- 5Commerce: Agentic Commerce Protocol and product feedsOpenAI Developers · developers.openai.com
- 6Advertiser guidance for allowing OpenAI web crawlersOpenAI Help Center · help.openai.com
- 7Testing ads in ChatGPTOpenAI · openai.com
- 8ChatGPT Atlas retirement (formerly the publishers and developers FAQ)OpenAI Help Center · help.openai.com
- 9Introducing ChatGPT AtlasOpenAI · openai.com
- 10ChatGPT agentOpenAI Help Center · help.openai.com
- 11Using cloud browser in ChatGPTOpenAI Help Center · help.openai.com
- 12ChatGPT Work's cloud browser allowlistingOpenAI Help Center · help.openai.com
- 13Using site tools in the ChatGPT desktop appOpenAI Help Center · help.openai.com
Studies and platform docs
- 14ChatGPT's research depth doubledSearchable · searchable.com
- 15GPT-5.6 retrieval changesPeec AI · docs.peec.ai
- 16ChatGPT tripled its fan-out queriesNectiv · nectivdigital.com
- 17How ChatGPT does query fan-out (15 million queries studied)Ahrefs · ahrefs.com
- 18Why ChatGPT cites the pages it doesAhrefs · ahrefs.com
- 19AI search overlap with Google and BingAhrefs · ahrefs.com
- 20Do AI assistants prefer to cite fresh content?Ahrefs · ahrefs.com
- 21AI brand visibility correlations (75,000 brands)Ahrefs · ahrefs.com
- 22Understanding ChatGPT citations: evidence from production web searcharXiv · arxiv.org
- 23How to optimize for ChatGPT (129,000-domain study)SE Ranking · seranking.com
- 24The science of how AI pays attentionGrowth Memo · growth-memo.com
- 25Reddit's citations in ChatGPT fallSemrush · semrush.com
- 26ChatGPT search insightsSemrush · semrush.com
- 27The branded link updateProfound · tryprofound.com
- 28The rise of the AI crawlerVercel · vercel.com
- 29Your site, your rules: new AI traffic optionsCloudflare · blog.cloudflare.com
- 30AI Performance in Bing Webmaster ToolsMicrosoft Bing · blogs.bing.com
- 31What's new in Google Analytics (AI Assistant traffic)Google Analytics Help · support.google.com
- 32Default channel group (AI Assistant)Google Analytics Help · support.google.com