Pick an engine

11 engine guidesUpdated

AI SEO guides, engine by engine

Eleven AI engines, and no two find pages the same way. Google's answers come from Google's index and Copilot's from Bing's, while ChatGPT, Claude, Perplexity, Meta AI and Mistral Vibe each send a crawler of their own. Do the shared work once, then look up what each engine needs.

Compare all elevenWhere to start

The short answer

AI SEO (also called generative engine optimization, GEO, or answer engine optimization, AEO) is the work of getting your pages found, quoted and linked when an AI engine answers a question. The base is ordinary SEO: Google says optimizing for its AI features is “still SEO”. What changes from engine to engine is the plumbing. Each searches a different index, sends a different crawler and shows citations its own way. So let every engine’s search crawler in, put the answer in plain HTML, and write passages that answer the sub-questions engines search for.

The source map

Where each engine looks things up

When an engine searches before it answers, the results come from an index. Two engines use Google’s and Copilot uses Bing’s, five build their own with a crawler, and several won’t name their source. Pick an engine to trace it, or open its guide.

Stated in the vendor's current docsImplied, reported, or stated before 2026

  • On request: a live fetch

    When someone shares a link or an engine opens a result, a separate fetcher reads the page: ChatGPT-User, Claude-User, Perplexity-User, MistralAI-User, Meta-ExternalFetcher. Several of them don’t follow robots.txt on these user-triggered visits.

  • No search at all: training data

    When an engine answers without searching, it relies on what the model learned in training, which is what GPTBot, ClaudeBot, MistralAI-Training, Meta-ExternalAgent and Google-Extended govern. Opting out leaves OpenAI, Anthropic and Mistral search untouched; Google-Extended also ends Gemini-app grounding.

The matrix

Eleven engines, side by side

Seven things that change from engine to engine. Every cell is written in that engine’s guide, so the table can’t drift from the pages it summarizes.

The frontier five

More engines & agents

ChatGPT

OpenAI · Frontier

Where answers come from
Search partners (Microsoft, Shopify named) plus OpenAI's own crawler, OAI-SearchBot
Crawler to allow
OAI-SearchBot
Training opt-out
GPTBot — blocking it doesn't affect search
Runs your JavaScript?
Not documented; Vercel saw OpenAI's bots fetch JS files without running them
Shows up in analytics as
chatgpt.com, GA4 AI Assistant channel; links seen with utm_source=chatgpt.com
How citations look
Inline citation pills, a Sources panel, inline brand links
Biggest lever
Official first-party pages for prices, specs and policies that match its site: searches
Read the ChatGPT guide

Cells are summaries. Each guide carries the detail and cites its sources.

Shared and specific

Do it once, then per engine

Most AI SEO is shared work. Six things hold on every engine; the rest is a lookup in the matrix.

True on every engine

  1. 01

    Let each search crawler in, separately

    Official

    Vendors split search from training. Allow OAI-SearchBot, Claude-SearchBot, PerplexityBot, Meta-WebIndexer, MistralAI-Index, Googlebot and Bingbot, then decide on the training bots on their own merits. Check your CDN as well: from September 15, 2026, Cloudflare’s defaults for new domains block Training and Agent bots on pages that show ads.

  2. 02

    Put the answer in the HTML

    Observed

    In Vercel’s tests, none of the major AI crawlers ran JavaScript; Gemini, riding on Googlebot, was the one that rendered pages fully. Google processes JavaScript that isn’t blocked and Manus drives a real browser, but most engines read only what your server sends.

  3. 03

    Compete for the sub-questions

    Observed

    Engines turn one prompt into several searches; Google calls it query fan-out. Ahrefs found only 12% of the links ChatGPT, Gemini and Copilot cite rank in Google’s top 10 for the prompt itself (Perplexity: 28.6%). Depth across a topic beats one page per keyword.

  4. 04

    Write passages that stand alone

    Our read

    Citations point at sentences, not whole articles. Lead each section with its answer, and make that answer make sense out of context, with the name, number or date it depends on.

  5. 05

    Date it, and update for real

    Observed

    Ahrefs measured AI assistants citing content 25.7% fresher than Google’s organic results, with ChatGPT skewing newest. Show a real updated date, and change the substance when you change it.

  6. 06

    Get discussed off your site

    Observed

    Engines lean on third-party pages: Wikipedia led ChatGPT’s most-cited sources and Reddit led Perplexity’s in Profound’s 2024–25 data. Gemini named brands in 83.7% of appearances but linked them in 21.4% (Semrush, June 2026). Mentions elsewhere put your name in answers you don’t own.

Different on every engine

Look these up before you touch robots.txt, rendering or analytics. Each is a row in the matrix.

  • Where answers come fromThe index you need to be in
  • Crawler to allowBlock it and you can't be cited
  • Training opt-outWhat you can block and stay in
  • Runs your JavaScript?If not, scripted text is invisible
  • Shows up in analytics asWhere to find the visits
  • How citations lookWhat a reader sees and clicks
  • Biggest leverThe one move to make first
Look them up in the matrix
Where to start

Where to start

Reach and measurable referrals rank the engines differently. Do the shared work first, then follow your buyers.

1

Fix the shared foundation

Crawler access, server-rendered answers and dated pages help on all eleven at once. Access is the fast part: OpenAI says robots.txt changes reach ChatGPT search in about 24 hours, and Perplexity says up to 24.

robots.txt · every documented search crawler
# Search and citation: let these inUser-agent: OAI-SearchBotUser-agent: Claude-SearchBotUser-agent: Claude-UserUser-agent: PerplexityBotUser-agent: Meta-WebIndexerUser-agent: MistralAI-IndexUser-agent: MistralAI-UserUser-agent: GooglebotUser-agent: BingbotAllow: / # Training only: your call. Search stays on,# but Google-Extended also covers Gemini grounding.User-agent: GPTBotUser-agent: ClaudeBotUser-agent: MistralAI-TrainingUser-agent: Google-ExtendedDisallow: /

A named group replaces your User-agent: * rules for those bots, so copy any Disallow lines you rely on into it. Grok, DeepSeek and Manus document no token to add.

2

Then the frontier five

Google’s AI surfaces reach the most people, but their clicks count as Organic Search. ChatGPT sends most of the AI referrals you can see in analytics.

Reach

Share of AI referral visits, Jan–Apr 2026
  • ChatGPT74.8%
  • Gemini11.6%
  • Perplexity7.2%
  • Copilot3.5%
  • Claude2.6%

Across 101,574 sites. All AI platforms together sent 0.32% of their traffic. Google’s AI Overviews aren’t here: those clicks count as Organic Search. Source: SE Ranking

3

Add the other six where your buyers are

Each has one job worth doing if your audience uses it.

The writing is the part Rankbox does

Rankbox finds the questions your buyers ask, writes source-backed, answer-first articles, scores every draft and publishes on your schedule.

Start your free trial

7-day free trial · Card required · Cancel anytime

FAQ

AI SEO questions, answered

The cross-engine questions. Each guide answers the ones specific to its engine.

AI SEO is optimizing your site so AI engines retrieve, quote and link it in their answers: ChatGPT, Google AI Overviews and AI Mode, Gemini, Claude, Perplexity and the rest. It also goes by generative engine optimization (GEO), answer engine optimization (AEO) and LLM SEO. The names differ; the work is the same.

Less than the labels suggest. Google’s own guide says optimizing for generative AI search “is optimizing for the search experience, and thus still SEO”, and that AI Overviews and AI Mode have “no additional requirements”. The differences show up outside Google: separate crawlers to allow, other indexes (Bing, Brave and the engines’ own) and little JavaScript rendering.

Start with the shared foundation, then follow your buyers. By reach, Google leads: AI Overviews has over 2.5 billion monthly users, and ChatGPT over 1.2 billion weekly. By referrals you can measure, ChatGPT sent about 75% of AI referral visits in early 2026, Gemini about 12% and Perplexity about 7% (SE Ranking). AI Overviews clicks count as Organic Search, so they aren’t in that split.

Partly, by implication. OpenAI says ChatGPT search “sometimes partners with other search providers” and links Microsoft’s and Shopify’s privacy policies for them, without naming Bing outright, and it builds its own index with OAI-SearchBot. Allow OAI-SearchBot, the step OpenAI documents, and get indexed in Bing too: it covers that path and Copilot depends on it.

Usually not. GPTBot, ClaudeBot and MistralAI-Training are training-only, and blocking them leaves you in ChatGPT, Claude and Mistral Vibe search. Two exceptions: disallowing Google-Extended also removes you from Gemini-app grounding, and a noarchive tag keeps your content out of Copilot’s answers.

Yes. Since August 31, 2026, every Search Console property has a Search generative AI control under Settings. Inclusion is the default. Choosing exclude keeps your links and content out of AI Overviews, AI Mode and Discover’s generative features, usually within a day or two. It doesn’t affect training (that’s Google-Extended) and isn’t a ranking signal.

Not for Google: its AI optimization guide says Google Search doesn’t use llms.txt and you can ignore it there. None of the other engines’ crawler docs listed below mention it either. SE Ranking found about 10% of nearly 300,000 domains had one, and no link between having one and being cited. It does no harm, but it isn’t a priority.

Since May 2026, GA4 has a default AI Assistant channel for visits from assistants such as ChatGPT, Gemini, DeepSeek, Copilot and Grok. Google’s AI Overviews and AI Mode stay in Organic Search. ChatGPT tags most cited links with utm_source=chatgpt.com, but visits that arrive without a referrer, common from apps and agents, still land in Direct. Analytics shows clicks, not citations, so pair it with a fixed set of test prompts you rerun.

Access fixes are fast: OpenAI says robots.txt changes take about 24 hours to reach ChatGPT search, and Perplexity says up to 24 hours. Content changes take longer, because pages have to be recrawled and weighed against other sources. Plan in weeks, and test with the same prompts each time so you can see the shift.

If your buyers use them, and each has one main job. Copilot runs on Bing, so Bing Webmaster Tools and IndexNow cover it; Meta AI needs Meta-WebIndexer; Mistral Vibe needs MistralAI-Index; Grok also searches X; DeepSeek names no crawler, so server-rendered HTML is the job; and Manus visits in a cloud browser your bot rules can block.
Sources

Where these facts come from

Checked on . The matrix cells are sourced in each engine’s guide.

Vendor documentation and announcements

  1. 01Optimizing your website for generative AI features on Google SearchGoogle Search Central
  2. 02AI features and your websiteGoogle Search Central
  3. 03Search generative AI controlGoogle Search Console Help
  4. 04Google's common crawlers (Google-Extended)Google
  5. 05Expanding AI Mode (query fan-out)Google
  6. 06Default channel group (AI Assistant)Google Analytics Help
  7. 07Overview of OpenAI crawlersOpenAI
  8. 08Introducing ChatGPT searchOpenAI
  9. 09Searching the web with ChatGPT (search providers)OpenAI Help Center
  10. 10Anthropic's crawlers: ClaudeBot, Claude-User, Claude-SearchBotClaude Help Center
  11. 11Subprocessors (Web Search: Brave Search, TurboPuffer, Parallel)Anthropic Trust Center
  12. 12Perplexity crawlersPerplexity
  13. 13Copilot Chat privacy and protections (web grounding via Bing)Microsoft Learn
  14. 14New options for webmasters to control usage of their content in Bing ChatBing Webmaster Blog
  15. 15Web Search toolxAI
  16. 16X Search toolxAI
  17. 17Meta web crawlersMeta for Developers
  18. 18Introducing new AI experiences across our family of apps (2023)Meta
  19. 19Meta AI terms (search partners), effective May 13, 2026Meta
  20. 20DeepSeek privacy policyDeepSeek
  21. 21Mistral AI crawlers (robots)Mistral AI
  22. 22Cloud BrowserManus
  23. 23Block AI bots (new defaults from September 15, 2026)Cloudflare Docs
  24. 24Google I/O 2026 keynote (AI Overviews and AI Mode users)Google
  25. 25The Gemini app passes 1 billion monthly usersGoogle
  26. 26ChatGPT ads expand to 26 new countries (weekly users)OpenAI

Studies, data and reporting

  1. 27Anthropic appears to be using Brave to power web searches for its Claude chatbotTechCrunch
  2. 28The rise of the AI crawlerVercel
  3. 29Only 12% of AI-cited URLs rank in Google's top 10Ahrefs
  4. 30Do AI assistants prefer to cite fresh content?Ahrefs
  5. 31AI platform citation patternsProfound
  6. 32The ghost citations studySemrush
  7. 33AI traffic research study, 2026SE Ranking
  8. 34Does llms.txt impact AI visibility and citations?SE Ranking
  9. 35Top AI chatbots and tools websites, September 2026Similarweb
  10. 36ChatGPT UTM trackingLLM Pulse
Keep exploring

Understand AEO, GEO and how they differ from SEO

Key terms

Write the pages every engine can cite

Rankbox researches the questions your buyers ask, writes source-backed, answer-first articles, scores each draft and publishes on your schedule.

Start your free trial

7-day free trial · Card required · Cancel anytime