How to Optimize Content for LLMs: Writing for Machines That Don't Read Like Google

The slices are small. OpenAI's file search, for example, splits every file into 800-token chunks by default, about 600 words each. Perplexity says its search API treats "the individual sections and spans of documents as first-class units" in its architecture write-up. A chunk that says "it" instead of your product's name, or hides its numbers in an image, arrives at the model half empty.

Meaning beats repetition inside those chunks. In Rankbox's keyword density experiment, per point of density, a definition sentence moved similarity about 3.2 times as much as a keyword mention (median of 8 open embedding models), and repeats stopped helping after the second mention.

This guide shows how to optimize content for LLMs from the pipeline up: how the machines read a page, how embeddings see topical authority, why entity definitions win, and five formatting fixes for technical and B2B articles. For a full worked rewrite of one section, see our before-and-after rewrite of content for LLMs. For passage-by-passage editing, see how to optimize content for AI search.

How LLM Pipelines Read a Page

An LLM answer engine reads your page in six steps, and each step is a place to optimize content for LLMs or to lose it. Vendors publish some steps and keep others private, so treat this table as a model built from their docs, not any one product's spec.

Fetch and parse: the page the machine gets

The fetch step decides what exists. When Vercel and MERJ studied crawler traffic in December 2024, they found "none of the major AI crawlers currently render JavaScript", including OpenAI's OAI-SearchBot, ChatGPT-User and GPTBot, and Anthropic's ClaudeBot. Gemini is different, because it uses Googlebot's rendering. Anthropic's own web fetch tool "does not support websites dynamically rendered with JavaScript."

Parsing decides what survives as structure. Google Cloud's Agent Search docs describe two parsers. The basic one "detects text blocks, but not document elements such as tables, lists, and headings." The layout parser detects all of them in HTML. Perplexity writes parsing rules per site, and notes that a site "heavy on well-structured data in list or table form" can use more formulaic rules.

Chunk, embed and retrieve: the page in pieces

OpenAI's default chunker is a sliding window: 800 tokens per chunk, each overlapping the last by 400. A token is about three-quarters of a word, per OpenAI's token guide. Anthropic calls chunks of "no more than a few hundred tokens" typical.

Each chunk then becomes an embedding, a list of numbers that places its meaning in space (see vector embeddings). Perplexity searches its index "via both modalities," keyword and meaning, then lets cross-encoder rerankers make the final cut. Some search tools then pass the model only short excerpts of each page; our Reverse Prompt playbook works through what that budget means.

What Google does differently

Google's AI Overviews and AI Mode retrieve pages through Google's core ranking systems. Its AI optimization guide says "You don't need to write in a specific way just for generative AI search," and that there's "no requirement to break your content into tiny pieces." Take that at face value for Google. The effort to optimize content for LLMs pays off in the pipelines that fetch, chunk and embed pages themselves, and every fix below also reads better for people.

How Vector Embeddings Evaluate Topical Authority

Embeddings don't score authority directly. No AI search engine publishes a topical authority score built from embeddings, and Google has confirmed a "topic authority" system only for news, as our topical authority entry explains. What embeddings do is decide which of your chunks sit close to which questions.

Authority as coverage

AI engines split one question into several searches, a step called query fan-out. Each sub-query has its own nearest chunks. So in retrieval terms, topical authority looks like coverage: the share of a topic's sub-questions for which your site holds a chunk that is close in meaning and specific enough to quote. This is a model, not a documented ranking factor. It explains, in retrieval terms, why covering a subject's follow-up questions gives a site more chances to be picked.

Here is an illustrative coverage map for one buyer question, "best invoicing software for agencies that bill in several currencies." Tallyfold is a made-up invoicing and payments app for agencies, and Brindlework is a made-up rival.

Brindlework may be the better product. Tallyfold simply has more chunks near more of the questions an engine is likely to ask. To optimize content for LLMs across a topic, map the sub-questions first, then give each one its own section.

Why coverage matters more for small brands

Models know little about rare entities from training alone. Kandpal and colleagues showed that a model's ability to answer a factual question "relates to how many documents associated with that question were seen during pre-training," and that retrieval reduces that dependence. A B2B product with a few hundred mentions on the web is a rare entity. For it, retrieved pages are where the model learns what the product is, so your own pages have to carry the definitions.

Why Clear Entity Definitions Beat Keyword Stuffing

An entity is a specific thing with a name: a product, a feature, a company, a standard. Google framed this shift in 2012 when it launched the Knowledge Graph as "things, not strings." Embedding models make the same move at the level of each chunk. When you optimize content for LLMs, define the thing; don't repeat the string.

What the evidence shows

• Definitions move meaning. In Rankbox's test of 263 paragraphs, per point of density, a definition sentence moved similarity 3.2 times as much as a keyword mention (median of 8 open embedding models). Repeats stopped helping after the second mention. On prompts that described the need without the keyword, each extra mention lowered similarity in all 8 models.
• Self-contained facts retrieve better. The Dense X Retrieval study by Chen and colleagues indexed text as "propositions," short statements that each carry one fact and replace pronouns with full names. That beat indexing by passage, with a 25% relative gain in top-5 recall on a set of entity questions for a weaker retriever, and most of the gain came from questions about rare entities.
• New product names are hard for embeddings. Google Cloud's hybrid search docs say "brand new product names that were added recently" don't work with semantic search alone, because the embedding model never saw them in training.

Together, those two findings set the job. The exact name gives keyword search something to match, and the definition around it gives the embedding a meaning to place.

The entity definition sentence

Write one sentence per entity in this shape: [full name] is a [category] that [does what] for [whom], [with one number or limit]. Put it at the start of the section where the entity first matters.

Every entity-first sentence can be lifted into an answer as it stands. That's the practical meaning of "optimize content for LLMs": each claim carries its own subject. For how many times to name the keyword, use the Twice-Four Rule from the keyword density study: name it twice per passage of about 100 words, and give four defining facts.

Keywords still count, a little

The claim that LLMs "completely ignore" keywords is too strong. In Rankbox's data, the second mention still gave a lift. OpenAI's retrieval API lets developers blend "semantic embedding matches vs. sparse keyword matches," and Perplexity runs both. So when you optimize content for LLMs, keep the exact product, feature and category names. Just stop at two per passage and spend the rest of the words on facts.

The Chunk-Ready Five: 5 Ways to Optimize Content for LLMs

Technical and B2B articles often hide their best facts where machines read worst: spec tables drawn as images, pricing in tabs, setup steps in PDFs. The Chunk-Ready Five is Rankbox's formatting checklist for these articles. Each fix protects one step of the pipeline.

1. Serve every fact as HTML text

Put every fact a buyer might ask about in the HTML your server sends. Render it on the server (see server-side rendering), and keep answers out of tabs and accordions that load on click. Microsoft's Bing team puts it plainly in an October 2025 guide: "AI systems may not render hidden content, so key details can be skipped."

The same guide warns against keeping core information only in PDFs, which "often lack the structured signals" of HTML, or only in images. That covers most B2B hiding places: spec sheets, rate cards, security whitepapers and architecture diagrams. Publish each as an HTML page and keep the PDF as a download. It's the cheapest way to optimize content for LLMs on an existing site, because nothing needs rewriting.

2. Size each section to fit one chunk

Use a heading wherever the topic changes, and keep each section to about 120 to 300 words. Bing's guide says headings "act like chapter titles that define clear content slices." Layout-aware chunkers follow them: in Google Cloud's version, "all text in a chunk will come from the same layout entity, such as headings, subheadings, and lists."

Fixed-size chunkers ignore headings, and the arithmetic explains the 300-word ceiling. With OpenAI's defaults, a new chunk starts every 400 tokens and runs for 800. Any passage of 400 tokens or fewer (about 300 words) therefore sits whole inside at least one chunk, wherever it starts. A passage longer than 800 tokens (about 600 words) never does. Between those sizes, it depends on luck. The lower bound comes from citation data: our passage-level guide shows why very short sections get cited less.

3. Re-anchor each section with full names

Open every section by naming its subject in full, as if the reader skipped straight to it, because a chunker may deliver it that way. Anthropic's contextual retrieval research exists because chunks lose this context: adding 50 to 100 tokens of context to each chunk cut failed retrievals by 35% on its own. You can do the same in the text itself.

Three habits cover it:

• Start the section with "Tallyfold's payout schedule," not "Payouts."
• Spell out each acronym once per section: "foreign exchange (FX)."
• Use one name per thing. Don't rotate "Tallyfold," "the platform," "our solution" and "the app" for variety.

4. Put specs, prices and comparisons in HTML tables

Use real table markup, with a header row, units in the headers and one fact per cell. In the Table Meets LLM study (Sui and colleagues, WSDM 2024), tables given to the model as HTML beat plain text with separators by 6.76%. Google Cloud's layout parser detects tables in HTML and can attach a written description to each one, so the table can be found by what it's about.

Add one sentence above each table that says what it shows and when it was checked. That sentence is what a retriever matches when the question is "how much does Tallyfold charge to convert currencies?"

5. Write steps as numbered lists and code as code blocks

Write procedures as numbered steps, one action per step, using the exact button and field names from the product. Put code, API calls, config and error messages in code blocks as text, never as screenshots. Exact strings such as invoice.currency, error codes and version numbers are the kind of "out of domain" terms that keyword search can match and embeddings may miss, so they belong in the text word for word. For API guides and developer docs, this fix does most of the work to optimize content for LLMs.

What Surfer and Clearscope Measure Now

The content plan behind this post claimed that SEO writing tools like Surfer and Clearscope "still obsess over keyword density and TF-IDF counts that large language models completely ignore." Half of that holds up as of 28 September 2026. Both tools still build guidance from term lists drawn from Google's top results, with suggested counts. But none of the Surfer or Clearscope pages checked for this post describes the method as TF-IDF, and both tools have added AI search features.

Surfer's own Content Score guide now says "how those terms are used in context matters more than how often they appear," and advises writers to "avoid over-optimizing any single suggested term." Clearscope calls its typical-use range "a guide rather than a rule." Clearscope's pages disagree on engines, too: its pricing page lists ChatGPT and Gemini for Query Tracking, and its homepage adds Claude.

So state the gap this way. Count targets were built for lexical ranking, and LLM pipelines stop rewarding repeats early. Both tools' term lists are still useful when you optimize content for LLMs, as a checklist of entities and facts to cover. Treat the suggested counts as a ceiling, not a target.

Worked Example: How a Chunker Cuts a Tallyfold Guide

Here is the arithmetic behind fix 2, on a made-up 2,400-word Tallyfold guide to multi-currency invoicing. Token counts use OpenAI's rule of thumb of 100 tokens per 75 words, and the chunker uses OpenAI's file search defaults.

The guide makes seven chunks, starting at token 0, 400, 800 and so on up to 2,400. The exchange-rate section is spread across four of them, so no single chunk holds the whole answer to "what rate does Tallyfold use?"

Now add 100 words to the introduction. Every later section moves down 133 tokens. The tax section now runs from about 1,867 to 2,467, which fits neither the chunk from 1,600 to 2,400 nor the one from 2,000 to 2,800. A paragraph added at the top of the page just split a section near the bottom.

The fix is to split the long sections at natural headings. Break the exchange-rate section into three H3s of about 230 words (when the rate is locked, which rate is used, what happens if it moves before payment), and the tax section into two of 225. Every section is then under 400 tokens, so each one lands whole in some chunk no matter what changes above it.

Two caveats keep this honest. This is OpenAI's default for developer file search, not a documented setting of ChatGPT search. And layout-aware chunkers cut at headings instead, which the same fix also serves. Either way, section length is part of how you optimize content for LLMs, not a style choice.

How to Check a Page Before You Optimize Content for LLMs

Five checks show where a page loses facts, in about 20 minutes per page and with no paid tools.

1. Fetch it raw. Run curl -s https://tallyfold.example/guide | grep -i "0.9%" with a fact from your page. If the fact isn't in the raw HTML, a crawler that doesn't run JavaScript won't see it.
2. Outline the headings. Paste the page's HTML into our heading structure checker. Every topic change should have its own H2 or H3.
3. Count words per section. Flag any section over 300 words, and split anything over 600.
4. Read each section's first sentence alone. It should name the product or feature in full.
5. Count exact-keyword mentions per passage. Our keyword density checker counts them. Trim anything past two per 100 words.

For the basics around the content, the AI search readiness check scans a URL for 12 signals, including AI crawler access, a single H1 and structured data.

Where Rankbox Fits

If you'd rather not optimize content for LLMs by hand on every new article, Rankbox's Citation-Ready Writer researches the live web and writes 2,000–3,500-word, source-backed articles with headings, definitions and tables. Brand Voice applies the tone, audience, style rules and product details you give it, and mentions your product where it fits. Articles reach your site through Rankbox's API on the Business plan, $49.50 a month with a 7-day trial.

Rankbox doesn't invent your spec table, though. It doesn't import proprietary data or run benchmarks, so your real limits, fees and integration details have to come from you, through the product details in Brand Voice. It also doesn't track AI citations today or publish straight to a CMS. See plans and pricing.