Field guide

Contradiction Decoder: Google's AI optimisation guide vs ChatGPT, Perplexity, and Claude

Google's May 2026 AI optimisation guide is authoritative for Google Search AI features. It is not a universal rulebook for ChatGPT, Perplexity, or Claude. This field guide separates what travels everywhere from what is engine-dependent, in one operator table.

Last updated 13 September 2026

Opening answer

Do the durable work once. Decode the contradictions before you copy a hack across engines.

Google Search Central's guide on optimising for generative AI features on Google Search states that those features are rooted in core Search ranking and quality systems, and that SEO remains relevant. From Google's perspective, "AEO" and "GEO" are labels people use for AI-search visibility work, and optimising for generative AI search is still SEO for Search.

That sentence is true and useful. It is also easy to over-apply. Google's mythbusts, for example on llms.txt, content chunking, rewriting only for AI, and chasing inauthentic mentions, govern Google Search, including AI Overviews and AI Mode. They do not automatically rewrite how OpenAI, Perplexity, or Anthropic crawl, retrieve, or cite you.

The operator job is to keep one shared foundation, then run an engine-dependent control layer, especially robots.txt user-agents, without treating every agency AEO checklist as Google doctrine.

What Google actually said, scoped tightly

For Google Search generative AI features, Google's guide emphasises:

Keep SEO fundamentals: valuable, non-commodity, people-first content; crawlable and indexable pages eligible for snippets; sound technical structure.
Retrieval, grounding and query fan-out: generative answers draw on retrieved Search results and related concurrent queries, with clickable supporting links.
Measure in Search Console: the Generative AI performance report covers Google's own AI features.
Ignore, for Google Search: special AI files such as llms.txt as a ranking lever; breaking pages into tiny AI chunks; rewriting prose just for AI systems; farming inauthentic mentions; treating schema as a special generative-AI requirement, since schema remains useful for rich results rather than as an AI cheat code.

Frase's commentary on the guide is useful as a scope reminder: the document is about Google's surfaces drawing from Google's index. It is not a verdict on every AI engine's retrieval stack.

Where industry AEO checklists diverge

HubSpot's answer engine optimisation guide, updated 11 September 2026, still frames AEO as improving how often and how accurately a brand appears in AI answers across engines such as ChatGPT, Perplexity, and Gemini. Its tactics include question-shaped content, FAQ, How-To and Article schema, entity signals, prompt mapping, and off-site PR and social corroboration, with measurement via citation frequency and AI share of voice.

That is not wrong. It is a multi-engine marketing playbook. Parts of it overlap Google's durable advice: clear answers, non-commodity expertise, entity clarity. Parts of it push harder on schema-as-AI-lever and off-site mention programmes than Google's Google-Search guide endorses for Google's own generative features.

Contradiction Decoder rule: when someone says Google killed schema for AEO, they have misread Google. When someone says Google requires FAQ schema to win ChatGPT, they have invented a rule Google never wrote.

The crawler split most teams still miss

ChatGPT, Perplexity, and Claude each separate training-oriented crawl from search and retrieval crawl, and often from a user-initiated fetch. Blocking the AI bot as one blob is how brands accidentally opt out of citations while intending to opt out of training.

OpenAI, primary documentation

OAI-SearchBot: surfaces sites in ChatGPT search features. Opting out means you will not be shown in ChatGPT search answers, although navigational links may still appear. Allow this if you want search visibility.
GPTBot: crawl that may be used for training generative AI foundation models. Disallow it to signal a training opt-out.
The settings are independent. Allowing SearchBot while disallowing GPTBot is a documented pattern.
ChatGPT-User: user-initiated fetches, where robots.txt rules may not apply.
Search robots.txt changes can take about 24 hours to take effect.

Perplexity, primary documentation

PerplexityBot: surfaces and links sites in Perplexity search results, and is not used to crawl for AI foundation model training, per Perplexity's crawler docs. Allow it if you want search visibility, and verify WAF allowlists against the published IP JSON.
Perplexity-User: user-requested fetches, which generally ignore robots.txt because a user requested the fetch, per Perplexity's crawler docs.

Anthropic and Claude, primary documentation, Help Center updated April 2026

ClaudeBot: web content that could contribute to model training. Restricting it signals that future materials should be excluded from training datasets.
Claude-SearchBot: indexing for search result quality. Disabling it may reduce visibility and accuracy in user search results.
Claude-User: user-initiated retrieval. Disabling it may reduce visibility for user-directed web search. Anthropic states its bots honour robots.txt, including these agents.
Anthropic prefers robots.txt over IP blocks for a durable opt-out, and supports Crawl-delay for ClaudeBot.

Google-Extended is not a Search ranking switch

Google documents Google-Extended as a robots.txt product token, not a separate HTTP user-agent, that controls whether content may be used for training future Gemini models powering Gemini Apps and Vertex, and for grounding in Gemini Apps and Vertex Grounding with Google Search. Google states this control does not impact inclusion in Google Search or Search ranking.

Operator takeaway: do not treat a Google-Extended disallow as opting out of AI Overviews. Search generative AI features still sit on Googlebot and Search systems. Gemini Apps grounding and training controls are a separate lever.

Operator table: do, don't, engine-dependent

Tactic	Google Search AI features	ChatGPT (OpenAI)	Perplexity	Claude (Anthropic)
Non-commodity, people-first content; clear answers	Do (Google's highest long-run lever)	Do (durable; industry AEO guides agree)	Do	Do
Technical crawl and index hygiene; snippet eligibility	Do (required for Google AI feature eligibility)	Do (pages must be fetchable by relevant bots)	Do	Do
Treat Google's guide as the rule for every engine	Don't over-extend (guide is Google-scoped)	Don't	Don't	Don't
llms.txt or special AI markdown files as a Google ranking lever	Don't (Google Search does not use them)	Engine-dependent (optional for other systems; not a Google Search lever)	Engine-dependent	Engine-dependent
Break content into tiny AI chunks for Google	Don't (not required)	Don't as a Google-mythbust copy-paste; write for readers	Don't as primary strategy	Don't as primary strategy
Rewrite copy for AI only	Don't (Google)	Prefer clear human answers; no Google mandate to contort prose	Same	Same
FAQ, How-To or Article schema	Keep for rich results; not required for Google generative AI search	Engine-dependent (common in multi-engine AEO playbooks; not an OpenAI robots requirement)	Engine-dependent	Engine-dependent
Chase inauthentic mentions or mention farms	Don't (Google)	Don't (fails the pub test; quality systems still matter)	Don't	Don't
Genuine off-site corroboration (reviews, PR, consistent entity)	Useful for brand understanding; do not fake it	Do as entity corroboration (industry guidance); measure carefully	Do with the same honesty bar	Do with the same honesty bar
Allow the search or retrieval bot; optionally disallow the training bot	Search: Googlebot. Gemini Apps and Vertex training and grounding: Google-Extended (does not change Search ranking). UK CMA publisher controls: Article 3	Do pattern: allow OAI-SearchBot, decide GPTBot separately	Do pattern: allow PerplexityBot for search visibility; training is not that bot's documented job	Do pattern: decide ClaudeBot against Claude-SearchBot and Claude-User separately
Assume user-fetch agents always honour robots.txt	Not applicable	Don't assume (ChatGPT-User may ignore)	Don't assume (Perplexity-User generally ignores)	Anthropic states its bots honour robots.txt, including Claude-User; still verify behaviour
Measure only in Google Search Console's Generative AI report	Do for Google AI features	Incomplete alone (does not cover ChatGPT)	Incomplete alone	Incomplete alone
Blindly block all AI bots	May be fine for privacy goals	Don't if you want ChatGPT search citations (OAI-SearchBot)	Don't if you want Perplexity surfacing (PerplexityBot)	Don't if you want Claude search indexing (Claude-SearchBot)

How to use the table in practice

Write for humans, with extractable answers. That row holds across every column.
Keep Google SEO sound if Google AI features matter. Use Search Console's Generative AI report for that surface only.
Decide training against search bot policy per vendor, in writing, with legal and comms. Do not collapse them into one Disallow.
Keep schema for the jobs it still does, rich results and clearer structure. Do not sell it as a secret Google AI Overview cheat code.
Run a contradiction check on every agency recommendation: is this claimed for Google Search, for ChatGPT retrieval, or for all engines without evidence?

FAQ

Does Google's AI optimisation guide apply to ChatGPT?

No. It governs Google Search generative AI features, such as AI Overviews and AI Mode, rooted in Google's Search systems. ChatGPT search visibility is controlled separately, including OAI-SearchBot.

Did Google kill llms.txt?

Google says Search does not use llms.txt, or similar special AI files, as a lever for Google Search visibility. Maintaining such a file for other systems neither helps nor harms Google Search, per Google. That is not the same as saying no AI system ever reads it.

Should we block GPTBot?

That is a training-policy decision. Blocking GPTBot does not, by itself, block OAI-SearchBot. If you want ChatGPT search answers to be able to show your pages, allow OAI-SearchBot, and its published IP ranges, unless you intentionally opt out of search surfacing.

Is FAQ schema required for AEO?

Not for Google generative AI search, according to Google. In this pass we also found no OpenAI, Anthropic or Perplexity primary publisher document that positions FAQ schema as a citation ranking requirement. Schema remains useful for rich results and is still pushed in vendor AEO playbooks, for example HubSpot. Treat it as structured clarity, not a universal citation cheat code.

Why do Google and HubSpot-style AEO guides disagree?

They often answer different questions. Google documents Google Search. Multi-engine AEO guides describe tactics across ChatGPT, Perplexity, Gemini and more. They overlap on fundamentals and diverge on hacks, and on how hard they push schema and off-site programmes.

Closing

AEO leads SEO for how we talk about AI-answer work, but Google still calls its own surface SEO. The Contradiction Decoder is how we stop paying twice: once for durable content and crawl hygiene, and again for engine-specific controls that actually match each vendor's bots. Copy-pasting Google's mythbusts into a ChatGPT robots policy, or copy-pasting a ChatGPT checklist into Search Console strategy, is how good teams ship contradictions.

How AI actually chooses, the full mechanics →

Sources

Notes

Research pack for this article received 13 September 2026: it confirms the draft direction, and the Google-Extended clarification was added as a result.
The OpenAI publishers and developers FAQ returned 403 on fetch, so UTM and noindex nuances remain snippet-backed until a browser re-verification.
The Perplexity Help Center robots article is snippet-backed. The primary bots page is solid.
HubSpot's schema and off-site tactics are vendor secondary sources, not operator law. No OpenAI, Anthropic or Perplexity primary source was found that makes FAQ schema a citation ranking requirement.
We do not assert that Google-Extended blocks AI Overviews. It does not affect Search ranking.
We do not assert that Google's mythbusts apply unchanged to ChatGPT, Perplexity or Claude.
We do not assert that a GPTBot disallow alone removes ChatGPT search visibility.
The UK CMA and Google-Extended publisher commencement detail sits in Article 3.