Google-Extended Token for AI Training: Complete 2026 Guide

Google-Extended is a robots.txt product token, not a crawler. It governs two downstream use cases: training future Gemini foundation models and grounding answers in the Gemini app and Vertex AI apps that use grounding. The token never issues an HTTP request itself.

Per Google's official robots.txt documentation, robots.txt is primarily a crawl-traffic management mechanism, not a content-hiding mechanism (https://developers.google.com/search/docs/crawling-indexing/robots/intro). Google-Extended lives inside that same file but controls a separate downstream layer.

Per Marie Haynes's analysis, Google-Extended does not impact a site's inclusion in Google Search and is not used as a ranking signal (https://www.mariehaynes.com/should-you-use-google-extended-in-robots-txt/). Blocking it leaves Search traffic untouched.

The token does not block AI Overviews. AI Overviews are part of Google Search and follow Google's standard snippet and preview controls. Blocking Google-Extended leaves AI Overview citations intact, since the cited sources are pulled through Google's standard Search infrastructure.

The token does not block AI Mode in Search Labs either. AI Mode runs Gemini under a Search product surface, separate from the Google-Extended directive. The two systems are deliberately separated by Google's architecture.

The token cannot un-train data already absorbed into current Gemini models. Blocking stops future training runs from absorbing your content, not past runs from retaining it. Historical embeddings are not retractable.

Training crawlers like GPTBot, ClaudeBot, and CCBot fetch content over HTTP. Google-Extended never issues a fetch. It is a directive embedded inside robots.txt, not a User-Agent string, so it never appears in server access logs and cannot be observed at the WAF layer.

Per No Hacks's 2026 AI user-agent taxonomy reference, Google-Extended and Applebot-Extended sit in their own category of opt-out tokens, separate from crawlers and separate from search-retrieval bots (https://nohacks.co/blog/ai-user-agents-landscape-2026). The category distinction matters for log analysis.

Per Digital Applied's June 2026 access-control decision matrix, only 2.98% of the top one million sites actively block AI training bots, and most of those blocks target crawlers, not tokens (https://www.digitalapplied.com/blog/ai-crawler-access-control-2026-robots-llms-txt-decision-matrix). The opt-out default has not flipped yet.

Block when your content carries licensed value. Paywalled research, premium journalism, and original analysis with downstream revenue attached benefit from blocking, since Gemini citation as a free source erodes the paid model without driving equivalent traffic.

Block when your IP lives in the prose. Original reporting, fiction, and first-person essays fit this category. The token keeps the words out of Gemini's training set while preserving Search snippet visibility for organic discovery.

Block when your competitive moat depends on word choice. If a competitor could replicate your editorial voice from model outputs, the token is one defensive layer among several.

Allow when Gemini citation drives discovery. The Gemini app, Deep Research, and Assistant products surface cited pages from grounding. Blocking Google-Extended removes your pages from that citation layer entirely.

Allow when your business model is exposure-based. Consulting, advisory, and freelance practices earn more from being quoted than from being scraped, and the citation serves as a top-of-funnel signal that compounds across conversations.

Allow when your schema already wins snippet space. Sites with strong FAQ and HowTo structured data already convert Search impressions into qualified sessions, and the token does not change that flow. The grounding layer adds new exposure on top.

After a Google crawler announcement drops, you sit down at your terminal and re-read your robots.txt. You grep for the Google-Extended block. If blocking, you verify the Disallow path still matches the live URL tree. A site migration can break the rule quietly.

You pull server logs from the past 30 days and search for Google-Extended as a User-Agent. You find nothing. The token never issues a fetch, so its absence in logs is the expected state. Treating log silence as evidence the rule is failing is a common mistake.

You cross-reference robots.txt against your monetization model. The defensible default for most publishers is to allow search crawlers while blocking training tokens, since training crawlers contribute to model weights without sending equivalent referral traffic.

Note the gap. This post synthesizes 2025 and 2026 data from four sources: Google Search Central, Marie Haynes, No Hacks, and Digital Applied. Three vendor IP range publications remain undisclosed as of writing. Replication required on your own property before adjusting the robots.txt layer.