Google-Extended in logs: The Complete 2026 Guide

Google-Extended sends a distinct user-agent. There is no version number.

Google-Extended

That is the full string. It does not include browser tokens or platform identifiers. Google designed it to be easy to filter. The simplicity makes it one of the easiest AI crawlers to find in log data. Google's AI crawler documentation confirms the exact string.

Google also operates Google-CloudVertexBot and Google-Experimental. These are different crawlers for different products. Only Google-Extended is used for Gemini training data.

Google-Extended crawls from Google's published IP ranges. Google publishes a full list of crawler IP addresses as a DNS TXT record and a JSON file.

dig TXT _netblocks.google.com

The resolved ranges overlap with Googlebot. You cannot distinguish Google-Extended from Googlebot by IP alone. You must check the user-agent string. Google's crawler IP list shows the complete range.

Extract Google-Extended requests from access logs.

grep "Google-Extended" /var/log/nginx/access.log

Use extended regex to capture the exact string and avoid false positives on the word "Extended" in URLs.

grep -E '"Google-Extended"' /var/log/nginx/access.log

For Apache combined log format, the user-agent field is the last quoted string. The same grep works. For rotated logs, use zgrep.

zgrep -E '"Google-Extended"' /var/log/nginx/access.log.*.gz

Tools like lnav support interactive filtering. Grafana Loki's logql can parse the user-agent field with a regex pipeline stage.

{job="nginx"} | regexp `(?P<agent>[^"]*)"$` | agent =~ "Google-Extended"

Google-Extended crawls at a moderate pace. It does not burst like ClaudeBot. It follows a steady, even crawl rate. This matches Google's standard crawler infrastructure.

Google-Extended respects robots.txt. It reads the directives at the start of each session. It obeys Crawl-delay if set. Use robots.txt to control access.

User-agent: Google-Extended
Disallow: /

The crawler sends a valid User-Agent header. It sends standard Accept and Accept-Encoding headers. It does not send a Referer header. The Connection header is keep-alive.

Google-Extended requests come from IPs that also serve Googlebot traffic. The crawl patterns are identical. This makes log separation essential. You cannot rely on IP-based filtering. Google's guidance on AI crawler identification states this clearly.

Google-Extended traffic increased significantly in 2025. Google uses it to train Gemini 2.0 and later models. Early 2026 data from Cloudflare shows Google-Extended in the top three AI crawlers. Cloudflare's 2025 AI bot report documented a 200% increase in Google-Extended requests across their network.

The spike correlates with product launches. When Google releases a new Gemini version, crawl volume jumps. Plan for traffic to grow with each model cycle.

Googlebot is the search index crawler. Google-Extended is the AI training crawler. They serve different purposes. Googlebot sends Mozilla/5.0 ... Googlebot. Google-Extended sends just Google-Extended. A single page can receive requests from both crawlers in the same minute. Google Search Central explains the distinction.

Your log analysis must treat them separately. Blocking one does not block the other. Rate limiting applies independently. Many site operators mistakenly block Googlebot and wonder why AI crawl traffic persists. They are separate crawlers with separate directives.

The crowdsec AI bot collection includes Google-Extended. The botd library from FingerprintJS classifies Google-Extended as an AI bot. FingerprintJS's bot detection shows how to distinguish verified crawlers from impostors.

Most CDN log dashboards support Google-Extended filtering. Cloudflare, Fastly, and Akamai include it in their bot classification databases.

The Google-Extended audit. Note the gap between Google-Extended and Googlebot identification. They share IPs but have different user-agents. Your monitoring pipeline must separate them at the agent level, not the IP level. Audit quarterly.