Identifying GPTBot in logs: The Complete 2026 Guide

GPTBot sends a specific user-agent string. The canonical form is:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible with GPTBot/1.0; +https://openai.com/gptbot

The key identifiers are the token GPTBot/1.0 and the URL https://openai.com/gptbot. OpenAI documents this publicly. OpenAI's GPTBot documentation lists all official crawler variants. OpenAI also operates ChatGPT-User for user-facing chatbot traffic. Do not confuse the two.

GPTBot crawls from a published IP range. OpenAI publishes the full IPv4 and IPv6 prefix in their documentation. OpenAI's IP ranges show the exact CIDR blocks. These ranges change occasionally. Check them every quarter.

You can also run reverse DNS lookups on connecting IPs. GPTBot hostnames resolve under the openai.com domain. A typical PTR record looks like crawl-xxx-xxx-xxx-xxx.openai.com. Any IP that does not match this pattern is not GPTBot.

Grep your access logs. Filter on the GPTBot user-agent string.

grep GPTBot /var/log/nginx/access.log

For Apache, the same pattern works across any log format that captures user-agent. Use zgrep on rotated .gz files.

zgrep GPTBot /var/log/nginx/access.log.*

Count requests per day.

grep GPTBot /var/log/nginx/access.log | awk '{print $4}' | cut -d: -f1 | sort | uniq -c | sort -rn

Pipe to jq if your logs are JSON-formatted. Cloudflare's logpull API supports user-agent filtering for sites behind their proxy.

GPTBot respects robots.txt. It reads the directives before every crawl session. It fetches at a moderate rate compared to other AI crawlers. OpenAI limits GPTBot to a single concurrent request per host by default. Cloudflare's bot detection notes that GPTBot sends standard TLS handshakes and valid HTTP headers. It does not spoof other browsers.

Check the Accept header. GPTBot sends */*. Check the Accept-Encoding header. It includes gzip and brotli. These small signals help confirm a real GPTBot visit versus a scraper masquerading as one.

OpenAI runs two crawlers. GPTBot is for model training. ChatGPT-User is for real-time user queries from ChatGPT. ChatGPT-User sends this agent:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible with ChatGPT-User/1.0; +https://openai.com/chatgpt-user

The ChatGPT-User crawler hits pages when a ChatGPT user pastes a URL. It behaves differently. It follows redirects aggressively. It sends a shorter timeout. OpenAI's data controls explain how site owners can block ChatGPT-User separately from GPTBot.

GPTBot traffic grew through 2025. OpenAI continues expanding training data collection. Expect more crawl volume as new models launch. Google's Gary Illyes confirmed at PubCon that AI crawler traffic is rising across the web. Search Engine Land reported that AI crawler requests now exceed traditional search bot requests on many sites.

Several log analysis tools classify GPTBot automatically. GoAccess supports custom parser rules. Grafana Loki can label GPTBot traffic with regex extraction. The crowdsec project maintains a curated list of bad bot user-agents. CrowdSec's AI bot blocklist includes GPTBot variants. You can plug this into your log pipeline.

The GPTBot audit. Note the gap between GPTBot and ChatGPT-User detection. One agent is for training, the other for live queries. Your block and monitor strategy should treat them differently. Audit quarterly.