Agentic commerce
What is the difference between GPTBot, OAI-SearchBot and ChatGPT-User?
Updated
They are three OpenAI agents doing different jobs. GPTBot collects content for model training — blocking it costs nothing in search rankings or AI answer visibility. OAI-SearchBot builds the search index ChatGPT answers from — blocking it removes you from AI answers. ChatGPT-User is a live fetch made on behalf of a specific person asking about you right now — blocking it turns away a customer mid-question. A merchant should block at most the first.
The names are the problem
All three belong to OpenAI, all three appear in your logs as OpenAI traffic, and their names give almost no clue which is which. That is not a criticism of the naming so much as an explanation of why the most expensive mistake in this area is an accident rather than a decision.
A rule written to "block OpenAI" — the intuitive thing to write — catches all three. The intent is usually to opt out of model training. The effect includes opting out of ChatGPT search results and refusing live customer lookups.
| User-agent | What it does | Blocking it costs you |
|---|---|---|
| GPTBot | Crawls content into training corpora for future models | Nothing in rankings or AI answer visibility. A legitimate commercial choice. |
| OAI-SearchBot | Builds the index ChatGPT search answers from | Your presence in ChatGPT answers. This is the expensive one. |
| ChatGPT-User | Fetches a page live because a specific user asked about it | A real person, mid-question, who was being pointed at you. |
Why blocking GPTBot genuinely is free
This surprises people, so it is worth being explicit. Training crawlers have no bearing on Google Search rankings — Google does not use OpenAI's crawl — and no bearing on whether ChatGPT surfaces you in search, because that comes from a different index built by a different agent.
What you lose by blocking GPTBot is harder to measure: future models may know less about your brand from having read less of it. Whether that matters depends on whether you would rather be described from your own words or from third-party coverage. Both are defensible. Neither costs you a ranking.
A robots.txt that reflects the actual intent
If your intent is "do not train on my content, but do surface me", the rules that express it name each agent explicitly rather than relying on a pattern.
- Disallow GPTBot if you want to opt out of training. Also consider CCBot, Bytespider, Google-Extended and Applebot-Extended, which are the equivalent training crawlers from other operators.
- Allow OAI-SearchBot, PerplexityBot, Claude-SearchBot and DuckAssistBot. These are the answer engines.
- Allow ChatGPT-User and Perplexity-User. These are live fetches on behalf of a person.
- Do not rely on a wildcard rule to express any of this. A `User-agent: *` disallow that you meant for scrapers will apply to all of the above.
- Watch for consecutive User-agent lines. Several agents listed together before one Disallow share that rule — which is how GPTBot and OAI-SearchBot end up blocked by a line someone wrote for one of them.
Check the CDN, not just the file
The rule you can read is rarely where the block lives. When we checked thirty well-known DTC brands, not one blocked answer engines in robots.txt — but four refused an identified crawler outright with a 403 or 429, and three failed to connect at all.
That decision sits in a bot-management ruleset at the CDN or WAF, usually added by someone protecting the origin from scrapers long before AI referral traffic was a consideration. It does not appear in any robots.txt audit, and the person who would notice the missing traffic has no visibility into the console where it was configured.
The question worth asking whoever administers your edge is specific and has a definite answer: are OAI-SearchBot, PerplexityBot and ChatGPT-User on the allowlist?
Frequently asked questions
- Does blocking GPTBot hurt my SEO?
- No. GPTBot is a model-training crawler and has no effect on Google Search rankings or on whether ChatGPT surfaces you in search results. Those come from Googlebot and OAI-SearchBot respectively — different agents entirely.
- If I block GPTBot, will ChatGPT still find my products?
- Yes, provided OAI-SearchBot and ChatGPT-User are allowed. ChatGPT search is built from a separate index and live fetches, neither of which depends on training crawls.
- Which one shows up when someone asks ChatGPT about my brand?
- Usually ChatGPT-User, fetching your page live for that specific question, and OAI-SearchBot for whatever was already indexed. If you see ChatGPT-User in your logs, a real person is asking about you at that moment.
- Is there an equivalent split for other AI companies?
- Yes, and it follows the same shape. Anthropic runs ClaudeBot for training and Claude-SearchBot for search; Perplexity runs PerplexityBot for indexing and Perplexity-User for live fetches; Google separates Google-Extended (training) from ordinary Googlebot. The pattern is consistent enough to write a policy around.