Skip to content
New storyWe built Skald to count visitors. It caught an AI-key thief.
SkaldEvery visit, truly told.

Insights · OpenAI's crawlers

GPTBot vs ChatGPT-User: a training crawl, a live fetch, and OAI-SearchBot in between

GPTBot vs ChatGPT-User is the question behind most AI traffic confusion. Both come from OpenAI and both read your pages, but one reads to train a model and the other reads because a person just asked ChatGPT something.

GPTBot is OpenAI's training crawler: it reads pages ahead of time and nobody is waiting. ChatGPT-User is the fetcher ChatGPT sends when a person's question needs your page right now. OAI-SearchBot crawls ahead of time too, to build the index ChatGPT's search draws on.

By · Updated · 7 min read

Three OpenAI names, three different jobs

Search a busy site's access log for OpenAI and you will usually find three names in the user agent: GPTBot, OAI-SearchBot and ChatGPT-User. People often add them up and call the total "ChatGPT traffic". That number means nothing, because the three do different jobs.

NameWhy it comesIs a person waiting?What it tells you
GPTBotTraining crawl: collects pages to train OpenAI's modelsNoYour pages are in the pool a model may learn from
OAI-SearchBotIndex crawl: builds the index ChatGPT's search draws onNoYour pages can be found when ChatGPT searches
ChatGPT-UserLive fetch: reads a page because someone asked a question nowYesSomeone's answer was being built from your page

There is a fourth kind of OpenAI traffic that never names OpenAI at all: the person who reads ChatGPT's answer and clicks the link. Their browser is an ordinary browser. What gives them away is the referrer, chatgpt.com.

GPTBot: the training crawler

GPTBot reads pages ahead of time, many at once, and comes back to pages it has read before. In a log it looks like this (sample data):

203.0.113.24 - - [22/Oct/2026:02:10:07 +0000] "GET /blog/pricing-pages/ HTTP/1.1" 200 21544 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)"
203.0.113.24 - - [22/Oct/2026:02:10:09 +0000] "GET /blog/ HTTP/1.1" 200 14210 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)"

Note the hour and the pace: it comes when it likes, and nobody is on the other end. A GPTBot request is not a reader. It is a sign your page may end up in what a model knows, with no link back to you. It also costs your server something: requests, bytes and answer time, which are worth watching per crawler.

OAI-SearchBot also reads ahead of time, but for a different reason. It builds the index that ChatGPT's search looks things up in, much as Googlebot does for Google. It crawls ahead of any question, so it is not a reader either.

198.51.100.61 - - [22/Oct/2026:05:44:31 +0000] "GET /docs/setup/ HTTP/1.1" 200 9876 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot"

Because nobody is waiting on it, Skald files OAI-SearchBot with the crawlers, as a training crawl in the wide sense: reading to train or to build an index. When you set crawler rules, though, it is kept apart from GPTBot, so blocking training does not take you out of search. More on that below.

ChatGPT-User: the live fetch

ChatGPT-User is the one that matters most and gets the least attention. It fetches a page because a person asked ChatGPT something and the answer needed your page, right now.

198.51.100.140 - - [22/Oct/2026:09:01:13 +0000] "GET /pricing/ HTTP/1.1" 200 18342 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot"
198.51.100.140 - - [22/Oct/2026:09:01:14 +0000] "GET /compare/ HTTP/1.1" 200 15020 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot"

The pattern is different. In the sample: two pages, a second apart, a pricing page and a comparison. Those are the pages an answer was built from. Every ChatGPT-User line is a reader one step removed: a real person asked, and your page was part of what they were told.

It still is not a visit. The person may read the answer and never click. So ChatGPT-User belongs with the machines, counted in requests, not with people.

The click-through: chatgpt.com in the referrer

When the person does click a link in the answer, their own browser arrives:

198.51.100.23 - - [22/Oct/2026:09:02:40 +0000] "GET /pricing/ HTTP/1.1" 200 18342 "https://chatgpt.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.0 Safari/605.1.15"

That is a person, with ChatGPT as their source. Look at the timing in the sample: ChatGPT-User fetched /pricing/ at 09:01, and a person landed on /pricing/ from chatgpt.com at 09:02. The fetch and the visit are two records of the same answer.

Clicks also arrive in other shapes. The ChatGPT Android app sends its package name, com.openai.chatgpt, as the referrer. Some links carry utm_source=chatgpt. And many arrive with no referrer at all, as a plain direct visit. That last group is why every count of AI clicks is a floor.

How to count each one in your own log

With shell access, each is one line on an nginx or Apache log in the combined format:

# training crawl and index crawl
grep -c "GPTBot" /var/log/nginx/access.log
grep -c "OAI-SearchBot" /var/log/nginx/access.log

# live fetches, and the pages they read
grep "ChatGPT-User" /var/log/nginx/access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head

# people from ChatGPT, by referrer (requests, not visits)
awk -F'"' '$4 ~ /chatgpt\.com|chat\.openai\.com/' /var/log/nginx/access.log | wc -l

Before you trust any of it, check the addresses. Anyone can write GPTBot in a user agent, and scanners do. OpenAI publishes the address lists of all three (for example openai.com/gptbot.json, openai.com/searchbot.json and openai.com/chatgpt-user.json). A GPTBot line from an address that is not on the list is a fake. The steps are in how to see AI bots in server logs.

Which one to block, if any

Because the three are separate names, you can treat them apart. Most sites want to stay in ChatGPT's answers, so they let ChatGPT-User and OAI-SearchBot in. Some do not want their pages used for training, so they keep GPTBot out. Both at once is possible.

The polite way is robots.txt:

User-agent: GPTBot
Disallow: /

robots.txt asks; it does not enforce. A server rule enforces. In Skald, the crawler rules give each AI company three choices: allow, block training only, or block. For OpenAI, block training only refuses GPTBot and lets OAI-SearchBot and ChatGPT-User through. A blocked crawler gets a 403 for everything except robots.txt, so it can still read your rules. The rules are applied by nginx, after your server admin sets them up, and always tested with nginx -t before a reload.

A rule by user agent stops crawlers that say who they are, which the big ones do. It does not stop a scanner that lies, but then a scanner that lies was never GPTBot. See block AI crawlers for every name and choice.

Four common mix-ups

Most wrong conclusions about ChatGPT traffic come from one of these.

  • Adding the three names into one figure. A night of GPTBot crawling can be hundreds of requests. A morning of ChatGPT-User can be a dozen. Added together, the crawl drowns the signal you care about: people asking ChatGPT about you.
  • Counting ChatGPT-User as visits. A fetch is a reader one step removed, not a person on your site. Count it in requests, beside the people from chatgpt.com, never inside them.
  • Trusting the name. A GPTBot line that asks for /wp-login.php or /.env is a scanner wearing a badge. Until the address is checked against OpenAI's list, a name is only a claim.
  • Thinking a GPTBot block hides you from ChatGPT. It keeps your pages out of training. OAI-SearchBot and ChatGPT-User are separate names with their own rules, so ChatGPT can still find and read your pages when someone asks, unless you block those too.

The fix for all four is the same: keep each name in its own column, in its own unit, and check the address before you believe it.

What Skald shows for each

Skald reads the same log and keeps the four kinds apart, all the way through:

  • GPTBot and OAI-SearchBot on the Crawlers tab, checked against OpenAI's lists: verified, not verifiable, or "claims to be GPTBot" under bots. With each crawler's cost: requests, bytes, answer time, errors and re-crawls.
  • ChatGPT-User on the Assistants tab, with the pages it fetched and what it asked for: pages, llms.txt, Markdown, plain text.
  • People from ChatGPT as people, with ChatGPT as their source, from chatgpt.com, the Android app and tagged links.
  • Likely from ChatGPT: a direct visit that lands on a page ChatGPT-User fetched up to 15 minutes before. An estimate, marked likely, never added to the count.

On each visit from ChatGPT, Skald also lists the pages ChatGPT-User fetched in the 15 minutes before, nearest first: the 09:01 fetch and the 09:02 visit in the sample above, joined. It is context, not a count of people, because a fetch can be for someone who never clicked. All of it is in ChatGPT traffic, and every screen is in the live demo with sample data.

The short version: GPTBot reads to learn, OAI-SearchBot reads to be found, ChatGPT-User reads because someone asked, and a click from chatgpt.com is a person. Count them apart, and the numbers start to mean something.

questions

GPTBot vs ChatGPT-User FAQ

Straight answers, including what Skald does not do.

What is the difference between GPTBot and ChatGPT-User?

GPTBot is OpenAI's training crawler: it reads pages ahead of time to train models, and nobody is waiting on it. ChatGPT-User fetches a page because a person asked ChatGPT a question just now, so a reader is waiting at the other end.

What is OAI-SearchBot?

OAI-SearchBot is OpenAI's search crawler. Like GPTBot, it reads pages ahead of any question, but to build the index ChatGPT's search draws on, not to train a model. Skald counts it with the crawlers, because nobody is waiting on it.

Should I block GPTBot?

Only if you do not want your pages used to train OpenAI's models. Blocking GPTBot does not stop OAI-SearchBot or ChatGPT-User, which are separate names, so you can keep training out and stay readable when someone asks ChatGPT.

Does ChatGPT-User mean someone visited my site?

No. It means ChatGPT read your page while answering someone. The person may never click. A visit is a person whose browser arrives, often with chatgpt.com as the referrer, and Skald counts those as people.

$ tail -f access.log | skald

See what AI does with your site. Free for 30 days.

No card. We set up hosted Skald for you within one working day, or you self-host it free. Every line of your log, sorted and added up.

Start free trial Live demo