Insights · notes from the log
AI traffic insights: practical notes from the server log
AI traffic insights you can check on your own server: how to find AI crawlers in a log, what each OpenAI name means, and why your analytics shows so little of it.
AI traffic insights are short, practical notes on what AI does with a website, written from what the access log really records: training crawls, live fetches and the people AI sends. Each one has commands you can run yourself, with sample data. The full method is in the AI traffic guide.
30-day free trial · no card · we set it up within one working day · free to self-host
[01] start here
AI traffic notes: start with the question you have
Three notes so far, each with commands you can run on your own log. Pick the question closest to yours.
How to see AI bots in server logs
Find GPTBot, ClaudeBot and ChatGPT-User in an nginx or Apache log with grep, see which pages each read, and check that each one is real.
grep -oE "GPTBot|ClaudeBot|ChatGPT-User" access.log | sort | uniq -c → requests by name
GPTBot vs ChatGPT-User
A training crawl, a live fetch for someone's question, and OAI-SearchBot's index crawl in between. What each means in your log, and which to block.
198.51.100.140 "GET /pricing/" "ChatGPT-User/1.0" → someone asked, just now
Why Google Analytics misses AI crawlers
No script runs, known bots are dropped on purpose, and AI clicks lose their referrer. What that leaves out, and where to find it.
203.0.113.24 "GET /pricing/" "GPTBot/1.2" → no hit sent to Google
in this section
Every note, newest first
keep reading
More on AI traffic insights
questions
AI traffic insights FAQ
Straight answers, including what Skald does not do.
Who writes the Skald insights?
Rakesh Patel, CEO of Space-O Technologies, who built Skald for Space-O's own small-business site when its analytics could not say whether ChatGPT, Claude or Perplexity read the pages. About Rakesh Patel.
Do I need Skald to follow these notes?
No. Every note shows how to check it yourself with grep and awk on an nginx or Apache access log. Skald is for doing it every day, with names checked and every figure counted twice.
Is the data in the notes real?
No. Every log line and figure in the notes is sample data, with addresses from the ranges set aside for examples. Run the commands on your own log to see your own numbers.
$ tail -f access.log | skald
See what AI does with your site. Free for 30 days.
No card. We set up hosted Skald for you within one working day, or you self-host it free. Every line of your log, sorted and added up.