Solutions · SEO teams
AI traffic analytics for SEO teams: Search Console, crawlers and real visits on one screen
AI traffic analytics for SEO teams puts Google's and Bing's own figures beside the visits your server really answered, and shows what Googlebot, bingbot and every AI crawler fetched, page by page.
AI traffic analytics for SEO teams is log file analysis with search and AI in one place: Search Console and Bing queries, crawl coverage per crawler from the access log, and the AI crawlers and assistants that read the site, each checked against its owner's published addresses. Built on server log analytics.
30-day free trial · no card · we set it up within one working day · free to self-host
[01] crawl health
Crawl budget, per crawler: what Googlebot, bingbot and each AI crawler fetched
Your sitemap beside your access log. For each crawler: the share of sitemap pages it fetched in the window, the ones it never reached, the pages it fetched that you do not list, and the errors it met. Part of AI traffic analytics by role.
search crawler · verified by DNS. Sample data, last 30 days and today.
Five problem lists sit under it: not found, server errors, redirects, slow answers (a second or more) and large bodies (1 MB or more), each with the crawlers that met it and whether the page is in your sitemap. A page no crawler reaches is a page no search engine or assistant can show.
[02] search, beside real visits
Search Console and Bing Webmaster figures beside the visits your server answered
Your log cannot see a search: what someone typed, how often you were shown, where you ranked. Skald syncs it from Google and Bing every 6 hours and puts it on Traffic, Search.
Queries, pages, countries, devices and search appearance, with clicks, impressions, CTR and position, for 7 days, 28 days, 3 months or 16 months. Rare queries Google leaves out show as Not given by Google, never hidden. The newest days stay marked provisional until Google calls them final.
- Search Console clicksGoogle · 214
- Skald visits from Google searchyour log · 188
- Ratio this week0.88 · steady
- Ratio last week0.86
Sample data. Never forced to match: what matters is the ratio over time.
[03] the cross-check
Why Google's clicks and your visits differ, and when the gap is a problem
The cross-check puts Search Console's clicks beside Skald's visits from Google search, by day and landing page. When the ratio moves sharply, the panel says so. That is usually a tracking break: a page that stopped loading, or a redirect that drops the referrer.
A click is not a visit
Two clicks by one person can be one visit. A click closed before the page drew is no visit at all.
Lost referrers
Some browsers and apps send no referrer. Skald counts those visits as direct, never as Google.
Pacific days
Google's days are Pacific Time. Yours may not be. One day differs more than a week does.
Checked by a second road
Every stored table is added up again, and one day is asked of Google again. If they disagree, the tab says so.
Connecting takes no shell. An install admin uploads a Google service-account key once. For each site, add that read-only robot to Search Console as a Restricted user, pick the property, test, and fetch up to 16 months of history. For a client's site, Skald writes the request for you to send. Bing needs one API key.
[04] AI crawlers
AI crawlers for SEO: checked, priced, and kept to your rules
Every crawler is named and checked against its owner's published addresses or DNS: verified, not verifiable, or spoofed. A fake GPTBot is filed under bots as "claims to be GPTBot", never as AI.
Crawler cost
Per crawler: requests, bytes, mean and p90 answer time, errors, distinct paths and re-crawls.
robots.txt beside the log
Whether robots.txt lets each crawler in, and how often it fetched what the rules forbid.
Crawler rules
Per AI company: allow, block training only, or block. Applied by nginx, once your server admin sets it up. robots.txt stays readable.
Suggested llms.txt
Built from the pages people and assistants read most over 90 days. You check it and publish it; Skald never writes to your site.
[05] what changed
SEO alerts from the log: AI starts reading, a crawler goes quiet
Alerts run on whole, checked days only. Each states its rule and its figures, in visits or in requests, never both.
- AI started reading a page
- AI stopped reading a page
- First AI citation
- A page's first AI citation
- Crawler gone silent 7 days
- 404 surge
- 5xx burst
- Sep 02 first Googlebot fetchsearch
- Sep 06 first AI crawler: GPTBotAI · training
- Sep 09 first person, from google.compeople
- Oct 04 first leadlead
Sample data. The page story follows one page from its first crawl to its first lead.
[06] what it does not do
The limits of Skald for SEO, said plainly
What the log and the engines' own figures can and cannot prove.
What it does not do
- No rank tracking of its own. Queries, impressions and position are Google's and Bing's, synced, never recounted.
- Clicks from Google AI Overviews cannot be told apart. AI Mode can, when the click carries
udm=50. - A user agent is self-reported. Crawler rules stop crawlers that say who they are; the address check catches fakes.
- Crawler rules work only through nginx, after your server admin sets them up. Some names, such as Google-Extended, are robots.txt only.
- Slow and large answers need the response time and body size in your log. Redirect chains are counted one redirect at a time.
keep reading
More on AI traffic analytics for SEO teams
questions
AI traffic analytics for SEO teams FAQ
Straight answers, including what Skald does not do.
Does Skald replace Search Console?
No. Search Console still owns queries, impressions and position. Skald syncs those figures from Google and Bing every 6 hours and shows them beside the visits it counted from your log, so you see both on one screen. The search figures stay Google's and Bing's, never forced to match.
Why don't Search Console clicks match Skald's visits?
A click is not a visit: two clicks by one person can be one visit, and a click closed before the page drew is none. Some browsers send no referrer, so those visits count as direct. Google's days are Pacific Time. Watch the ratio over time: Skald flags it when it moves sharply.
Can Skald show Google AI Overviews?
No. Google's API does not give AI Overviews or AI Mode apart yet, and an AI Overview click looks like any Google click. Skald labels a click google.com (AI Mode) only when its address carries udm=50. Google AI Mode traffic.
Does crawl health need a sitemap?
Coverage and orphan pages need one: Skald reads the sitemaps named in robots.txt, or /sitemap.xml, once a day. Without a sitemap, the problem lists still work: 404s, server errors, redirects, slow answers and large bodies that crawlers hit.
Can I block AI training crawlers and stay in AI answers?
Yes, where your server admin has set up crawler rules through nginx. Per AI company you choose allow, block training only, or block. Block training only keeps out crawlers such as GPTBot and lets search indexes and live fetchers in. Some names, such as Google-Extended, are robots.txt only. Block AI crawlers.
$ tail -f access.log | skald
See what AI does with your site. Free for 30 days.
No card. We set up hosted Skald for you within one working day, or you self-host it free. Every line of your log, sorted and added up.