Why some tools pass for people
Every request in an access log carries a user agent, the line where the client names itself. Anyone can write anything there, and a scraper can copy a browser's word for word. So a user agent alone proves nothing, and Skald asks for more before it counts a visit as a person: evidence that a browser drew the page. The usual evidence is that the visit went on to fetch the page's own scripts and styles a moment later, which a simple scraper never does.
That test catches the scrapers that take the HTML and leave. It cannot catch a tool that drives a real browser, because that tool does load the scripts and styles. Skald 2.35 added a rule for three such tools, each of which had been counted as people: a link checker, a scanner walking through a list of admin pages, and an attack hidden by encoding. The hero above replays one sample visit of each.
None of the three is caught by how long it stayed or how many pages it read. Each is caught by one kind of request that no reader sends.
The link checker sends HEAD for every link on the page
A link checker opens a page, then tests every link on it to see whether it still works. To test a link cheaply, it sends HEAD: a request for the headers only, no page body. A browser opens a page with GET.
So the rule, numbered 8b in Skald's classification notes: a browser-like visit that sent HEAD for two or more different pages (not images, scripts or framework payloads) is a tool, and every request of that visit goes to bots as "browser-like link checker (HEAD requests)". Two, not one, because a page's own script may check a single address with HEAD, and that visitor is still a person.
198.51.100.40 - - [21/Oct/2026:10:02:11 +0000] "GET /services/ HTTP/1.1" 200 18210 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) ... Chrome/129.0 Safari/537.36"
198.51.100.40 - - [21/Oct/2026:10:02:12 +0000] "HEAD /pricing/ HTTP/1.1" 200 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) ... Chrome/129.0 Safari/537.36"
198.51.100.40 - - [21/Oct/2026:10:02:12 +0000] "HEAD /old-offer/ HTTP/1.1" 404 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) ... Chrome/129.0 Safari/537.36"Left alone, a tool like this can make one "visit" with hundreds of page views, because every link it tests is a page. It also makes your most linked-to pages look like your most read ones.
The probe sweep knocks on five doors that do not exist
A probe is a request for a path that attackers try on every site: /wp-login.php, /phpinfo.php, /admin/ and many more. Skald already filed each probe under bots. But some of these paths are real on some sites (a WordPress site has a login page), so a single probe moved only itself, and the plain pages the same scanner read between probes stayed people.
The new rule, 6c: an address and agent that asked for five or more different probe paths the site answered with a 4xx or 5xx (not found, forbidden, an error) is a scanner for the whole day, filed as "scanner (asked for many missing probe paths)". A reader may ask for one of those paths; asking for five the site lacks is what a scanner's list does.
Three things never count towards the five, so real people stay people: redirects (a WordPress admin at work gets 302 answers from wp-admin), files such as images (a page that still embeds an old site's images makes the browser ask), and paths you list as your site's old addresses in its settings.
An attack in disguise, its dots encoded twice
Path traversal is an attack that asks the server for a file outside the website, written with ../ steps, such as ../../etc/passwd. Filters look for the dots and slashes. So attackers encode them: %2e is a dot, and %252e is a dot encoded twice, because %25 is the percent sign itself.
203.0.113.77 - - [23/Oct/2026:14:20:05 +0000] "GET /?page=templates%252f%252e%252e%252f%252e%252e%252fetc%252fpasswd HTTP/1.1" 200 9120 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) ... Safari/605.1.15"Skald's list of attack strings read %2e%2e but not %252e%252e. It now reads the dots and slashes of ../ and /etc/passwd encoded any number of times. Once an address and agent send one, they are a scanner for the day (rule 6b), every request of theirs, form posts included. That last part matters if you count leads from form posts: an attack that posts to your home page every few seconds is a very busy "lead".
Being a scanner in Skald's count is a label on traffic. Skald shows who probed; it does not stop them, and it does not tell you whether an attack worked. That is for your server, your host and your security tools.
A crawler's name proves nothing either
The same trick works with crawler names. A scanner writes Googlebot or OAI-AdsBot in its user agent, and a tool that believes the name files its probes under search engines or AI crawlers. Skald checked the name against the owner's published address lists, but only on servers where those lists had been fetched, and on Docker installs they had not been, until 2.35 switched the check on there.
So 2.35 added rule 2b: a request that names a crawler and is scanning (the request is a probe, or the same address and agent asked for secret files that day) is filed under bots, unless its address is proven to be the owner's. A proven fake still shows by name, as "claims to be OAI-AdsBot". The crawler checks themselves are on the AI crawler tracking page.
What we left alone, on purpose
The easy rule would have been "a visit of one page in three seconds or less is a bot". It would have made every people count much smaller, and it would have been wrong: real people open one page and leave all the time. Short visits stay people.
The same goes for visits from hosting and cloud networks. Some come from scrapers, but VPNs exit there too. Skald marks them with a hosting-network flag so you can look, and keeps them as people. A rule earns its place by what a visit asked for, never by a guess about who tends to sit behind an address.
Likely from AI, minus the visits chance explains
Bots that look human are one way a count drifts. An estimate that credits too much is another. Some people who click a link in a ChatGPT or Perplexity answer arrive with no referrer, so Skald marks a direct visit as "likely from an AI assistant" when that assistant fetched the same page in the same minute or up to 15 minutes before. That works for a quiet page. It fails for a home page that an assistant fetches all day: its fetch windows cover so many minutes that ordinary typed-in visits land inside them by chance.
Since 2.35, for each assistant and page of a day, Skald works out how many direct visits chance alone would put in those windows: the page's direct untagged visits times the minutes the windows cover, over the 1,440 minutes of a day. At a quarter of a visit or more, that day's fetches of that page count as no evidence. Try it below.
pick a page
1 visit × 48 minutes ÷ 1,440 = 0.03 chance visits. Under 0.25: the fetches count as evidence.
On skaldanalytics.com this rule took "likely from an AI assistant" from 54 visits to 1 over 18 to 24 October, most of the 54 on our free check page, which assistants fetch often and which gets many direct visits. The figure was an estimate all along, kept apart from the real counts; now it is a smaller and more honest one. Chance still cannot be ruled out for any single visit, so the label stays "likely".
What it changed on our own site
We recounted every stored day of skaldanalytics.com from its raw log with the new rules. The figures below come from Skald's 2.35.0 release notes (25 Oct 2026).
| skaldanalytics.com | Days | Before | After |
|---|---|---|---|
| Readers (people's visits that read a page) | 12 to 24 Oct | 753 | 752 |
| People's page views | 12 to 24 Oct | 1,198 | 1,180 |
| People's form posts | 12 to 24 Oct | 479 | 479 |
| Page views, all groups (machine addresses out) | the days the log held | 6,426 | 5,551 |
| Likely from an AI assistant | 18 to 24 Oct | 54 | 1 |
For our own site the tool rules moved little: one reader and 18 page views. That fits a small site whose team runs no link checker against it. The machine-address rule moved more, because 874 of the views it took out were AI assistants, crawlers and other tools fetching the .md copies of our pages, which the note on Markdown copies for AI covers. A site with a busy scanner or a link checker of its own can see larger moves, which is why every figure gets the same treatment.
Check your own log in three commands
You can look for all three tools in an nginx or Apache log in the common "combined" format before you install anything. Each command prints the address and agent pairs worth a look (sample output).
Link checkers: pairs that sent HEAD for two or more different paths.
awk -F'"' '$2 ~ /^HEAD / {split($2,r," "); split($1,a," "); print a[1] "\t" $6 "\t" r[2]}' access.log | sort -u | cut -f1,2 | uniq -c | awk '$1 >= 2' | sort -rn | head
38 198.51.100.40 Mozilla/5.0 (Windows NT 10.0; Win64; x64) ... Chrome/129.0 Safari/537.36Probe sweeps: addresses with five or more different PHP or admin paths answered 403 or 404.
grep -E '"(GET|POST) [^ ]*(\.php|/admin/|/wp-login)[^ ]* HTTP[^"]*" 40[34]' access.log | awk '{print $1, $7}' | sort -u | awk '{print $1}' | uniq -c | awk '$1 >= 5' | sort -rn | head
12 203.0.113.9Encoded attacks: any request with a dot or slash encoded twice.
grep -iE '%25(2e|2f|5c)' access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head
64 203.0.113.77These are rough cuts: they skip the rendering test, the session gap (the pause after which a new visit starts), your old addresses and everything else the real rules read. They are enough to tell you whether your people count has a problem.
Every rule, counted twice
A new rule that moves people to bots is exactly the kind of change that can go wrong quietly. So each of these rules has a second road: separate code that finds the same pairs and visits by its own logic, sharing only the name lists with the main count. If the main count keeps people the second one does not, or the other way round, the day is withheld instead of shown. A visit moves whole, so the five groups (people, AI assistants, search engines, bots and scanners, your own traffic) still add up to every request in the log. The method is on the analytics data accuracy page.
Stored days keep their old figures until they are counted again from the raw logs: run skald backfill --days 14 --force for each site (as many days as the logs still hold), then skald audit. To see your own people count without the costumes, start with the plans and the 30-day trial, or open the live demo with sample data.