set-up · aws cloudfront · standard logs
CloudFront log analytics, from standard logs
CloudFront log analytics means reading CloudFront's own standard logs, because a page served from CloudFront's cache never reaches your server. Skald reads them. Read the limits first: today those days are withheld.
CloudFront log analytics is counting the requests AWS CloudFront answered for your site, cache hits included, from the standard logs CloudFront writes to an S3 bucket. Skald reads them with the log format cloudfront_w3c.
30-day free trial · no card · we set it up within one working day · free to self-host
- 2026-10-23 09:14:02 203.0.113.24 GET www.example.com /pricing/ 200 - GPTBot/1.2AI · training
- 2026-10-23 09:14:03 198.51.100.7 GET www.example.com / 200 https://www.google.com/ Chrome/130people
- 2026-10-23 09:14:06 198.51.100.140 GET www.example.com /docs/ 200 - ChatGPT-User/1.0AI · live fetch
- 2026-10-23 09:14:09 192.0.2.91 GET www.example.com /.env 404 - zgrab/0.xbots · probe
Sample lines. A page CloudFront answers from its cache is in this log, and never in your server's own.
[01] read the cdn's log
Why CloudFront's log, not your origin server's
A site behind CloudFront is mostly served by CloudFront. A page in its cache never reaches your server, so your server's access log never sees that request. CloudFront's standard log sees every one, cache hits included.
Server log analytics then works as for any log: every line sorted into people, AI assistants, search engines, bots and scanners, and your own traffic. Read CloudFront's log only, never your server's as well: a cache miss is in both, and would be counted twice.
Read this first: what it does not do yet
- Days are withheld today. Every Skald figure is counted twice, the second time by an independent audit with its own reader. That reader does not read CloudFront lines yet, so the two counts cannot agree, and the daily run withholds those days rather than publish figures nobody checked.
- The one command on the server behind CloudFront works, and its days are checked, but it misses every page CloudFront answers from its cache.
- No country: CloudFront's standard logs carry none. A GeoIP file on Skald's server fills it in; without one it says Not recorded.
- The address is whoever connected to CloudFront. When a visitor came through a proxy, that is the proxy's address.
- Logs arrive late, usually within an hour, some up to 24 hours.
[02] in aws
Turn on CloudFront standard logging to S3
CloudFront writes small gzip files to a bucket. Skald reads one log and its rotations, so the files are copied to Skald's server and appended to one log.
Standard logging
Turn it on for the distribution, and choose Amazon S3 (Legacy) as the destination.
A bucket with ACLs
The bucket must have ACLs enabled. Give the logs a prefix.
Its file names
CloudFront writes
<prefix>/<distribution id>.YYYY-MM-DD-HH.<id>.gz, one set per hour.Format
cloudfront_w3cPoint Skald at the appended log with this format, or
auto, which picks it from the first lines.
Each file starts with #Version: and #Fields: lines. Skald skips them and never counts them as unreadable. Each line carries x-edge-request-id, so a file appended twice is counted once.
[03] what skald reads
The CloudFront log fields Skald reads
Skald reads the standard fields in their documented order, and follows a #Fields: line when a file lists them in another order. Click any part of this sample line.
cloudfront standard log · one sample line
click any partcs(User-Agent): what it claims to be
What it is
The name the caller gives itself, decoded once like the referrer.
What Skald does with it
Read against a named list and filed in one of five groups. This one: AI assistants, training crawl.
time-taken gives the request time, in seconds. A dash means empty. There is no country field.
[04] on skald's server
Get the CloudFront files to Skald's server
A short script, run from cron every 15 minutes, copies the bucket and appends each new file to one log. logrotate rotates it daily, as it does nginx's.
- Files are appended in name order. Their names carry the date and hour, which keeps visits whole.
delaycompresskeepsaccess.log.1plain, as nginx's rotation does.- If Skald runs on a hosting platform, run the same script on any machine and send the lines with the shipper.
#!/bin/sh
set -eu
RAW=/var/lib/cdn-logs/raw
OUT=/var/log/cdn/access.log
DONE=/var/lib/cdn-logs/done.txt
mkdir -p "$RAW" "$(dirname "$OUT")"
touch "$DONE"
aws s3 sync s3://my-log-bucket/my-prefix/ "$RAW" --only-show-errors
find "$RAW" -type f -name '*.gz' | sort | while read -r f; do
grep -qxF "$f" "$DONE" && continue
zcat "$f" >> "$OUT"
echo "$f" >> "$DONE"
done
logs: dir: /var/log/cdn base: access.log format: auto
[05] what you get
What Skald can show from a CloudFront log
Once a day's figures pass both counts, the same dashboard as any site: AI crawlers checked against their owners' addresses, live fetches, people and where they came from, scanners and your own traffic, apart.
Until the audit reads CloudFront lines, those days stay withheld. If you need checked figures now, read the server behind CloudFront with the one command, and know its figures leave out the cached pages.
keep reading
More on CloudFront log analytics
questions
CloudFront log analytics FAQ
Straight answers, including what Skald does not do.
Can Skald read AWS CloudFront logs?
Yes. Skald reads CloudFront's standard logs, the tab-separated W3C files CloudFront writes to an S3 bucket, with the log format cloudfront_w3c. It reads the standard fields in their documented order and follows a #Fields: line when a file lists them in another order.
Why are my CloudFront days withheld?
Every Skald figure is counted twice, the second time by an independent audit with its own reader. Today that reader does not read CloudFront lines, so the two counts cannot agree, and the daily run withholds those days rather than publish figures nobody checked. That is the honest state until the audit reads this format.
Can I read my origin server's log instead?
You can, with the one command on the server behind CloudFront, but pages CloudFront answers from its cache never reach that log, so its figures are too low. Never read both logs for one site: a cache miss is in both, and it would be counted twice.
Why do CloudFront visits have no country?
CloudFront's standard logs carry no country. Skald looks each address up in a GeoIP country file when one is set on its server. Without a file, the country says Not recorded, and it is never guessed.
How late do CloudFront logs arrive?
CloudFront usually delivers within an hour, and says some entries can be up to 24 hours late. Its lines are not always in time order, so Skald places each request by its own timestamp. Run the daily report late enough for yesterday's files to be in.
$ tail -f access.log | skald
See what AI does with your site. Free for 30 days.
No card. We set up hosted Skald for you within one working day, or you self-host it free. Every line of your log, sorted and added up.