Markdown copies for AI: what a .md request gets back, and why page views fell

Why AI asks for page.md at all

A Markdown copy is the same page as plain text: the heading, the paragraphs and the lists, with no menu, no footer and no scripts. It is cheaper for a machine to read than the HTML around it, so AI tools ask for one when they know where it is. In a log, the request is your page's address with .md on the end: /pricing.md, /guides/setup/index.md, sometimes .mdx or .markdown.

They learn the address in two ways that show up in a server log. Some plugins and site builders redirect AI assistants from a page to its .md copy, so the log shows a 301 or 302 on the page and, a second later, a request for the copy from the same address. And an llms.txt file, the plain list of a site's pages for AI that our llms.txt page explains, may link the Markdown copies instead of the pages. The proposal asks for that: “The links in an llms.txt file should therefore point to LLM-friendly content, such as the markdown versions of pages” (the llms.txt proposal at llmstxt.org, read 25 Oct 2026).

Since 24 October 2026 every page of skaldanalytics.com has a Markdown copy (/features/ is also /features.md). The log names the copy asked for, not how the asker found it. When Skald 2.35.0 recounted our log, 874 page views turned out to be .md fetches, 164 of them from the AI assistants group: AI crawlers and the fetchers that read a page for someone's question.

How to find .md requests in your own log

You do not need Skald to see how often this happens. On an nginx or Apache log in the usual combined format, the path is the seventh space-separated field and the status the ninth. These commands use sample output from a made-up site.

How many requests asked for a Markdown copy:

grep -cE '"(GET|HEAD) [^ ]*\.(md|mdx|markdown)[ ?]' /var/log/nginx/access.log
312

What the server answered them:

awk '$7 ~ /\.(md|mdx|markdown)(\?|$)/ {print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn
    271 200
     38 404
      3 301

Which copies answered 404, most asked first:

awk '$9 == 404 && $7 ~ /\.(md|mdx|markdown)(\?|$)/ {print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head
     14 /guides/webhooks.md
      9 /blog/old-launch-post.md
      6 /docs/api/index.md

And who asked. Splitting the line on the double quote puts the request in field 2 and the user agent in field 6:

awk -F'"' '$2 ~ /\.(md|mdx|markdown)[ ?]/ {print $6}' /var/log/nginx/access.log \
  | grep -oE 'GPTBot|OAI-SearchBot|ChatGPT-User|ClaudeBot|Claude-User|PerplexityBot|Perplexity-User' \
  | sort | uniq -c | sort -rn
     88 ClaudeBot
     61 GPTBot
     40 PerplexityBot
     23 ChatGPT-User

Two cautions. A user agent is whatever the caller writes, so a scanner can call itself GPTBot; Skald checks such a claim against the address lists the AI companies publish, where they publish one, and files a fake under bots. And these one-liners read one file: a log that rotates daily needs zcat over the older files too. Seeing AI bots in server logs goes further with the same tools.

When the copy answers 404, the assistant gets nothing

The costly case is a redirect to a copy that was never made. The plugin sends the assistant away from a page that works, to an address that does not exist:

198.51.100.14 - - [24/Oct/2026:10:02:11 +0000] "GET /guides/webhooks/ HTTP/1.1" 302 0 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)"
198.51.100.14 - - [24/Oct/2026:10:02:11 +0000] "GET /guides/webhooks.md HTTP/1.1" 404 162 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)"

Your page answered. It just never reached the assistant. A script-tag analytics tool sees neither line, because no browser ran a script.

Skald shows it in three places. A page's story has an AI readiness block, and its Markdown copy line says "AI asked for the Markdown copy 6 times; it answered 404 each time", with what to do. AI readiness check walks through all six lines. Site › Not found has a tab, AI asks for Markdown, listing the .md addresses AI got a 404 for, each with the live page it belongs to when there is one. And when AI asked at least 5 times in the last 30 days for copies of pages that are live, Overview › Insights adds a to-do: "Give AI the Markdown copies it asks for".

Skald files a copy under its page by the address alone: /a/b.md, /a/b/index.md and /a/b/.md all belong to /a/b/, and /index.md to the home page.

Why your page views went down in Skald 2.35.0

If your page views fell the day you upgraded, this is usually why. Before 2.35.0, .md was not on the list of file endings Skald keeps out of page views (stylesheets, scripts, images, .txt, .xml and the like). So every Markdown fetch was a page view, in whatever group made it.

2.35.0 added a list of machine and utility addresses that are not pages, on every site by default:

A request for one of them still counts as a request, in its usual group, so the five groups still add up to the log. It leaves the page views, the readers, the landing pages, the exits and the journeys. Site › Data quality shows each day's Not counted as pages, by entry, and Site › Requests marks each such line "a machine address, not a page".

On skaldanalytics.com the recount moved page views in all groups from 6,426 to 5,551. 874 of the 875 that left were .md fetches. People's page views stayed at 1,152: no person had read a Markdown copy. The counting rule, and its second count that matches the same list with code of its own, are in Skald's 2.35.0 release notes (25 Oct 2026).

Serve it, redirect it or leave it

Skald only reports; what the server answers is your choice. There are three honest answers to a .md ask, and each shows differently in Skald afterwards.

# inside server { }: send .md files as Markdown
location ~* \.(md|mdx|markdown)$ {
    types { text/markdown md mdx markdown; }
}

# inside server { }: send .md asks to the page itself
rewrite ^/(.*/)?index\.md$ /$1 permanent;
rewrite ^(/.+)\.md$ $1/ permanent;

Whichever you pick, mark the to-do done in Overview › Insights. Skald then re-checks it on every whole day after the mark, with the same measure that raised it.

See what AI gets from your pages, 30 days free →

Counting Markdown as pages again, if you want to

Some sites publish Markdown on purpose for people: a docs site whose readers open README.md in the browser, say. For them a .md address is a page. Settings › Sites › Not pages lists every entry with a tick; untick Markdown copies and they count as pages again from the next day counted. The same screen has Also not pages for your own machine addresses, up to 20, each written one of three ways: /path for one address, /prefix* for everything under it, or ?key for any address with that query key.

Stored days keep the figures they were counted with. To apply a change to the past, count the days again from the raw logs you still keep: skald backfill --days 14 --force for each site, then skald audit, as the 2.35.0 upgrade notes say.

What the log cannot tell you

A log line says that a copy was asked for, by which name, and what the server answered. It does not say what the assistant did with the text, whether it quoted your page to anyone, or whether a person ever saw the answer. The fetch is a sign of demand; the clicks from AI answers, counted separately, are the visits it sent.

It also cannot say where a redirect led, because the destination is in a response header the log does not keep. That is why Skald names redirects on a page's AI readiness block without counting them as a problem, and judges the Markdown copy on its own line. And it says nothing about the copy's content: a copy that answers 200 with an empty file still looks fine. Open one or two in your browser after you fix them.