The map nobody picks up — and whether it helps when it arrives.

Addendum · added 2026-08-17

Movement one

Where the file came from.

llms.txt was proposed in September 2024 by Jeremy Howard of Answer.AI: a markdown file at an agreed-on path, a title and a one-line summary and some lists of links, so that language models — whose context windows were then far too small to swallow a whole website — could get the site's shape in one cheap read.

What happened next is a story about conventions outrunning their spec. Documentation platforms adopted it in waves; one of them popularized a companion file, llms-full.txt — the entire site's content in one markdown blob — which appears nowhere in the original proposal (the proposal's own bundled files had other names). The convention grew anyway, and it grew without limit: the largest llms-full file we measured, from Anthropic's own developer documentation, now weighs 30.7 MiB — call it eight million tokens at four bytes each, an estimate rather than the measured count every other token figure on this page carries, and several times the million-token windows in common use. It is two hundred and forty-five times the context window the models on our own bench ran at. The file proposed because websites were too big for models has, in its largest instance, arrived back at the same problem.

And the price tag makes it concrete, at the very APIs the file was written for. One read of that single document — input tokens alone, before a word of reply — runs to about eighty dollars at Claude Fable 5's published rate ($10 per million input tokens), and about forty at Claude Opus 5 or GPT-5.5 (both $5 per million; rates as published 2026-08, and the dollar figures inherit the token estimate's error bars). The file invented to spare models the cost of reading a website now costs more to read once than most crawls it replaced. For scale: the two map files this bench actually served — the family front door's llms.txt and this site's own, which together are the whole of C-MAP — weigh 3,211 measured tokens between them, about three cents at the dearest of those rates. That is two files, not the estate: fourteen hosts here serve one, and nobody has ever concatenated them.

The search engines split. Google's Search documentation declines the file in writing (updated July 2026: Google Search “doesn't use them”); Google's Chrome quietly added a check for it to Lighthouse, the site-health scorer built into the browser (May 2026) — an audit that, read closely, treats a missing file as “not applicable” rather than a defect, which is a narrower endorsement than it looks. And this month the spec itself moved: version 2 (2026-08-10) added discoverability link relations — a way for ordinary pages to point at the map — which the spec's own changes page calls the most-requested addition, after two years in which nothing guessed the agreed path.

Receipts. The origin and the spec text: the Answer.AI proposal of 2024-09-03 and llmstxt.org v2, fetched 2026-08-13, with the platform origin of llms-full.txt in Mintlify's own documentation. Google's two positions are its Search Central guidance (updated 2026-07-10) and the Chrome Lighthouse audit documentation (updated 2026-05-05). The 30.7 MiB figure is our own HEAD request on 2026-08-16 — 32,161,074 bytes, converted at the four-bytes-per-token rule of thumb and marked as an estimate because it is one. Every source with its date and its quote ships in data/history-sources.md; that HEAD probe, beside what every other major publisher serves, ships in data/fetch-receipts.md.

Movement two

The confession.

We adopted llms.txt across this workshop's sites on August 13th. This article's own reconnaissance, three days later, found the estate serving seven real files, one phantom and ten 404s — and the front door of the whole family, strata2signal.com/llms.txt, answering 404.

The mechanism is named, with commits, because that is the house rule. Our static-site deploys swap the entire served directory for a freshly staged one; anything living on the server but not in the site's source tree — the master copy we publish from — is deleted by the next deploy, silently, with a green receipt. The llms.txt files had been placed on the server but not committed to every source tree — so the hosts that lost their file were exactly the hosts that were redeployed. One host was worse than missing: our game's invite door answers every path with the same welcome page, so it served a cheerful 200 — the web's code for “here it is” — for llms.txt, robots.txt, and a nonsense control path alike: a phantom file that inventory-by-status-code counts as present. The door stays a door by ruling, and the phantom is disclosed here instead of papered over.

All of it was fixed the same day, the correct way — every file that can live in a source tree now does, so a redeploy carries the map instead of eating it. The dated probe that closes this movement counts fourteen hostnames serving genuine markdown of the sixteen it reached, and the phantom is not one of them. The denominators move because the populations do: eighteen is every property the first pass censused — the seven, the phantom and the ten 404s, which is that pass's whole verdict, and the rule is that a nineteenth name answered 401 and is excluded as gated rather than miscounted as absent — sixteen is those that answered this probe, and movement three's twenty is the vhosts that keep logs. Eleven of the fourteen byte-match a source tree this probe can read; the other three live in repositories it does not scan, and one of those copies is still parked outside its site's directory. One host, asr.strata2signal.com, is still 404 as this publishes — its file was never written at all. All thirty-six distinct URLs the two map files assert resolve, and half of spec v2 now rides on the pages themselves: the describedby discovery relation, not markdown page twins, because we do not serve markdown twins and will not advertise ones that do not exist.

And one more beat, because the loop deserves its receipt: while hardening this article's plan, our own review panel accused the estate of a second drift — a stale model-count in the research hub's llms.txt. The check that runs before movement four's questions lock hunted it mechanically: zero of thirty-six distinct asserted URLs broken, zero of eleven figures contradicted. The map states nothing its page denies — but it does not mention the addendum either, so the charge lands as an omission rather than the contradiction this bench was built to catch. And the acquittal cost us a stratum: the bench had pre-registered at least one map/page contradiction to score, found none, and publishes that stratum VACANT — eleven of twelve sealing preconditions met, with the twelfth printed here rather than quietly. We had repaired the drift the stratum existed to measure, four days before the corpus froze. That is the honest shape of it: we fixed the evidence. A confession column that only ever confesses would be advertising too.

Receipts. Every count in this movement is read out of a dated artifact rather than typed: the seven-real-files inventory is the recon pass of 2026-08-16T13:40Z, and the fourteen-serving count is the content-classified host probe of 2026-08-16T14:41:23Z, published whole in data/estate-probe.json with every host's status, bytes and sha256. The 13:40Z pass's own working document does not ship, and the reason is the redaction policy rather than tidiness: it names the serving host's container, its log volume and its filesystem paths, which are exactly the estate detail that never reaches a public page here. Its surviving published record is the movement_two_estate_probe note in data/counting-rules.json, which carries the pass's timestamp and states in writing that the seven-real-files figure comes from it and not from the later probe — the two are three hours and one estate fix apart. The fix landed as four commits across four repositories — 9631d54, ea2f5b8fe, 7b5aadf, 4a73bd1 — each one named with its repository and its subject line in the kit README. The acquittal is data/map-stale-hunt.json; the eleven-of-twelve preconditions and the vacant stratum are in data/seal-manifest.json.

Movement three

Does anyone fetch it?

Our web server keeps a plain notebook of requests — one notebook per site, thirty days' retention, as the plain-english walk beside this one explains. We read them. In the window measured, named AI-associated crawlers made 9,483 requests to this estate. They asked for robots.txt — the thirty-two-year-old convention — 664 times. They asked for llms.txt zero times. Including the general search crawlers widens it to 10,878 requests, 956 robots.txt, and still zero; the headline figure is the AI-only one.

ClaudeBot asked our servers for robots.txt 161 times in thirty days. It asked for llms.txt zero times.

The published studies that measured this before us — Ahrefs across 137,210 domains, SE Ranking across roughly 300,000, Otterly's own 90-day server logs — found the same shape, on far more sites than we have. Our contribution is not scale: it is that these logs are ours, so the counting rule is ours to publish and ours to re-run.

Four honesty notes ride with the number, on the page and not in a footnote. The windows are per-log — our this site's own log begins 2026-08-11, so it contributes five days, not thirty. User-agent strings — the name each visiting program gives for itself — are self-reported, and we matched those names as plain text anywhere in the line, not as a parsed field. Our own files were about thirty-two hours old at measurement, so the claim we registered in advance — written down before we read a line of the logs — is “the path was never requested in the window”, not “our files were ignored”; though the sharper reading is that age barely matters here, because thirteen of these hosts serve no robots.txt either and crawlers asked for it anyway, 956 times. They probe a path that is usually missing. They never probed this one. And the fourth, at our own expense: of every llms.txt request these logs have ever held, ninety-one percent came from us — our probes, our audits, this article's own recon.

A path nobody asked us for is the finding, and it is exactly the finding the spec's v2 discoverability additions exist to answer. Our Monday estate audit now carries a fetch-count item, and it has not yet run once — so the commitment this page makes is the one it can keep by hand: we will re-run this exact count, by this exact rule, on 2026-09-16, and publish it here beside this figure, whether or not the zero moves.

The counting rule, verbatim, so the re-measure reconciles. Requests whose log line contains the UA token, one request per line, summed across the twenty vhost logs, each log covering from its own first entry to the read date. The rule and the per-crawler table ship in data/counting-rules.json, and the per-crawler table — all sixteen named crawlers, with the log method, the per-log windows and the honest limits — in data/fetch-receipts.md; the wiring for the Monday count landed in the estate audit's prompt rather than its spec, and has not yet been exercised on a real run, which is stated here rather than implied away.

Movement four

The bench: if the map arrives, does it help?

No AI crawler asked for ours in thirty days of logs. But every argument about llms.txt quietly assumes an answer to a different question: if you put it in front of a model, is it worth the tokens? That conditional had, as far as we can find, no published measurement. So we sealed one.

The shape. Eight models — four local quantized seats from 25.8 to 70.6 billion parameters on one box of ours, four frontier cloud, seven families, every tag and weight class printed in the table below, because a stacked panel is a thumb on the scale. No Claude arm sits, deliberately: the same pen wrote the map, wrote the questions, wrote the code that grades them and wrote this page, and seating a Claude arm would stack a conflict on a conflict. The absence is the disclosure.

Three conditions, matched on token budget — each block gets the same amount of text — because budget is the honest constraint: C-MAP (two llms.txt files — the family front door's and this site's, concatenated in that order with no added prose — 3,211 tokens on the one reference tokenizer both sides were measured against — call it eight pages of text), C-HTML (the site's own pages, text-extracted, front pages first, 3,088 tokens — deliberately the strongest control we could build, since our front page is structurally an llms.txt wearing HTML), and C-NONE (the bare URL and nothing else — the check on what the models already knew).

Twenty-four sealed questions, written and locked before a single model was asked: twenty with exact-match keys, four scoring abstention and stamped proxy, never added to the first twenty. They are checked by code we wrote and auditioned against paraphrases and decoys we also wrote — 335 cases, zero failures, and the whole audition ships in the kit so you can add the decoy we did not think of. The questions cover facts published on this site within the last seventy-two hours (our freshness armor: past every roster cutoff we could verify, by design), navigation items (“which URL answers X”), and refusal bait about pages that do not exist, where the right answer is saying so. One draw per cell at temperature zero, with a three-draw probe on six of the eight arms that found zero flips.

The armor held. The registered filter excluded zero of fourteen core questions: exactly one was ever answered from training alone — one item, by one arm, which is a lucky guess by the filter's own registered definition, and it stands flagged, published, and re-scored out in a sensitivity row at denominator nineteen. Seven arms voted on every item; the eighth, gpt-5.5, had its closed-book leg cut to 5 of 24 items by the budget event below, so on nineteen items the roster's most expensive arm never voted. What that filter removes is outright knowledge, not the chance that a served block cued something half-remembered — that interaction is invisible to it and stays a bound on everything below. And the flip side printed the same integrity: on questions whose answers appeared in neither context block, every arm scored zero, every time — no model retrieved silently. That stratum has a registered name and a registered floor: NEITHER, five of the twenty headline items against a pre-registered minimum of four, and it scored zero for every arm in both grounded conditions. Routing is not a stratum here at all — all six navigation items sit in MAP-ONLY, because the extractor keeps visible text and drops href attributes, so the HTML slice carries no address to route to by construction. The one stratum that never filled is MAP-STALE, registered to catch a map contradicting its own pages; the frozen corpus held no such contradiction, so it sealed vacant at zero items and publishes as a reduction rather than a re-scope.

The headline, both ways — because how we pulled the text out of our own pages decides it. On the registered reading, the map direction leads: across the discordant cells — the questions where one condition got it right and the other got it wrong, the only ones that carry information about a difference — 61.5% fell toward C-MAP (64 of 104 discordant pairs over a twenty-item headline set), just clearing the 60/40 line we registered in advance as the least lopsided split we would call a direction at all.

A reader should see what that number is made of before believing it: the map-only count came back as exactly 8 for all eight arms — the eight items the map states and the slice does not — and our answer-presence audit, computed before a model was called, had predicted every item's stratum correctly. 61.5% is 8 over 13: the ratio of map-only to html-only items in a set we wrote. The models agreed with a zero-call audit almost perfectly, which is a finding about retrieval before it is a finding about llms.txt.

That lead is carried entirely by the navigation items, and for a reason a hostile reader should know before believing us: our text extractor keeps the visible words and drops the web addresses behind the links, so the llms.txt block carried fifty web addresses into context — every http occurrence in the served block, thirty-six of them distinct, and those are the same thirty-six movement two resolved — while the HTML block carried none. Both counts are one grep on the published block. One reconciliation, because the kit prints two numbers for the control and a reader will find them: scores.json records urls_in_c_html_raw_regex: 2 beside urls_in_c_html_real: 0. The 2 is a looser pattern catching a bare host string, and both hits are the same footer email address, hello@strata2signal.com, once from each front page. Under the rule published above — occurrences of http:// or https:// in the served block — the control carries zero, and it has zero link targets. Both numbers are right; they count different things, and the headline uses the stated rule. The navigation questions were unanswerable from the control by the way we built it. Remove them — a cut we did not register in advance, made after seeing the direction it moves, and published for transparency rather than as a result — and the fact questions alone read 71.4% toward C-HTML (40 of 56 pairs, 14 items): at an equal token budget, the site's own prose answered more of them than the map that summarizes it.

Read together, the two numbers say something neither camp's slogan does: these two files behaved like a map, not an encyclopedia. Their facts lost to the site's own words at the same budget — on a cut we made after the fact, not on the registered reading — and their addresses were the only addresses in the room. If what a model needs is what's true here, budget spent on real pages beats budget spent on the summary. If what it needs is where things are — the thing an agent needs first — the map was the only block in the test that knew, which is a fact about the two blocks rather than a demonstrated navigation skill. That is a smaller claim than “every site needs one” and a kinder one than “cargo cult,” and it is the one our receipts support.

The honesty column. The refusal bait: sixty-three of sixty-four cells — one model, one condition, one question apiece — correctly said the page doesn't exist, with the caveat that the prompt gave them the sentence to say, so this scores whether they took the exit rather than whether they found it. The one failure invented a plausible feed URL, from the HTML control rather than from the map: the exact confabulation the bait was planted to catch, caught, and not where we expected to catch it.

And one cost story worth its sentence, because our own safeguard missed it: the priciest cloud arm ran a median 3,633 output tokens per question — several pages of prose, at real money — with no site content in front of it, composing elaborate ways to say “the site doesn't say.” The pre-leg probe we registered to price that arm had sampled a C-MAP-shaped prompt and measured a 58-token median. With nothing to ground on, the same model reasoned at four times our registered nine-hundred-token line, which is what halted the leg — at forty-six cents of a $2.60 cap it never came near. Nineteen cells of five hundred and seventy-six published as not-collected-for-budget rather than quietly absent. A probe that only samples the grounded condition is blind to the ungrounded one, and ours was. Saying “I don't know” cheaply is apparently its own capability.

What this bench does not say, in its own words: it does not measure crawling, fetching, adoption, or agent behaviour in the wild — movement three answered the fetching, and the answer was zero. It measures which representation of a site carries more answerable information per token when the text is handed to the model rather than fetched by it. The plan said so in those words, before the first reply existed, because pretending otherwise is how benchmarks become marketing.

Receipts. Every figure in this movement is a projection of data/scores.json — all 576 cells, the contamination exclusion with its rule, the three fences, the headline table, the pooled discordance and the post-hoc structural sensitivity. The questions and their keys are sealed in golden-set.sealed.json; the 335-case audition that qualified the checkers is checker-audition.json beside the checker module itself; every reply is in replies.jsonl with its wire hash; both served blocks publish byte-for-byte (C-MAP, C-HTML) so the 61.5% and the 71.4% can be recomputed from the same bytes we scored. The budget halt and its nineteen uncollected cells are in budget-event-c-none.json, and every rule that governs a figure above — including the registered floors and the forbidden-claims register this movement is written against — is in counting-rules.json.

The kit

Provenance

• Registered before running — the plan of record with its three-lens hardening fold (51 findings: 23 from the methodology lens, 14 from the hostile-reader lens, 14 from the ops lens, counting one uniquely-numbered row per finding and excluding the four proposals the panel rejected and its three escalation rulings, which are not findings) and its forbidden-claims register, sealed into a sha256 manifest over 22 files before the first model call: seal fb59bac8…, 2026-08-16T14:49:33Z, eleven of twelve preconditions met with the twelfth — a registered MAP-STALE minimum that came back vacant — printed as a reduction rather than re-scoped. The plan itself is behind that seal rather than in the kit, for the same redaction reason movement two's recon document is: it names working-copy paths and the machines that ran it. What binds a published figure is extracted verbatim into counting-rules.json instead, and the seal is what makes the extraction checkable (seal-manifest.json)
• Which weights answered — ollama 0.32.13, num_ctx 32768, think false, temperature 0.0, every local digest read before the first scored call and the cloud shelf re-probed at registration; the reference tokenizer that measured the two blocks' budgets ran on an earlier daemon build, and the receipt says so (runtime-pins.json, slice-receipt.json)
• What ran — 663 calls crossed the wire, against a sealed plan of 674: 557 of the 576 scored cells, 30 discarded warmups, a 72-call flake probe and 4 metered output probes. The 674 is the pre-registration's own roster arithmetic, sealed twenty-six seconds before the first call — the measured figure leads here and the registered one rides beside it, because a plan is not a measurement. The eleven-call gap decomposes exactly, and every term of it is a receipt: minus the 19 cells the budget event never collected, published as not-collected rather than quietly absent; plus the 6 extra warmups the flake leg took; plus 2 probe calls the plan never budgeted. That last term is the one worth naming — the plan registered two metered output probes, one per metered arm, but a probe is not a call: each arm's probe sampled two items, N03 and B01, so four probe calls crossed the wire where the plan had counted two. All four are itemized with their token counts in runtime-pins.json, and the bill prices all four (run-receipt.json, budget-event-c-none.json, warmups.jsonl)
• Question authorship — authored in-lane against the frozen corpus by the same pen that wrote this page, sealed before the first call; the answer-presence audit is what makes that conflict mechanically checkable rather than merely declared, and it publishes whole (answer-presence-audit.md)
• Counting rules — every rule that governs a figure here, in one file, including the log-counting sentence verbatim so the 30-day re-measure reconciles (counting-rules.json)
• Corrected after publication — 2026-08-16, the evening of release, by a content-accuracy pass run over this page against its own kit. Two figures moved, and both are named here rather than quietly swapped: 661 calls became 663, and the gap against the sealed plan went from thirteen to eleven. The cause is one distinction the page had missed — the plan registered two metered output probes, one per metered arm, and each probe turned out to be two calls, so four crossed the wire. runtime-pins.json has itemized all four since the run and the bill priced all four; only this page's arithmetic was short. The registered log claim had lost three words in the retelling and reads “the path was never requested in the window” again, which is what counting-rules.json has said in both places throughout. Four clarifications carry no change of figure: C-MAP is scoped to the two files it actually is rather than to the estate; the strata sentence now uses the registered stratum names; the control block's two different URL counts are reconciled with the rule each one obeys; and the rules behind 51 findings, the eighteen-host census and the 13:40Z provenance are now stated where those figures are used. No scored result moved — every accuracy, count, denominator and band on this page is the one it published with.
• Limits — one site, one day, one runtime version, two files, eight arms, four of them quantized local seats; a served-context bench and not a crawling one; n=1 per cell with a flake probe instead of retakes; and this page will itself join the corpus it benched, so a re-run needs a fresh question set
• Authorship — benched, drafted, and audited by the workshop's own agents under a human operator's rulings, then revised with that operator — often across many rounds; nothing releases until they have read it and signed off. The same division of labor this whole hub practices — told in full here
• Raw rows on request — drop a line.

Licence: CC BY 4.0 — the whole page, not only the kit. The prose, the tables and the data are yours to quote, re-plot, translate and argue with, including commercially. What we ask back is the one thing the licence already requires: name the source and link to it — strata→signal research, research.strata2signal.com — so a reader of your version can reach ours and check it against the files. Something like — strata→signal research, “The map nobody picks up”, research.strata2signal.com/llms-txt/, CC BY 4.0. And if you quote a figure, quote its denominator beside it — the tables above carry theirs.

One honest edge, which we would rather say than have you find. Two things here are not ours to license. The replies in replies.jsonl were produced by other people's models — eight arms across seven families, every tag printed in the table above — and we cannot license to you what we do not own; each stays subject to whatever terms its own vendor attaches. The same goes for the passages we quote from the convention's own documents: the 2024 proposal, the v2 spec and the two Google pages belong to their authors, and we quote them with their dates so you can go and read the originals. Our questions, our checkers, our scoring, our served blocks, our measurements and every word around them are ours, and those are CC BY 4.0. Republishing the replies or the quoted passages at length is between you and their owners — the rest of this page is between you and us, and the answer is yes.

Keep reading

The newest bench on the shelf is the new kid — qwen3.8:27b through four house exams the week it shipped, one seat filled, one floor missed. The audition that seats our narrator is the open call. Or start at the whole shelf: fourteen exhibits, receipts under every claim.