# Ask About This Page — data kit **What this directory holds.** Every receipt behind the exhibit *Ask About This Page*: the frozen question bank the assistant was examined on, every arm file the exam produced across its nine sittings, the printed logs of the two most recent sittings, and the harness that scored them all. Nothing on the article's bench table is asserted here — each figure is a field in a file in this directory, and `NOTES.md` says which field, in which file, for every one. **The window, at both ends.** The first arm in this kit opened **2026-09-09T16:41:42Z**; the last arm opened **2026-09-13T03:54:41Z** and the sitting it belonged to closed at **2026-09-13T03:56Z**. Each arm file carries its own `run_utc` — the instant that arm began — and those, not this sentence, are the stamps to count against. The sitting the article's table prints, **run 9**, opened **2026-09-13T03:50:16Z** and closed **2026-09-13T03:56Z**, on the service version that launched. **Licence: CC BY 4.0.** Take these rows, re-score them, check our arithmetic, publish what you find. Attribution: **the strata→signal workshop**. If a number on the page does not reproduce from this kit, we want to hear about it: **https://strata2signal.com/contact/**. --- ## Read this first: two runs asked a shorter exam, and three are a different instrument Two facts about this kit have to be met before any row is read beside any other, because a reader who lines all nine sittings up in one table will otherwise line up three different things. **1. Runs 1 and 2 asked 60 questions, not 62.** The bank grew on 2026-09-09, when two articles poured after it was frozen got their hand-written questions. Runs 1 and 2 walked 39 on-page + 21 off-page = 60; runs 3 through 9 walked the **41 + 21 = 62** that `bank.json` carries. No question's text changed: the sets differ by exactly the two added rows, `on-40` and `on-41`, and that is checkable from this kit alone — compare the `rows[].id` lists of `runs/r1-cold-c1.json` and `runs/r3-cold-c1.json`. The on-page floor moved with the bank, at the same proportion, and `PREREG.md` dates that move. **2. Runs 1, 2 and 3 are `schema_version` 1.0. Runs 4 through 9 are `schema_version` 1.1, and the two are NOT comparable.** Every arm file says so in its own words, in `instrument.not_comparable_with`: *"reports whose schema_version is 1.0"*. Two things moved between the two schemas, both of them instrument corrections disclosed at the time and neither of them a floor: - the figure column was computed over the **raw** answer in 1.0 and over **cite-cut prose** in 1.1, so 1.0's `non_verbatim_figures` counts digits that were inside a citation address rather than in a sentence; - the warm arm **warmed all 41 pages in one pre-pass** in 1.0 and warms **one page immediately before that page's own questions** in 1.1, so a 1.0 warm latency is a cold latency wearing the warm arm's name. Each file's `instrument` object states which instrument took its numbers. Read 1.0 rows as the record of how the bench was wrong, which is what they are — and read a figure off run 9 against run 8, run 7, run 6, run 5 or run 4, never against run 3 or earlier. ## What is in here Fifty-two files. `index.json` lists every one of them with its size and sha256; this table says what each *kind* of file is. | file | what it holds | what was applied to it | |---|---|---| | `bank.json` | The frozen exam: 62 questions — **41 on-page**, one or two per article, and **21 off-page** — each carrying the article slug it is asked against, its `id` and its `kind`. Also the bank's own pre-registered gates, its counts, and the note explaining why the off-page questions were re-cut. | `author-to-role`, `absolute-path-to-filename` | | `runs/r--c.json` | One arm of one sitting, **36 of them — nine sittings, four arms each**: the instrument that took it, the UTC instant it opened, the arm and its requested and **effective** concurrency, the seat class, the aggregate `results` block every figure on the page is read from, the `verdicts` against the pre-registered floors, and **every scored row** — the question, the draft, what was served, which of the six checks fired, the cites, the figures, the seconds, the tokens in and out and how many of them came from the prompt cache. | `absolute-path-to-filename`, `host-to-role` | | `runs/r8--c.log` · `runs/r9--c.log` | The harness's own printed verdict for the four arms of **run 9** — the sitting the article's table prints — and of **run 8**, the sitting before it, exactly as they reached the operator's terminal, floors and all. **Those two sittings only**: the earlier sittings' logs are not in this kit, and their numbers are all in their arm files. | none — byte for byte | | `harness/run.py` | The scorer, so *"the harness that scores them"* is a file and not a claim. Its docstring is the bench's own working record: what it measures, what it refuses to measure, and the three instrument corrections that were made to it mid-arc. | `host-to-role` | | `harness/packnorm.py` | The normaliser that decides what a **figure** is — *"verbatim"* means nothing without the rule that decides what verbatim is. The ruled definition is `v3`, and each arm file stamps which definition was recorded under `packnorm_definition_recorded_not_governing`. | `author-to-role` | | `PREREG.md` | The pre-registered floors, verbatim from the bank, with the UTC date each was registered and the sitting it was registered **before**. A rung judged by its own gate needs its gate published first. | authored for publication | | `NOTES.md` | The measurement record: every headline figure the exhibit prints, with the file and the field it is read from, and the two comparisons this data does not support. | authored for publication | | `provenance.json` | Every file above with its size, its sha256, the sha256 of the bytes it was made from, the named rules applied to it and how many times each one fired. | — | | `index.json` | This directory, listed: names, sizes, sha256s, and what each file answers. | — | ## What "cold" and "warm" mean here They are two different readers, and the bench measures both. - **Cold** is the first reader of the day on a page nobody has asked about. The arm walks the whole bank in bank order, so by the time a page's question is asked the prompt cache holds somebody else's page. The cold arm is the first-read case, and its `cache_hits` field is **null** by construction. - **Warm** is the second reader of the same page. The arm warms **one page immediately before that page's own questions**, then asks them, then moves to the next page — the reader's own pattern, and what the room actually does when a page has just been read. The article's table has a **prompt cache** column, and on both cold rows it prints an **em dash (—)**. That is not zero and not "missed". It means **not measured on this arm**: a cold arm has no warmed page to hit, so the harness writes `null` rather than a number, and the page prints the dash rather than inventing one. The warm rows print `62 / 62`, which is the count of rows that read **at least half** of their prompt from cache, out of the rows where the seat reported a cache figure at all — `results.cache_hits` over `results.cache_hit_of`. **One warm number is not what its label suggests, and the kit says so before the page does.** On the warm arm a *page* is the unit of parallelism, and no page in this bank carries more than two questions — so `--concurrency 6` on a warm arm puts at most **2** calls in flight. Every arm file computes that and carries it as `effective_concurrency`. The warm "six in flight" row is an effective-concurrency-2 row, and it may not be read as six readers at once. ## What an off-page question is, and why abstaining is a pass Twenty-one of the 62 questions are **deliberately unanswerable from the page they are asked on**. Each one asks its article for something the article cannot know — a result published in a future year, a price nobody published, a fact about the workshop as a business, a count no page keeps. They are not trick questions about obscure corners of the text; they are outside it. So for an off-page question there is exactly **one** correct behaviour: the room says the page does not answer this, and points at the nearest section. The floor is **21 of 21** — every one of them, every arm. An off-page question that comes back *answered* is the failure, however true the answer happens to be, because the answer did not come from the page. The harness counts an abstain two ways and prints both, because they are different facts. `off_page_abstained_by_gate` is the model producing something the page could not support and a check replacing it. `off_page_abstained_by_model` is the model refusing correctly, in its own words, and the service restating that refusal in the workshop's own. Both reached the reader as a refusal; only the second means the seat was right on its own. This is also the floor that **failed once**, on run 7's cold single-stream arm — 20 of 21 — and `NOTES.md` traces that row to the field it is read from. ## What the bench does not measure, said here as well as on the page The bench measures **groundedness by the only tests a machine can run**: every figure verbatim in the article, every citation resolving to a real section of it, no redaction hit, and the abstain when the page does not say. It measures **nothing about whether an answer is good**. There is no judge seat in this kit and no quality score in any file in it; nothing here should be read as one. ## Sanitised at the pen, not copied Nothing here is a raw copy of a file that named a machine. Each file was read and rewritten, and every transform has a NAME that `provenance.json` prints per file with the number of times it fired — so a reader can see which rules touched a file and be sure nothing else did. The rules, in full: - **`absolute-path-to-filename`** — every arm file recorded the bank it was run against as a path into a private filesystem. It becomes the file's own name, `bank.json`, which is what it means in this kit anyway. The same rule cut one path out of the bank's own note. - **`host-to-role`** — the arm files record the seat's **seed**, verbatim, because a fixed seed makes every visitor's same question byte-identical and a run that cannot say which side of that change it was taken on cannot be compared to the next one. In the first four sittings that field named the serving machine and its service unit. It now states the fact it was recorded for: **no `--seed` flag was sent**, and the seat's serving unit stopped sending one in the change that added the bearer, on 2026-09-09. The same rule replaced a machine's name and a machine's role in two comments in `harness/run.py`. - **`author-to-role`** — the bank records who wrote its questions, by name. It now records **the workshop**. The same rule replaced one name in a ruling citation in `harness/packnorm.py`. **What was not touched.** Every count, verdict, latency, token figure, draft sentence, citation and UTC instant is published as recorded. No row was dropped, reordered or resampled. No failed arm was excluded — **run 7's failing arm is in this kit**, with `passed: false` on its face, because the article tells that story and a kit that carried only the passing rows could not support it. ## Not a standard One seat class, on one machine, on one corpus of 41 articles, against one frozen bank of 62 questions. It is not a leaderboard and not a benchmark suite. A different model, a different runtime, a different corpus or a different bank will land somewhere else, and the numbers here are not evidence about any of them.