{
 "schema": "s2s-bench-v1",
 "kind": "data-kit-index",
 "exhibit": "the-free-speed-wasnt-free",
 "exhibit_title": "The free speed wasn't free — the runtime dividend, traced to one commit",
 "exhibit_url": "https://research.strata2signal.com/the-free-speed-wasnt-free/index.html",
 "published_utc": "2026-08-22",
 "licence": "CC BY 4.0",
 "attribution": "strata→signal research, research.strata2signal.com",
 "contact": "hello@strata2signal.com",
 "what_this_measures": "Why one model on one card went from ~138-146 (the well-controlled pre-upgrade medians; the kit timeline carries wider individual readings) to ~202-214 decode tok/s across a runtime upgrade, isolated three ways: a same-binary option toggle on a sealed byte-identical request (1.538x), a controlled cross-version pair — both runtimes side by side, same hour, same card, same prompt, n=10 per arm at temperature 0 (+31.7%) — and a three-cell toll probe that measures the removed work at ~2.5 ms per generated token across an 83-fold prompt-length spread. The cause is a named upstream commit changing the server's default repeat_penalty from 1.1 to 1.0; the kit carries the commit message fetched raw, the daemon's own sampler-chain prints, and the recon reports with their UNVERIFIED lists intact.",
 "not_a_standard": "One card, one runtime lineage, our own models and prompts. The toll is measured on this box's vocabulary size and CPU; other stacks will differ. The cross-version pair ran the seat as a five-minute visitor rather than the pinned resident our other readings use, and one published reading (213.07 beside traffic) sits 4.7% above its empty-card twin — an open anomaly the page names. The quality half of the change (penalty-on vs penalty-off answers, judged) is UNMEASURED and stated so; the two arms' answers differ at temperature 0 and the kit carries both hashes.",
 "files": [
  {
   "file": "AB-LEG-PREREG.md",
   "role": "the pre-registration for the version pair, verbatim as pinned before any scored call: the question, both arms' design, the n=10 rule with decoding parameters, falsifiable predictions P1/P2 with the fail case, and the recorded refusal paths",
   "media_type": "text/markdown",
   "bytes": 2385,
   "sha256": "cbf08654c328d372e5cf52a75912683d751c954e286e1b07af74f93240ed381d",
   "redactions": [
    "one character changed at publication: the window range's ASCII hyphen became an en-dash (U+2013) to satisfy the hub's redaction gate; reverting that single character reproduces the pinned sha in PREREG-INDEX.txt byte-for-byte (fdac0f3a67dd...), so the disclosure is itself verifiable"
   ]
  },
  {
   "file": "PREREG-INDEX.txt",
   "role": "the pin: the pre-registration's sha and pin timestamp, recorded before the window opened",
   "media_type": "text/plain",
   "bytes": 111,
   "sha256": "ad4dbdbd7955582644278c81150940e53dc47d536f2ba923d1d496e19ac811dc",
   "redactions": []
  },
  {
   "file": "ab-runs.json",
   "role": "the version pair's raw rows: all 20 scored calls (10 per arm), runtime counters per call, reply sha256 per call (10/10 byte-identical within each arm, different across arms — eval_count 128 vs 130), the staged old-instance metadata and the disclosed container-to-binary deviation",
   "media_type": "application/json",
   "bytes": 5815,
   "sha256": "96b2e3336d1647e34aea5c35cde22054b1644943da9f680e2e2fea1bd6645ab1",
   "redactions": []
  },
  {
   "file": "AB-RUN-2026-08-21.md",
   "role": "the version pair's run note with its own correction history: P1 confirmed (+31.7%), P2 confirmed as registered, the original contention decomposition RETRACTED in place with the recon receipts, and the old arm's served /api/version read recorded (this is the pre-registration's RESULTS-AB.md, renamed at publication)",
   "media_type": "text/markdown",
   "bytes": 2708,
   "sha256": "fb9e682d0c9f74962977602e0784fdfdaed838ad39fbd69b4f82e89fbd232fe6",
   "redactions": []
  },
  {
   "file": "TOLL-PROBE-2026-08-21.md",
   "role": "the toll probes: three cells (28 / 2,335 / 68-class prompt tokens) x two arms with every raw sample inline, the verdict refuting both candidate causes of the two-designs discrepancy, and the decomposition that explains the version pair's smaller implied toll",
   "media_type": "text/markdown",
   "bytes": 3399,
   "sha256": "3b94de3e515000a154cc73bb3c08dd91f5e3169f43996f9706bef41982010877",
   "redactions": []
  },
  {
   "file": "RECON-A-upstream.md",
   "role": "recon report, upstream: the 16-commit window, the commit identified with author and dates, the release-notes sweep v0.32.10-15, the llama.cpp engine lineage and penalties source, the candidate table, and its own §8 list of claims left UNVERIFIED — erratum block at top resolves the parameter count and vocab source against later first-party reads",
   "media_type": "text/markdown",
   "bytes": 34019,
   "sha256": "572449d52067ce1970dd7ad320378e0a39ac08a443663a16f7569d1d3d9e3070",
   "redactions": [
    "estate-internal box names, home paths, and private overlay addresses replaced with public aliases; quoted clock strings re-expressed in UTC (same instants); disclosure header in-file"
   ]
  },
  {
   "file": "RECON-B-timeline.md",
   "role": "recon report, shipped-data timeline: every published gemma4:26b decode reading with posture and comparability flags, medians recomputed by hand from shipped values, the strongest cross-link (byte-identical reply hash across two dated kits), and the UNVERIFIED findings that retired the ~145 baseline and the co-resident premise",
   "media_type": "text/markdown",
   "bytes": 25639,
   "sha256": "092645533f0526c00b05e0cccfc26312bf7438aa6921fb6071aaf8d6a93b516b",
   "redactions": [
    "estate-internal box names, home paths, and private overlay addresses replaced with public aliases; quoted clock strings re-expressed in UTC (same instants); disclosure header in-file"
   ]
  },
  {
   "file": "RECON-C-legend.md",
   "role": "recon report, the estate's own record audited: the original observation and forensic traced in the archive, three corrections to the internal legend (including a cutover date off by one day in UTC), the not-claims with receipts, and the 13-reading dated table",
   "media_type": "text/markdown",
   "bytes": 43090,
   "sha256": "aa1f258902bd2da11a2c13b85e8ac768b07cdeb1b46d4d98eaf14c03685ebba2",
   "redactions": [
    "estate aliases + UTC re-expression per in-file disclosure; one follow-up item WITHHELD at publication (an internal serving-posture question still open), the withholding stated in place"
   ]
  },
  {
   "file": "COMMIT-6a261db7-MESSAGE-VERBATIM.txt",
   "role": "the upstream commit message, fetched raw from the source API (no summarizer in the path), byte-identical to the API's commit.message field at fetch time",
   "media_type": "text/plain",
   "bytes": 2081,
   "sha256": "25fe46cc6714c38548d87b5bc10a71bc5be511f261aacfee565d50e61bb2fb91",
   "redactions": []
  },
  {
   "file": "COMMIT-6a261db7-RAW-API.json",
   "role": "the raw commit API response the verbatim message was extracted from: sha, author identity and dates, and the +2/-2 stats",
   "media_type": "application/json",
   "bytes": 10855,
   "sha256": "1ca287b00a54586007f05a0213dfe659227a83de41b32d3a18419bd50e61d9f6",
   "redactions": []
  },
  {
   "file": "VOCAB-RECEIPT-2026-08-21.md",
   "role": "the first-party vocabulary receipt: the server's own load-log print n_vocab = 262144 attributed to the model's blob digest, plus /api/show parameter count and expert fields, with the note that vocab and context length are numerically identical by coincidence and receipted separately",
   "media_type": "text/markdown",
   "bytes": 1247,
   "sha256": "b293c1e5971b81465cd714ad5a9373c5bbba81a3481e48fd32c8092187071731",
   "redactions": [
    "the model-store filesystem prefix is elided from the quoted load line; the blob digest is quoted whole"
   ]
  },
  {
   "file": "speed-forensic-2026-08-16.md",
   "role": "the 2026-08-16 forensic that first solved the jump (the toggle): sealed-wire replay design, the daemon's own sampler-chain prints across the restart boundary, candidate table, and the contention refutation — filename desanitized from its internal name at publication",
   "media_type": "text/markdown",
   "bytes": 21282,
   "sha256": "c3b8c89fbbf6f192ff79ea27722806587cf646331e27c717d3f126d5210ae571",
   "redactions": [
    "estate-internal box names, home paths, and private overlay addresses replaced with public aliases; quoted clock strings re-expressed in UTC (same instants); disclosure header in-file"
   ]
  },
  {
   "file": "speed-probe-2026-08-16.json",
   "role": "the toggle's raw probe rows: both arms' options_sent, per-call counters, and the sealed request wire hash",
   "media_type": "application/json",
   "bytes": 4148,
   "sha256": "d7bded219ce3ca94b74ff1ac9c7136f07cf1e28b665deb50292c945df225fb2d",
   "redactions": [
    "estate-internal endpoint aliased; expires_at fields re-expressed in UTC (same instants; the year-2318 values are the runtime's never-expire sentinel)"
   ]
  },
  {
   "file": "perf-probe-qwen-penalty.json",
   "role": "the other 27B-class seat's probe (2026-08-16): presence_penalty 1.5 vs 0, n=3 per arm, the 2.407 ms/token stage cost the flat-toll table cites",
   "media_type": "application/json",
   "bytes": 2684,
   "sha256": "b03bbd3de4964e5c28216d77cb1239d608d565d71bb3635715b72fc741867b16",
   "redactions": []
  },
  {
   "file": "leg2b_ab.py",
   "role": "the version pair's runner, verbatim: the exact request shape (no penalty parameter anywhere), the scratch-store staging, and the port-conflict sidestep logic",
   "media_type": "text/x-python",
   "bytes": 5860,
   "sha256": "3229ea773c30b18f7b5273ec5a57a34c59b8dd6f0f6a0382cfa9023f22d79ede",
   "redactions": [
    "estate-internal box names, home paths, and private overlay addresses replaced with public aliases; quoted clock strings re-expressed in UTC (same instants); disclosure header in-file"
   ]
  }
 ],
 "owed_cells": [
  "the 2x2's missing cell: ollama 0.32.9 with repeat_penalty 1.0 sent explicitly, to place the newer runtime's penalty-path regression precisely",
  "the same-binary penalty-on-versus-off comparison at temperature 0 - replies hashed first, then judged - to settle what the quality half changed",
  "the fresh-load-versus-warm-resident gap measured as its own controlled experiment",
  "the v0.32.15 time-to-first-token dividend, pre-registered before it runs"
 ],
 "post_publication_corrections": []
}