{
  "schema": "s2s-bench-v1",
  "kind": "data-kit-index",
  "exhibit": "chair-trials",
  "exhibit_title": "The chair trials — five fresh exams for the thirty-billion class",
  "exhibit_url": "https://research.strata2signal.com/chair-trials/index.html",
  "published_utc": "2026-08-13",
  "licence": "CC BY 4.0",
  "attribution": "strata→signal research, research.strata2signal.com",
  "contact": "hello@strata2signal.com",
  "contamination_caveat": "Published 2026-08-13. Models with a later training cutoff may have seen these sets. We author fresh sets each cycle; this one is not a standard, it is our kit, yours to reuse.",
  "not_a_standard": "Our own trials, our own hardware, for our own chairs. Not first independent numbers, and not a benchmark.",
  "files": [
    {
      "file": "roster.json",
      "role": "the eleven arms: vendor-labelled architecture, quantization read first-hand off the daemon, measured residency in bytes and GiB at the context length every leg ran, and the postures each leg put each arm through",
      "media_type": "application/json",
      "bytes": 22742,
      "sha256": "bfc9ebea1f50320dc92f7265b66c72e0a824b2902c5c37aab6c9b65bacfd78dd"
    },
    {
      "file": "c1-judge.json",
      "role": "chair one, the judge: ten rows against two floors with Wilson intervals, self-consistency, the protocol-fence polarity probes, the markdown-fence caveat and the reasoning-dial delta",
      "media_type": "application/json",
      "bytes": 69197,
      "sha256": "7bbba26d0581ca241699692759fcc32e641eee952e8fc1932d24de8cd1ebdcc0"
    },
    {
      "file": "c2-assistant.json",
      "role": "chair two, the assistant: eight posture rows over twenty code-checked items, with every item's per-repeat verdict and the per-item matrix across all eight rows",
      "media_type": "application/json",
      "bytes": 358812,
      "sha256": "0ffee8f453cfbc838cebdc51e05f2ef38a22b24b14e003ae3a41ca96db729e4d"
    },
    {
      "file": "c3-toolbench.json",
      "role": "chair three, the toolbench: nine rows over nineteen tasks, with every task's calls — tool, arguments, disposition — and the pre-probe that decided which runtimes could carry the contract at all",
      "media_type": "application/json",
      "bytes": 570948,
      "sha256": "75098737f370310b288f9bdcfba46e5120fd8ce0fcfb31caa306f2f2ef1ae847"
    },
    {
      "file": "c4-narrator.json",
      "role": "chair four, the narrator: six arms, 1 080 pairwise comparisons with cluster-bootstrap intervals, the distinctness round, the canon adjudication with per-judge votes and notes, and the per-judge position-bias rates",
      "media_type": "application/json",
      "bytes": 131602,
      "sha256": "8de910fb384032990e446f008b69045d641d7f120da664b4204fe8e76cd9460e"
    },
    {
      "file": "c5-stopwatch.json",
      "role": "chair five, the stopwatch: eight arms across three prompt tiers with every raw repeat value, the two unexplained anomalies counted per arm, the drafter byte-compare receipt and the reasoning-strength sweep",
      "media_type": "application/json",
      "bytes": 231890,
      "sha256": "8673baf2c45546079a838d6d1017f565f449c4638629e2d07bc993894c337e02"
    },
    {
      "file": "c6-serving.json",
      "role": "the serving probes: twenty-six real asks against the live app's production path, per-ask walls with their overlap flags, the paired quiet baselines, and the burst, pair and cold probes with their receipts",
      "media_type": "application/json",
      "bytes": 231624,
      "sha256": "4245620c844da6ebb1b487d02e2afeb4e66c29015b155f622f45825624915ade"
    },
    {
      "file": "judge-c1.json",
      "role": "the runnable judge set: all twenty-one claim, span and verdict triples, whole",
      "media_type": "application/json",
      "bytes": 32324,
      "sha256": "c42d42aa908591dd9916da8ec10455ad9d810c91baf2cd4f98044861298e2e47"
    },
    {
      "file": "assistant-c2.json",
      "role": "the runnable assistant set: all twenty frozen items with their prompts and expectations, plus the two trimmed at freeze, marked rather than deleted",
      "media_type": "application/json",
      "bytes": 25509,
      "sha256": "dbf38ec7eeb3448cf5ba48e4eeb8ad906ba210a5995b972478b3a8ae7680c61c"
    },
    {
      "file": "CHECKERS.md",
      "role": "what each assistant checker actually tests — generated from the scorer module itself, so the published rule is the rule that ran",
      "media_type": "text/markdown",
      "bytes": 8572,
      "sha256": "95255bcbf04fae6764e1634d8b8c53f8c80966c90997483c1cfacb3c51e19d19"
    },
    {
      "file": "tools-tasks.json",
      "role": "the runnable toolbench set: all nineteen tasks with their expected calls and predicates",
      "media_type": "application/json",
      "bytes": 31186,
      "sha256": "593636e04340cb3a4599fdb7ddda8835209b69bc466583e84cc52cbcafd7a585"
    },
    {
      "file": "tools-schemas.json",
      "role": "the ten tool schemas exactly as every model received them, dumped from the suite that served them",
      "media_type": "application/json",
      "bytes": 10051,
      "sha256": "63caa4edd086b320b91328e6ae1b535f66dc85bf58925c38391910850a12b150"
    },
    {
      "file": "c3-example-trace.json",
      "role": "one complete toolbench transcript, round by round, so the per-task calls in the C3 companion can be read for what they are",
      "media_type": "application/json",
      "bytes": 8495,
      "sha256": "c07cda701ff63f1c9b49aa86fc6a394cde6f4d649df59e374084f7e6cdf723e2"
    },
    {
      "file": "voice-questions.json",
      "role": "every question put to a character, with its canon note and what it probed — questions only: the persona blocks are withheld and the fence is stated",
      "media_type": "application/json",
      "bytes": 12410,
      "sha256": "f10a83025d0acdf2fb337a6422baaac6345bcd520a5c5e5ff7de28064dd47a20"
    },
    {
      "file": "c6-asks.json",
      "role": "every ask the serving leg was allowed to send, frozen before the first one went out",
      "media_type": "application/json",
      "bytes": 10220,
      "sha256": "09e6d04821528672e633bb4010e106dff9504e594894453583621b83d4daa736"
    },
    {
      "file": "counting-rules.json",
      "role": "every leg's counting rules in one file, for a reader who wants the rules without the rows",
      "media_type": "application/json",
      "bytes": 24759,
      "sha256": "8a74239ac354760f94aeae40448548716b51fa07eb01793eca0d31d755e28ce4"
    },
    {
      "file": "provenance.json",
      "role": "the pre-registration and golden-set hashes: what was pinned, when it was frozen, and what each one governs",
      "media_type": "application/json",
      "bytes": 15252,
      "sha256": "73ccb14a4b8ee68f43adf70a238bbda86176c5afc6943650a6d0507a0ba023d8"
    },
    {
      "file": "README.md",
      "role": "the kit's own README: licence, receipts, the contamination caveat, and how to read a number here",
      "media_type": "text/markdown",
      "bytes": 10902,
      "sha256": "7c0691e15d28a1f7c7f772b0f21316a62707588efbceda2f252d7dd46c526ffd"
    }
  ],
  "self": {
    "file": "index.json",
    "note": "this listing; its own sha is not carried inside itself"
  },
  "post_publication_corrections": "2026-08-16: a content-accuracy pass corrected two stale label sentences in this kit — c4-narrator.json's judges.canon_and_distinctness_panel, which said the distinctness and canon rounds ran on three judges when both have carried all four seats since the file was written (432 distinctness assignments, 144 canon judgements, unscored_batches empty in every leg), and voice-questions.json's voice-c4 purpose, which opened 'Exhibit five' where the chair trials publish as exhibit six. Each correction is written into the corrected file's own text, and both files are re-hashed above, as is README.md, which records the pass in full. No figure, denominator, verdict, interval or counting rule moved. Nothing in this kit is sealed behind a seal manifest, so a correction is made in place rather than disclosed beside an untouchable file. The exhibit page carried the same three-judge sentence and was corrected in the same pass, which it records in its provenance list."
}