{ "schema": "s2s-bench-v1", "exhibit": "chair-trials", "published_utc": "2026-08-13", "status": "PUBLISHED — live at research.strata2signal.com.", "licence": "CC BY 4.0", "attribution": "strata→signal research, research.strata2signal.com", "hardware": "One 96G VRAM workstation, which also answered live requests for three of our apps throughout these trials — two public, one serving playtests for humans and agents alike. Contention is measured in both directions and published with the figures rather than assumed away.", "dataset": "chair-trials-roster", "table": "roster", "what_this_is": "The eleven arms of the chair trials — nine models, two of which ran both thinking postures and one of which ran two quantizations — with what each cost to keep resident at the context length every scored leg used. Residency is measured, one arm at a time, on the workstation that serves our own apps; architecture is a vendor label and says so.", "columns": [ "model", "params", "quant", "loaded VRAM GiB @ 32 768", "legs run", "postures run" ], "legs_run_note": "C1 through C5 are the five chairs below, in page order: judge, assistant, toolbench, narrator, stopwatch. Only five of the eleven arms sat all five, and this column is how a table-only reader learns that. An arm absent from a leg's table is absent because it never sat that chair, and the roster is where that is stated once rather than eleven times. Two row-tints recur on this page: the wine tint with a left bar marks the row a leg's own rule chose, and the grey tint marks rows carried in from an earlier published table for comparison.", "columns_note": "Five columns, and one registered column deliberately absent: the article's table sketch names a `disk GB` column, and no leg of this arc measured on-disk blob size. The only size this daemon reports is the resident one, already published as `loaded_vram_gib`. A column we cannot fill is dropped and disclosed rather than printed as eleven em dashes.", "counting_rules": { "residency": "size_vram is read verbatim from ollama's /api/ps for the named tag, in bytes, immediately after a load that generated exactly 1 token at the named num_ctx, with the model still resident and no other candidate resident. GB figures anywhere downstream are size_vram / 1073741824 (GiB, 1024^3), rounded to 2 dp, and the byte count always travels with them. A row is one (tag, num_ctx) pair: size_vram = weights + KV cache, and the KV cache is a function of num_ctx, so a figure quoted without its ctx is not a figure. Protected residents (the live seat, the live embedder) are READ, never loaded: their rows carry the ctx the daemon reports, which is whatever production last set, not a value we chose.", "row_is_an_arm": "One row is one ARM — a model tag as these legs ran it. Three of the eleven are the same weights under different serving choices (two quantizations, one served without its draft model), which is why the roster has eleven rows and the arc has nine models.", "ctx": "Every figure here is at num_ctx 32768, the length every scored C-leg ran at. size_vram is weights plus KV cache and the KV cache is a function of the context length, so a residency figure without its context length is not a figure. The census also holds rows at 16 384 for the four arms that sat the frozen seat exam; those belong to the previous exhibit and are published there.", "postures": "`postures_run` is derived from the legs' own scored artifacts — the postures each leg actually recorded for that arm — never declared by hand." }, "daemon": { "ollama_version": "0.32.9", "kv_cache_type": "q8_0", "flash_attention": "1", "context_length_default": "32768", "read": "MEASURED — read from the serving daemon's own unit environment at run time, not assumed. The census record names the host it was read on; that name is withheld here by the same house rule that keeps filesystem paths off this site." }, "outcome_vocabulary": { "MEASURED": "loaded alone, read, context length confirmed, released. A figure is published.", "READ-ONLY": "a protected production resident, read from /api/ps while production held it. A figure is published, with the context length production set.", "NOT-RUN": "a live workload this arc may not load, release or sweep, and which was not resident to be read AT CENSUS TIME. The reason rides the row. Where a later leg caught the same arm resident and read it in contract, the figure publishes with that leg named — see `residency_observed_elsewhere`." }, "limits": [ "size_vram is what this daemon allocated under this configuration — not what a model 'needs', and not what another runtime would allocate.", "The three drafter rows carry an open finding raised by the previous exhibit's census and unresolved at publication: a model served with a draft model beside it reports a tenth of what the same weights report without one, and two different quantizations of it report a byte-identical allocation. Read that finding before quoting those three residency figures anywhere.", "Quantization differs across rows and is printed on every one. Comparing across quantizations is a confound we name, not one we equalize." ], "open_findings": [ { "id": "drafter-rows-under-review", "rows": [ "muse-glimmer:30b-q8_0-dflash @ 32 768", "muse-glimmer:30b-q4_K_M-dflash @ 32 768" ], "what": "Both drafter tags report 2.34 GiB against 15.60 GiB for the same 27.9B weights served without a drafter, and both report a byte-identical 2 515 145 849 on different digests and different quantization levels — which two different quantizations cannot both allocate. The likeliest reading is that the daemon answered for the draft model rather than for the pair.", "status": "published verbatim, flagged, and held pending a re-measure. No number has been adjusted, and every other row is an independent load, unaffected." } ], "provenance": { "artifacts": [ { "role": "the residency record every figure here is copied out of", "artifact": "vram-census.json", "sha256": "8c53028e8b22a6e1b76c70aa7c208939f3f1e0f8974eda852db2920a55ebf130" }, { "role": "the registered denominator — the rows the census was allowed to measure", "artifact": "c7-roster.json", "sha256": "d6f67db93acf55f10b941aab3a64ee46d97df2881bb032891daac779be586844" }, { "role": "pre-registration, frozen before the first load", "artifact": "PREREG-C7-census.md", "sha256": "3ea13e0676300032c6ca31849c84eace65108e74b1f11be3b49ed6123350afc0" }, { "role": "the census itself", "artifact": "vram_census.py", "sha256": "0c87af2760c790f0fd8b71e3da7d77845c669e414863466508fd5b1ba378bcde" } ], "measured": "2026-08-12, evening, local time", "no_paths": "Artifacts are named, never located. No box names, no addresses, no filesystem paths." }, "rows": [ { "model": "muse-glimmer:30b-q8_0-dflash", "params_total": "27.9B", "params_active": null, "params_projector": "1.9B", "params_projector_source": "blob metadata, recorded in the 2026-08-12 freeze rulings (Hub lane). The vision projector is unused in every leg of this arc. This census reads size_vram and the daemon's parameter_size, which is the TEXT figure — the projector count is not re-measured here and is labelled as the vendor/blob figure it is. The model card's own 27.8B + 1.8B are vendor figures and differ.", "params_source": "read first-hand from the daemon's /api/ps entry for this load", "params_active_source": null, "quant": "Q8_0", "quant_source": "read first-hand from the daemon's /api/ps entry for this load", "roster_class": "dense 27.9B q8 + drafter", "ctx": 32768, "loaded_vram_gib": 2.34, "size_vram_bytes": 2515145849, "residency_outcome": "MEASURED", "residency_reason": null, "residency_observed_elsewhere": null, "load_s": 7.713, "protected": false, "legs": { "C1 judge": [ "think:false", "reasoning-medium", "reasoning-high" ], "C2 assistant": [ "think:false", "think:true" ], "C3 toolbench": [ "think:false", "think:true" ], "C4 narrator": [ "think:false", "think:true" ], "C5 stopwatch": [ "think:false", "reasoning-strength low/medium/high/xhigh" ] }, "legs_run": [ "C1 judge", "C2 assistant", "C3 toolbench", "C4 narrator", "C5 stopwatch" ], "legs_run_display": "C1 · C2 · C3 · C4 · C5", "postures_run": "think:false · think:true · reasoning-strength low/medium/high/xhigh" }, { "model": "muse-glimmer:30b-q4_K_M-dflash", "params_total": "27.9B", "params_active": null, "params_projector": "1.9B", "params_projector_source": "blob metadata, recorded in the 2026-08-12 freeze rulings (Hub lane). The vision projector is unused in every leg of this arc. This census reads size_vram and the daemon's parameter_size, which is the TEXT figure — the projector count is not re-measured here and is labelled as the vendor/blob figure it is. The model card's own 27.8B + 1.8B are vendor figures and differ.", "params_source": "read first-hand from the daemon's /api/ps entry for this load", "params_active_source": null, "quant": "Q4_K_M", "quant_source": "read first-hand from the daemon's /api/ps entry for this load", "roster_class": "dense 27.9B q4 + drafter", "ctx": 32768, "loaded_vram_gib": 2.34, "size_vram_bytes": 2515145849, "residency_outcome": "MEASURED", "residency_reason": null, "residency_observed_elsewhere": null, "load_s": 13.537, "protected": false, "legs": { "C3 toolbench": [ "think:false" ], "C5 stopwatch": [ "think:false", "reasoning-strength low/medium/high/xhigh" ] }, "legs_run": [ "C3 toolbench", "C5 stopwatch" ], "legs_run_display": "C3 · C5", "postures_run": "think:false · reasoning-strength low/medium/high/xhigh" }, { "model": "muse-glimmer:30b", "params_total": "27.9B", "params_active": null, "params_projector": "1.9B", "params_projector_source": "blob metadata, recorded in the 2026-08-12 freeze rulings (Hub lane). The vision projector is unused in every leg of this arc. This census reads size_vram and the daemon's parameter_size, which is the TEXT figure — the projector count is not re-measured here and is labelled as the vendor/blob figure it is. The model card's own 27.8B + 1.8B are vendor figures and differ.", "params_source": "read first-hand from the daemon's /api/ps entry for this load", "params_active_source": null, "quant": "Q4_K_M", "quant_source": "read first-hand from the daemon's /api/ps entry for this load", "roster_class": "dense 27.9B q4, no drafter", "ctx": 32768, "loaded_vram_gib": 15.6, "size_vram_bytes": 16750089337, "residency_outcome": "MEASURED", "residency_reason": null, "residency_observed_elsewhere": null, "load_s": 3.662, "protected": false, "legs": { "C5 stopwatch": [ "think:false" ] }, "legs_run": [ "C5 stopwatch" ], "legs_run_display": "C5", "postures_run": "think:false" }, { "model": "gemma4:26b", "params_total": "25.8B", "params_active": "3.8B active", "params_projector": null, "params_projector_source": null, "params_source": "read first-hand from the daemon's /api/ps entry for this load", "params_active_source": "vendor-labelled architecture, transcribed from the frozen census roster's class string; active parameters are never measured here", "quant": "Q4_K_M", "quant_source": "read first-hand from the daemon's /api/ps entry for this load", "roster_class": "MoE 25.8B total / 3.8B active — THE LIVE SEAT", "ctx": 32768, "loaded_vram_gib": 16.27, "size_vram_bytes": 17474319810, "residency_outcome": "READ-ONLY", "residency_reason": "read from /api/ps while production held it; never loaded, never released, never re-pinned by this census", "residency_observed_elsewhere": null, "load_s": null, "protected": true, "legs": { "C1 judge": [ "think:false" ], "C2 assistant": [ "think:false", "think:true" ], "C3 toolbench": [ "think:false", "think:true" ], "C4 narrator": [ "think:false" ], "C5 stopwatch": [ "think:false" ] }, "legs_run": [ "C1 judge", "C2 assistant", "C3 toolbench", "C4 narrator", "C5 stopwatch" ], "legs_run_display": "C1 · C2 · C3 · C4 · C5", "postures_run": "think:false · think:true" }, { "model": "gemma4:12b", "params_total": "11.9B", "params_active": null, "params_projector": null, "params_projector_source": null, "params_source": "roster-labelled — this arm was never loaded by this arc, so no first-hand metadata exists for it", "params_active_source": null, "quant": "q4", "quant_source": "roster-labelled — this arm was never loaded by this arc, so no first-hand metadata exists for it", "roster_class": "dense 11.9B q4 — LIVE WORKLOAD (interim ruling 2026-08-12)", "ctx": null, "loaded_vram_gib": null, "size_vram_bytes": null, "residency_outcome": "NOT-RUN", "residency_reason": null, "residency_observed_elsewhere": { "artifact": "summary-C6-cold.json", "size_vram": 8110022327, "size_vram_gib": 7.55, "ctx": 32768, "observed_at": [ "2026-08-12T18:34:03-06:00", "2026-08-12T18:40:19-06:00" ], "how": "Read as-found from /api/ps at the send of each cold-visitor ask, fifteen and twenty-one minutes after the census looked for it and did not find it resident. Both reads returned the identical byte count at the same context length. Nothing loaded, released or swept it: this arm is a live workload under the 2026-08-12 interim protection ruling, and an opportunistic read is the only measurement the ruling allows.", "why_published": "The census row stands as it was recorded — NOT-RUN, not resident at 18:19. But a figure this arc actually observed, under the same counting rule and the same context length, is not a figure we may leave off the page: an em dash means we hold nothing, and we held this." }, "load_s": null, "protected": true, "legs": { "C1 judge": [ "think:false" ], "C2 assistant": [ "think:false" ], "C3 toolbench": [ "think:false" ], "C4 narrator": [ "think:false" ], "C5 stopwatch": [ "think:false" ] }, "legs_run": [ "C1 judge", "C2 assistant", "C3 toolbench", "C4 narrator", "C5 stopwatch" ], "legs_run_display": "C1 · C2 · C3 · C4 · C5", "postures_run": "think:false" }, { "model": "qwen3.5:27b", "params_total": "27.8B", "params_active": null, "params_projector": null, "params_projector_source": null, "params_source": "read first-hand from the daemon's /api/ps entry for this load", "params_active_source": null, "quant": "Q4_K_M", "quant_source": "read first-hand from the daemon's /api/ps entry for this load", "roster_class": "dense 27.8B q4", "ctx": 32768, "loaded_vram_gib": 16.24, "size_vram_bytes": 17434966751, "residency_outcome": "MEASURED", "residency_reason": null, "residency_observed_elsewhere": null, "load_s": 9.045, "protected": false, "legs": { "C1 judge": [ "think:false" ], "C2 assistant": [ "think:false" ], "C3 toolbench": [ "think:false" ], "C5 stopwatch": [ "think:false" ] }, "legs_run": [ "C1 judge", "C2 assistant", "C3 toolbench", "C5 stopwatch" ], "legs_run_display": "C1 · C2 · C3 · C5", "postures_run": "think:false" }, { "model": "qwen3.6:27b", "params_total": "27.8B", "params_active": null, "params_projector": null, "params_projector_source": null, "params_source": "read first-hand from the daemon's /api/ps entry for this load", "params_active_source": null, "quant": "Q4_K_M", "quant_source": "read first-hand from the daemon's /api/ps entry for this load", "roster_class": "dense 27.8B q4", "ctx": 32768, "loaded_vram_gib": 16.24, "size_vram_bytes": 17434966751, "residency_outcome": "MEASURED", "residency_reason": null, "residency_observed_elsewhere": null, "load_s": 9.025, "protected": false, "legs": { "C1 judge": [ "think:false" ], "C2 assistant": [ "think:false" ], "C3 toolbench": [ "think:false" ], "C4 narrator": [ "think:false" ], "C5 stopwatch": [ "think:false" ] }, "legs_run": [ "C1 judge", "C2 assistant", "C3 toolbench", "C4 narrator", "C5 stopwatch" ], "legs_run_display": "C1 · C2 · C3 · C4 · C5", "postures_run": "think:false" }, { "model": "nemotron-3.5-lightning:30b-a3b", "params_total": "32.9B", "params_active": "3B active", "params_projector": null, "params_projector_source": null, "params_source": "read first-hand from the daemon's /api/ps entry for this load", "params_active_source": "vendor-labelled architecture, transcribed from the frozen census roster's class string; active parameters are never measured here", "quant": "Q4_K_M", "quant_source": "read first-hand from the daemon's /api/ps entry for this load", "roster_class": "MoE 32.9B total / 3B active q4", "ctx": 32768, "loaded_vram_gib": 23.64, "size_vram_bytes": 25386863820, "residency_outcome": "MEASURED", "residency_reason": null, "residency_observed_elsewhere": null, "load_s": 11.373, "protected": false, "legs": { "C1 judge": [ "think:false" ], "C2 assistant": [ "think:false" ], "C3 toolbench": [ "think:false" ], "C4 narrator": [ "think:false" ], "C5 stopwatch": [ "think:false" ] }, "legs_run": [ "C1 judge", "C2 assistant", "C3 toolbench", "C4 narrator", "C5 stopwatch" ], "legs_run_display": "C1 · C2 · C3 · C4 · C5", "postures_run": "think:false" }, { "model": "granite4.1:30b-q8_0", "params_total": "28.9B", "params_active": null, "params_projector": null, "params_projector_source": null, "params_source": "read first-hand from the daemon's /api/ps entry for this load", "params_active_source": null, "quant": "Q8_0", "quant_source": "read first-hand from the daemon's /api/ps entry for this load", "roster_class": "28.9B q8 (seated chair-holder)", "ctx": 32768, "loaded_vram_gib": 33.3, "size_vram_bytes": 35754994564, "residency_outcome": "MEASURED", "residency_reason": null, "residency_observed_elsewhere": null, "load_s": 12.731, "protected": false, "legs": { "C1 judge": [ "think:false" ] }, "legs_run": [ "C1 judge" ], "legs_run_display": "C1", "postures_run": "think:false" }, { "model": "nemotron3:33b", "params_total": "33.0B", "params_active": null, "params_projector": null, "params_projector_source": null, "params_source": "read first-hand from the daemon's /api/ps entry for this load", "params_active_source": null, "quant": "Q4_K_M", "quant_source": "read first-hand from the daemon's /api/ps entry for this load", "roster_class": "33.0B q4 (seated chair-holder)", "ctx": 32768, "loaded_vram_gib": 23.21, "size_vram_bytes": 24922428537, "residency_outcome": "MEASURED", "residency_reason": null, "residency_observed_elsewhere": null, "load_s": 11.907, "protected": false, "legs": { "C1 judge": [ "think:false" ] }, "legs_run": [ "C1 judge" ], "legs_run_display": "C1", "postures_run": "think:false" }, { "model": "olmo-3.1:32b-think-q4_K_M", "params_total": "32.2B", "params_active": null, "params_projector": null, "params_projector_source": null, "params_source": "read first-hand from the daemon's /api/ps entry for this load", "params_active_source": null, "quant": "Q4_K_M", "quant_source": "read first-hand from the daemon's /api/ps entry for this load", "roster_class": "dense 32.2B q4 (64K-ctx model)", "ctx": 32768, "loaded_vram_gib": 18.82, "size_vram_bytes": 20204874628, "residency_outcome": "MEASURED", "residency_reason": null, "residency_observed_elsewhere": null, "load_s": 8.471, "protected": false, "legs": { "C1 judge": [ "think:false" ] }, "legs_run": [ "C1 judge" ], "legs_run_display": "C1", "postures_run": "think:false" } ], "n_rows": 11, "n_models": 9, "contamination_caveat": "Published 2026-08-13. Models with a later training cutoff may have seen these sets. We author fresh sets each cycle; this one is not a standard, it is our kit, yours to reuse.", "not_a_standard": "Our own trials, our own hardware, for our own chairs. Not first independent numbers, and not a benchmark." }