# Exhibit seventeen — the active-weights derivation The arithmetic behind every "payroll / meeting" figure on the page. Method: each model's tensor list was read from the serving runtime's own `/api/show` metadata (read-only — no model loaded, no generation call made), every tensor's element count and byte count computed from its shape and GGUF storage type, and the results summed by bucket. Cross-checks: the element sums equal each model's `general.parameter_count` exactly, and the byte sums match the on-disk blob sizes to within a tenth of a percent (0.03% for the a3b, 0.09% for gemma4:26b — the gaps are GGUF headers and, where present, externally-stored projectors). "Active per token" = (total − routed experts − vision/audio towers) + (used ÷ count × routed experts). The vision-tower exclusion for text serving is an inference from the residency arithmetic closing (section 5), not a statement by the runtime. "The inference workstation" is the 96G-VRAM-class machine that serves this workshop's products; per house rule its name, address and card model are never published. This file publishes bucket totals and the tensor-name patterns each bucket was assigned by; the per-tensor lists (417 entries for the a3b, 1,014 for gemma4:26b) are reproducible with the one command in section 6. Licence: CC BY 4.0 — name the source and link to it. ## 1 · nemotron-3.5-lightning:30b-a3b — the confounded mixture Exact `model_info` fields, verbatim keys and values: ``` general.architecture = nemotron_h_moe general.parameter_count = 32913266240 general.size_label = 128x2.5B general.file_type = 15 nemotron_h_moe.expert_count = 128 nemotron_h_moe.expert_used_count = 6 nemotron_h_moe.expert_feed_forward_length = 1856 nemotron_h_moe.expert_shared_count = 1 nemotron_h_moe.expert_shared_feed_forward_length = 3712 nemotron_h_moe.expert_group_count = 1 nemotron_h_moe.expert_group_used_count = 1 nemotron_h_moe.expert_weights_norm = True nemotron_h_moe.expert_weights_scale = 2.5 nemotron_h_moe.block_count = 53 nemotron_h_moe.embedding_length = 2688 nemotron_h_moe.context_length = 1048576 nemotron_h_moe.vocab_size = 131072 nemotron_h_moe.attention.head_count = 32 nemotron_h_moe.nextn_predict_layers = 1 nemotron_h_moe.ssm.state_size = 128 nemotron_h_moe.ssm.inner_size = 4096 details.parameter_size = "32.9B" details.quantization_level = "Q4_K_M" parameters (serving defaults) = draft_num_predict 2 / temperature 1 / top_p 0.95 requires = 0.32.9 capabilities = completion, tools, thinking ``` **There is no field named `active_parameters` anywhere in the response.** The runtime publishes `expert_count` and `expert_used_count` and leaves the active count to the reader. **Structure, from the tensor list (417 entries, FIRST-HAND).** 53 blocks, of which only **7 carry attention**, **23 carry SSM (Mamba-style) blocks**, **24 carry the MoE FFN**, and 1 is a `nextn` draft/multi-token-prediction layer. Routed experts appear as `blk.N.ffn_up_exps.weight [2688, 1856, 128] Q5_0` and `blk.N.ffn_down_exps.weight [1856, 2688, 128] Q8_0` — the trailing 128 is the expert dimension. The always-on shared expert is separate: `blk.N.ffn_up_shexp.weight [2688, 3712] Q5_0` / `blk.N.ffn_down_shexp.weight [3712, 2688] Q8_0`. **DERIVED weight accounting** (method as in the header): | bucket | tensors | parameters | bytes | GiB | share of weight bytes | |---|---:|---:|---:|---:|---:| | routed experts (128 per MoE block) | 48 | 30,651,973,632 | 23,707,385,856 | 22.079 | 93.3% | | SSM (Mamba) blocks | 184 | 891,070,784 | 583,624,448 | 0.544 | 2.3% | | output / lm head | 1 | 352,321,536 | 374,341,632 | 0.349 | 1.5% | | shared expert (always on) | 48 | 478,937,088 | 370,427,904 | 0.345 | 1.5% | | token embeddings | 1 | 352,321,536 | 242,221,056 | 0.226 | 1.0% | | attention blocks | 81 | 163,916,928 | 104,305,152 | 0.097 | 0.4% | | router (`ffn_gate_inp`, `exp_probs_b`) | 48 | 8,260,608 | 32,354,304 | 0.030 | 0.1% | | nextn draft head | 4 | 14,458,752 | 8,160,768 | 0.008 | 0.0% | | norms / misc | 2 | 5,376 | 21,504 | 0.000 | 0.0% | | **TOTAL** | **417** | **32,913,266,240** | **25,422,842,624** | **23.677** | | **Cross-check that the accounting is sound:** the summed element count equals `general.parameter_count` **exactly** (32,913,266,240 = 32,913,266,240), and the summed byte count is within 0.03% of the on-disk blob size reported by `/api/tags` (25,422,842,624 vs 25,430,749,387 — the 7.9 MB difference is GGUF header and metadata). **DERIVED active path**, at `expert_used_count` 6 of `expert_count` 128 (= 0.046875 of routed expert weight): ``` always-on = TOTAL - routed experts = 2,261,292,608 params / 1,715,456,768 B (1.598 GiB) + 6/128 of routed = 30,651,973,632 x 0.046875 = 1,436,811,264 params / 1,111,283,712 B (1.035 GiB) = ACTIVE PER TOKEN = 3,698,103,872 params / 2,826,740,480 B (2.633 GiB) active / total, by parameters = 3,698,103,872 / 32,913,266,240 = 11.24% active / total, by bytes = 2,826,740,480 / 25,422,842,624 = 11.12% -> 8.99x less weight per token ``` **The tag's own name checks out — and shows the label's hidden convention.** Strip the token embedding (352,321,536) and the output head (352,321,536) from the active count: ``` 3,698,103,872 - 352,321,536 - 352,321,536 = 2,993,460,800 = 2.99 B -> "a3b" ``` So "a3b" counts active *transformer body* parameters and silently excludes embeddings and the LM head. Including them the honest figure is 3.70 B. Both numbers are correct; they answer different questions. Both numbers are correct; they answer different questions — which is why no active-parameter label should be read without asking which convention it counts by. ## 2 · gemma4:26b — the clean mixture (the live seat) ``` general.architecture = gemma4 general.parameter_count = 25805936462 gemma4.expert_count = 128 gemma4.expert_used_count = 8 gemma4.expert_feed_forward_length = 704 gemma4.feed_forward_length = 2112 <- a DENSE FFN in every block, alongside the experts gemma4.block_count = 30 gemma4.embedding_length = 2816 gemma4.context_length = 262144 gemma4.attention.sliding_window = 1024 details.parameter_size = "25.8B" details.quantization_level = "Q4_K_M" capabilities = completion, vision, tools, thinking ``` No `nextn` layer, **no `draft_num_predict` parameter**. This is the only MoE on the box whose measured speed is not confounded by a draft head. That makes it the cleanest sparsity receipt this workshop owns (sections 4 and 5). Confirmed by tensors (1,014 entries): 90 expert tensors, e.g. `blk.N.ffn_gate_up_exps.weight [2816, 1408, 128] Q4_K` and `blk.N.ffn_down_exps.weight [704, 2816, 128] Q8_0`, **plus** a per-block dense FFN (`blk.N.ffn_gate/up/down.weight`, width 2112) that is always on — gemma4's equivalent of a shared expert. **DERIVED weight accounting:** | bucket | tensors | parameters | bytes | GiB | share | |---|---:|---:|---:|---:|---:| | routed experts (128) | 90 | 22,837,989,120 | 15,130,180,608 | 14.091 | 84.2% | | vision tower + projector | 356 | 572,794,416 | 1,193,035,968 | 1.111 | 6.6% | | attention blocks | 205 | 1,110,282,240 | 642,727,936 | 0.599 | 3.6% | | token embeddings | 1 | 738,197,504 | 605,552,640 | 0.564 | 3.4% | | dense FFN (always on) | 120 | 535,349,760 | 354,951,168 | 0.331 | 2.0% | | router | 60 | 10,897,920 | 43,591,680 | 0.041 | 0.2% | | norms / misc | 182 | 425,502 | 1,702,008 | 0.002 | 0.0% | | **TOTAL** | **1,014** | **25,805,936,462** | **17,971,742,008** | **16.737** | | Element sum equals `general.parameter_count` exactly. Byte sum is within 0.09% of the `/api/tags` blob size (17,971,742,008 vs 17,987,581,215). **DERIVED active path**, 8 of 128 routed experts (= 0.0625), **vision tower excluded** (no leg of any published bench used vision on this arm): ``` always-on (text) = TOTAL - routed - vision = 2,395,152,926 params / 1,648,525,432 B (1.535 GiB) + 8/128 of routed = 22,837,989,120 x 0.0625 = 1,427,374,320 params / 945,636,288 B (0.881 GiB) = ACTIVE PER TOKEN = 3,822,527,246 params / 2,594,161,720 B (2.416 GiB) active / total(text+vision) by bytes = 2,594,161,720 / 17,971,742,008 = 14.43% -> 6.93x ``` **This independently confirms the published roster's vendor label.** Both `chair-trials/data/roster.json` and `august-arrivals/data/residency.json` carry `"MoE 25.8B total / 3.8B active — THE LIVE SEAT"` and say the 3.8B was *transcribed, never measured*. The computed figure is **3.822 B**. The site's own licence ledger writes the same model as `gemma4:26b (a4b)` — a third label for the same quantity, rounding 3.82 up to 4. **Three published labels (3.8B active / a4b / "MoE 25.8B total") for one number that had never been computed until now.** That is the article's cleanest demonstration of why a label is not a measurement. ## 3 · nemotron3:33b — the mixture the shelf had mislabelled ``` general.architecture = nemotron_h_omni general.parameter_count = 33013666128 nemotron_h_omni.expert_count = 128 nemotron_h_omni.expert_used_count = 6 nemotron_h_omni.expert_feed_forward_length = 1856 nemotron_h_omni.expert_shared_count = 1 nemotron_h_omni.expert_shared_feed_forward_length= 3712 details.parameter_size = "33.0B" details.quantization_level= "Q4_K_M" capabilities = audio, vision, completion, tools, thinking ``` **DERIVED:** total 27,630,129,984 B (25.733 GiB) of which routed experts are 23,068,803,072 B (83.5%) and the vision/audio towers 2,882,395,200 B (10.4%). Active per token (6/128, towers excluded) = **3,580,076,352 params / 2,760,281,856 B (2.571 GiB)** — **10.01x** less weight per token than resident. The published roster and residency census originally labelled this arm `"33.0B q4 (seated chair-holder)"` with `params_active: null` — as if it were dense. Both were corrected, dated, on 2026-08-19: the chair trials' roster row and the August arrivals' residency row now carry the MoE class and the computed active count, with the original labels preserved in each kit's corrections entries. ## 4 · The dense inventory — same method Confirmed dense by two independent signals: **no `expert_*` keys in `model_info`**, and **zero tensors whose name contains `_exps`**. ### 4.1 `qwen3.8:27b` — the model exhibit sixteen measured ``` general.architecture = qwen35 general.parameter_count = 27320697856 general.size_label = "27B" general.license = apache-2.0 general.version = 0814 qwen35.block_count = 65 qwen35.embedding_length = 5120 qwen35.feed_forward_length= 17408 <- ONE dense FFN width; no expert_* keys exist qwen35.context_length = 262144 qwen35.attention.head_count = 24 ; head_count_kv = 4 qwen35.nextn_predict_layers = 1 qwen35.ssm.state_size = 128 details.parameter_size = "27.3B" ; quantization_level = "Q4_K_M" parameters (serving defaults) = draft_num_predict 4 / temperature 1 / top_k 20 / top_p 0.95 requires = 0.32.12 ``` **Expert-tensor count: 0 of 866.** Dense, definitively. **DERIVED** weight bytes 16,799,719,424 (15.65 GiB); element sum equals `general.parameter_count` exactly. Note the `/api/tags` blob size is 17,741,872,154 — the ~942 MB gap is the vision projector, which `/api/show` reports under a separate `projector_info` key and not in the tensor list. **The 17.7 GB figure exhibit sixteen prints is the full blob** (17,741,872,154 / 1e9 = 17.742), i.e. it includes the projector. Consistent with the page's own framing ("on disk"), the text-only weight mass is ~15.65 GiB. ### 4.2 `gemma4:12b` — the dense sibling of the MoE seat ``` general.architecture = gemma4 general.parameter_count= 11907350576 general.size_label = "12B" general.license = apache-2.0 general.base_model.0.name = "Gemma 4 12B" (organization: Google) gemma4.block_count = 48 gemma4.embedding_length= 3840 gemma4.feed_forward_length = 15360 <- dense; NO expert_count / expert_used_count gemma4.context_length = 262144 details.parameter_size = "11.9B" ; quantization_level = "Q4_K_M" capabilities = completion, vision, audio, tools, thinking ``` **Expert-tensor count: 0.** **DERIVED** weight bytes 7,365,558,464 (6.86 GiB); element sum equals `general.parameter_count` exactly. `/api/tags` blob 7,556,508,396 (projector again). **Why this pair is the article's best single comparison:** `gemma4:26b` and `gemma4:12b` are the *same family, same vendor, same quantization, same runtime, same card, both benched under the same counting rule* — and one is sparse and one is dense. Everything else is held as fixed as this estate can hold it. ### 4.3 The other dense arms with published speed rows All confirmed dense first-hand: `general.parameter_count` equals the summed tensor element count exactly for every row, and **every one has zero `_exps` tensors**. | tag | arch | `general.parameter_count` | quant | expert tensors | DERIVED weight bytes | GiB | |---|---|---:|---|---:|---:|---:| | `qwen3.6:27b` | qwen35 | 27,781,427,952 | Q4_K_M | **0** | 17,408,408,512 | 16.21 | | `qwen3.5:27b` | qwen35 | 27,781,427,952 | Q4_K_M | **0** | 17,408,408,512 | 16.21 | | `olmo-3.1:32b-think-q4_K_M` | olmo3 | 32,233,522,176 | Q4_K_M | **0** | 19,478,450,544 | 18.14 | | `granite4.1:30b-q8_0` | granite | 28,865,728,512 | Q8_0 | **0** | 30,671,388,672 | 28.56 | | `mistral-small3.2:24b` | mistral3 | 24,011,361,280 | Q4_K_M | **0** | 15,168,950,272 | 14.13 | | `llama3.3:70b` | llama | 70,553,706,560 | Q4_K_M | **0** | 42,512,531,712 | 39.59 | Worth noting for the article's cost section: `granite4.1:30b-q8_0` is a **28.9 B dense model at q8_0** and is the most expensive resident on the whole census — **33.30 GiB / 35,754,994,564 B at ctx 32768** — more than the 32.9 B MoE (23.64 GiB) and more than double the 27.8 B dense arms (16.24 GiB). Quantization, not architecture, is the lever there, and quantization and architecture are different levers, easily confused in a residency table. (`qwen3.6:27b` carries a 460,730,096-param vision tower inside its tensor list, unlike `qwen3.8:27b` where the projector sits outside it. Do not compare their blob sizes without saying so.) `muse-glimmer:30b` — one arm of the published stopwatch table — is not installed today and could not be re-read; its published "dense 27.9B" class is a label this file cannot verify either way. ## 5 · The bandwidth arithmetic — the dense band, and both mixtures ### 5.1 What exhibit sixteen published `reading-is-fast/index.html` (exhibit sixteen, published 2026-08-19), verbatim: > "This model has 27.3 billion parameters — those are the weights from the top of the page — > and it is **dense**, meaning every one of those numbers is used for every token. At 17.7 > gigabytes, writing 135 tokens per second would mean moving about **2.4 terabytes of weights > through the card every second**. That is more than the memory of any card in this machine's > class can physically move. Our own measurement was too fast for our own explanation." Resolution, same page: > "the big model accepted **49% of the guesser's proposals** — an average of **2.95 tokens > riding each trip through the weights**, 128 tokens written in **43 trips** instead of 128. > The four guess slots earn their keep unevenly — the first lands 86% of the time, then 56%, > then 37%, then just 16%" Backing kit: `reading-is-fast/data/runs-prefill-decode.json` (n=10, measured 2026-08-18T06:20Z, ollama 0.32.13, `num_predict` 128, `temperature` 0, `keep_alive` -1) and `reading-is-fast/data/drafthead-acceptance-receipt.md` (84 of 171 proposals accepted = 49.1%; 128 tokens in 43 forward passes; 2.95 mean accepted per pass; per-slot 0.860 / 0.558 / 0.372 / 0.163; runtime `--spec-type draft-mtp --spec-draft-n-max 4`). **Re-derived from the kit rows** (`eval_count / (eval_duration/1e9)` over all ten runs): median **135.342** tok/s (134.882–135.465); prefill median **5,415.4** tok/s (4,942.1–5,434.1). Both reproduce the page's 135.3 and 5,415 exactly. ✅ **The counting rule behind "17.7 GB":** the `/api/tags` blob size in **decimal** GB — 17,741,872,154 / 1e9 = 17.742. Units in this section are decimal GB (bytes / 1e9), named at each point of use. **Sixteen's arithmetic, reproduced:** 17.742 GB x 135.342 tok/s = **2,401 GB/s = 2.401 TB/s**. ✅ **And the draft-head resolution, reproduced:** 17.742 / 2.95 = 6.014 GB per token actually moved; x 135.342 = **814 GB/s = 0.814 TB/s**. Which "closes", as the page says. ### 5.2 The same arithmetic applied to the a3b — DERIVED Rule held identical: decimal GB of weights per token x published median decode tok/s. | arm | rule | GB/token | tok/s (tier) | required traffic | |---|---|---:|---:|---:| | a3b | **naive: all weights, as if dense** | 25.423 | 112.520 (@1k) | **2,861 GB/s = 2.86 TB/s** | | a3b | **active path (6/128 + always-on)** | 2.827 | 112.520 (@1k) | **318 GB/s = 0.318 TB/s** | | a3b | naive, all weights | 25.423 | 107.740 (@32k) | 2,739 GB/s = 2.74 TB/s | | a3b | active path | 2.827 | 107.740 (@32k) | **305 GB/s = 0.305 TB/s** | | a3b | naive, all weights | 25.423 | 120.585 (@8k) | 3,066 GB/s = 3.07 TB/s | | a3b | active path | 2.827 | 120.585 (@8k) | **341 GB/s = 0.341 TB/s** | Show-the-work for the headline row: ``` active bytes per token (from section 1's accounting) = 2,826,740,480 B = 2,826,740,480 / 1e9 = 2.8267 GB x published median decode @1k tier = 112.520 tok/s = 318.09 GB/s (0.318 TB/s) naive comparison, same tier = 25,422,842,624 / 1e9 = 25.4228 GB x 112.520 = 2,860.6 GB/s (2.86 TB/s) ratio = 8.99x ``` ### 5.3 `gemma4:26b` — the cleanest one, because it has no draft head | arm | rule | GB/token | tok/s | required traffic | |---|---|---:|---:|---:| | gemma4:26b | naive: all weights incl. vision | 17.972 | 124.975 (@32k, MEASURED) | **2,246 GB/s = 2.25 TB/s** | | gemma4:26b | naive: text weights only | 16.779 | 124.975 | 2,097 GB/s = 2.10 TB/s | | gemma4:26b | **active path (8/128 + always-on)** | 2.594 | 124.975 | **324 GB/s = 0.324 TB/s** | | gemma4:26b | active path | 2.594 | 138.600 — a production observation, not a bench cell: derived p50 over n=1,982 live rulings, 2026-08-15, same eval_count/eval_duration definition | 360 GB/s = 0.360 TB/s | ### 5.4 The falsification test the shelf already ran Apply the identical rule to the dense arms with **no** draft head, from the same C5 leg: | arm | GB/token (full blob) | tok/s | required traffic | |---|---:|---:|---:| | `qwen3.6:27b` (dense, no drafter) | 17.408 | 66.485 (@1k) | **1,157 GB/s = 1.16 TB/s** | | `qwen3.5:27b` (dense, no drafter) | 17.408 | 66.450 (@1k) | 1,157 GB/s = 1.16 TB/s | | `muse-glimmer:30b` (no drafter) | 16.750 (resident, blob unavailable) | 68.195 (@1k) | 1,142 GB/s = 1.14 TB/s | | `gemma4:12b` (dense, no drafter) | 7.366 | 105.665 (@1k) | 778 GB/s = 0.78 TB/s | | `qwen3.6:27b` @32k | 17.408 | 61.305 | 1,067 GB/s = 1.07 TB/s | Two tensor-verified dense arms (the qwen 27.8 Bs — sibling versions of one architecture, so not independent evidence) land at 1.16 TB/s at the 1k tier and 1.07 at 32k; a third arm published as dense lands beside them at 1.14, but its tag is no longer installed and could not be re-read. The dense 11.9 B lands well below the band at 0.78 (1k tier). The band is a measured floor on what this card achieves — not a spec sheet. Then: - every **dense, no-drafter** arm's naive arithmetic **closes** inside that band; - `qwen3.8:27b` (dense, **with** a 4-deep draft head) needs 2.40 TB/s naively — **breaks**, and is resolved by fewer trips (2.95 tokens per trip → 0.81 TB/s); - `gemma4:26b` (**sparse**, no draft head) needs 2.25 TB/s naively — **breaks**, and is resolved only by fewer *bytes* per trip (→ 0.32 TB/s); - the a3b (**sparse and** a 2-deep draft head) needs 2.86 TB/s naively — breaks hardest. **Two different mechanisms, one shared symptom, one shared cure: stop assuming every token pays for every weight.** Honesty check: achieved traffic is always *below* peak, so the band is a floor on card capability — 2.25 TB/s is roughly twice the highest achieved dense figure in these rows. A strong argument, not a physical proof; no card spec is published here, by house rule. --- ## 6 · Reproduce it The metadata reads, against any ollama endpoint (nothing loads, nothing generates): ``` curl -s http://localhost:11434/api/show -d '{"model":""}' ``` The response carries `details`, `model_info` (the `expert_count` / `expert_used_count` fields quoted above), and the full `tensors` list — name, GGUF type, shape — from which every bucket row in this file re-derives: routed experts are the tensors named `*_exps.*` (their trailing dimension is the expert count), the shared expert is `*_shexp.*`, and everything else is always-on. Element count = the product of a tensor's shape; bytes follow from the GGUF type's bits per element. Twenty tags were enumerated on the inference workstation on 2026-08-19; exactly three carried `expert_*` keys — the three in this file.