{
 "schema_version": "1.0",
 "page_kind": "article",
 "pack_kind": "generate",
 "slug": "three-at-the-table",
 "title": "Everyone on the payroll, three at the table — dense, MoE, and active weights",
 "dek": "Model names grew a second number — 30b-a3b — and it is the one your card and wallet care about. The payroll and the meeting, in plain English, measured on our own machines: a 25.8B mixture out-writing its 11.9B dense sibling while holding twice the memory — and the robber asks both architectures.",
 "published": "2026-08-20",
 "series": [
  "guide"
 ],
 "licence": "CC BY 4.0",
 "status": "pending-judge",
 "pack_note": "no judge is seated, so every chip is held",
 "notice": "items with published:false are held and are not this page's published words",
 "source": {
  "url": "https://research.strata2signal.com/three-at-the-table/",
  "md_url": "https://research.strata2signal.com/three-at-the-table/index.md",
  "md_sha": "12873061e23008eb46ef727f3390a73962d29c88f59e9f629cd414961f8e8cb2",
  "html_sha": "41f223df385e3feb26a03f1cc2cc196870524a325a6ec5d075843cf91a4cfc44",
  "anchors_sha": "75f7982c71ef473b1d5c50ed3cadca28abc5f950c5c4db3d565626920c5e7f70"
 },
 "short": {
  "paragraph": "Model names grew a second number - 30b-a3b - and it is the one your card and wallet care about. The payroll and the meeting, in plain English, measured on our own machines: a 25.8B mixture out-writing its 11.9B dense sibling while holding twice the memory - and the robber asks both architectures.",
  "from_dek": true,
  "counts": {
   "words": 4475,
   "minutes": 20,
   "tables": 2,
   "kit": true
  },
  "bullets": [
   {
    "text": "the true active share for the live seat is 14.4%",
    "figure": "14.4%",
    "cite": "the-payroll-and-the-meeting",
    "quote": "Take our live seat: 8 of 128 experts is 6.25% of the expert weight, but the true active share is 14.4% of the model's bytes, once the always-on staff are counted.",
    "span": [
     5374,
     5382
    ]
   },
   {
    "text": "the dense 27.8 Bs are out-written by the mixtures by 2.0×",
    "figure": "2.0×",
    "cite": "measured-on-our-own-machines",
    "quote": "And both out-write the dense 27.8 Bs by a wide margin - 2.0× and 1.8× - while their meetings run a seventh and a ninth the size of their own staff by bytes (the 14.4% and 11.1% figures above).",
    "span": [
     8674,
     8679
    ]
   },
   {
    "text": "the demand for the MoE falls to 0.32 TB/s",
    "figure": "0.32 TB/s",
    "cite": "the-arithmetic-that-catches-a-mixture-red-handed",
    "quote": "On the active path - eight experts plus the always-on staff, 2.6 decimal GB per token, the vision tower excluded for the same reason the residency arithmetic excludes it - the demand falls to 0.32 TB/s.",
    "span": [
     12934,
     12946
    ]
   }
  ]
 },
 "sections": [
  {
   "id": "the-tag-with-two-numbers",
   "heading": "The tag with two numbers",
   "level": 2,
   "span": [
    1786,
    3067
   ],
   "chunks": [
    [
     1786,
     3067
    ]
   ],
   "chars": 1281,
   "digest": "the tag with two numbers distinguishes between the payroll, which is the parameters hired and trained, and the meeting, which are the parameters that show up for any given token. a mixture of experts shrinks the trip, not the staff.",
   "digest_skipped": null
  },
  {
   "id": "the-payroll-and-the-meeting",
   "heading": "The payroll and the meeting",
   "level": 2,
   "span": [
    3067,
    5706
   ],
   "chunks": [
    [
     3067,
     5706
    ]
   ],
   "chars": 2639,
   "digest": "an MoE replaces one wide feed-forward network with many narrower experts. the router scores which experts handle each token. the meeting is larger than the invitation list because attention layers and shared experts are always-on.",
   "digest_skipped": null
  },
  {
   "id": "the-part-nobody-warns-you-about-the-whole-payroll-needs-an",
   "heading": "The part nobody warns you about: the whole payroll needs an office",
   "level": 2,
   "span": [
    5706,
    7048
   ],
   "chunks": [
    [
     5706,
     7048
    ]
   ],
   "chars": 1342,
   "digest": "memory is priced by the payroll while speed is priced by the meeting. an MoE provides a bigger model's knowledge at a smaller model's speed, provided the user can afford a bigger model's memory.",
   "digest_skipped": null
  },
  {
   "id": "measured-on-our-own-machines",
   "heading": "Measured on our own machines",
   "level": 2,
   "span": [
    7048,
    11621
   ],
   "chunks": [
    [
     7048,
     11621
    ]
   ],
   "chars": 4573,
   "digest": "benchmarks show that both mixtures out-write the dense 11.9 B and the dense 27.8 Bs. the gemma family shows that a sparse mixture can write faster than its own dense sibling.",
   "digest_skipped": null
  },
  {
   "id": "the-arithmetic-that-catches-a-mixture-red-handed",
   "heading": "The arithmetic that catches a mixture red-handed",
   "level": 2,
   "span": [
    11621,
    13613
   ],
   "chunks": [
    [
     11621,
     13613
    ]
   ],
   "chars": 1992,
   "digest": "arithmetic shows that a model's speed can appear to exceed what its weights allow. by counting what a token actually touches, the demand falls to 0.32 TB/s for the MoE.",
   "digest_skipped": null
  },
  {
   "id": "what-the-label-doesnt-tell-you",
   "heading": "What the label doesn't tell you",
   "level": 2,
   "span": [
    13613,
    15748
   ],
   "chunks": [
    [
     13613,
     15748
    ]
   ],
   "chars": 2135,
   "digest": "no field in runtime metadata says how many parameters are active. vendors use different bookkeeping rules for labels, and the authors computed their own meeting sizes from tensor shapes.",
   "digest_skipped": null
  },
  {
   "id": "the-number-that-looks-like-sparsity-and-isnt",
   "heading": "The number that looks like sparsity and isn't",
   "level": 2,
   "span": [
    15748,
    16787
   ],
   "chunks": [
    [
     15748,
     16787
    ]
   ],
   "chars": 1039,
   "digest": "a model variant reporting 2.34 GiB resident where the same weights served without a draft companion report 15.60 GiB is an open anomaly that is likely a reporting glitch rather than sparsity.",
   "digest_skipped": null
  },
  {
   "id": "what-sparsity-does-not-buy",
   "heading": "What sparsity does not buy",
   "level": 2,
   "span": [
    16787,
    18788
   ],
   "chunks": [
    [
     16787,
     18788
    ]
   ],
   "chars": 2001,
   "digest": "sparsity does not guarantee smarts or speed. tokens per second measure how fast words arrive, not whether they are worth reading. quantization and attention geometry also affect memory and speed.",
   "digest_skipped": null
  },
  {
   "id": "the-model-that-was-proving-it-all-along",
   "heading": "The model that was proving it all along",
   "level": 2,
   "span": [
    18788,
    20458
   ],
   "chunks": [
    [
     18788,
     20458
    ]
   ],
   "chars": 1670,
   "digest": "gemma4:26b is the live seat that narrates the game world. the workshop's own notebooks predicted it would decode faster than a dense 12b model.",
   "digest_skipped": null
  },
  {
   "id": "the-robber-asked-a-third-time",
   "heading": "The robber, asked a third time",
   "level": 2,
   "span": [
    20458,
    23427
   ],
   "chunks": [
    [
     20458,
     23427
    ]
   ],
   "chars": 2969,
   "digest": "testing the robber question shows that the mixture writes about three-quarters again as fast as the dense model. the two architectures disagree about the rule.",
   "digest_skipped": null
  },
  {
   "id": "what-this-page-does-not-say",
   "heading": "What this page does not say",
   "level": 2,
   "span": [
    23427,
    24478
   ],
   "chunks": [
    [
     23427,
     24478
    ]
   ],
   "chars": 1051,
   "digest": "this page does not claim MoE causes the observed speeds, nor does it rank the stopwatch column or vouch for big cloud models.",
   "digest_skipped": null
  },
  {
   "id": "what-to-take-with-you",
   "heading": "What to take with you",
   "level": 2,
   "span": [
    24478,
    25781
   ],
   "chunks": [
    [
     24478,
     25781
    ]
   ],
   "chars": 1303,
   "digest": "the payroll and meeting define the tag. mixtures can out-write smaller dense models. the expert fraction understates the real meeting, and fast is not always smart.",
   "digest_skipped": null
  },
  {
   "id": "how-to-check-our-work-and-see-it-live",
   "heading": "How to check our work — and see it live",
   "level": 2,
   "span": [
    25781,
    27112
   ],
   "chunks": [
    [
     25781,
     27112
    ]
   ],
   "chars": 1331,
   "digest": null,
   "digest_skipped": "credits"
  },
  {
   "id": "the-rest-of-the-seminar",
   "heading": "The rest of the seminar",
   "level": 2,
   "span": [
    27112,
    28410
   ],
   "chunks": [
    [
     27112,
     28410
    ]
   ],
   "chars": 1298,
   "digest": null,
   "digest_skipped": "credits"
  }
 ],
 "chips": [
  {
   "id": "c-1d05dfd0",
   "q": "what do the two numbers in a model tag represent?",
   "a": "The first number represents the payroll, or the total parameters trained and taking up space. The second number represents the meeting, or the parameters that actually participate in processing any given token.",
   "cites": [
    "the-tag-with-two-numbers"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-8238644e",
   "q": "how do mixture of experts models differ from dense models?",
   "a": "A dense model has one wide feed-forward network per block that every token flows through. An MoE replaces that single network with many narrower experts and a router that selects which few experts handle each token.",
   "cites": [
    "the-payroll-and-the-meeting"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-93d8e9a9",
   "q": "why are vendor labels for active parameters unreliable?",
   "a": "No runtime metadata explicitly states active parameters; vendors use different bookkeeping conventions. For the same model, one label might result in 3.70 billion while another results in 2.99 billion depending on whether token embeddings and output heads are included.",
   "cites": [
    "what-the-label-doesnt-tell-you"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-61921701",
   "q": "what relationship exists between sparsity and model intelligence?",
   "a": "Sparsity is a speed limit raised, not a speed guaranteed. High decode speeds do not equate to higher intelligence, as some fast models may miss judge-exam floors or post lower win-rates in narration tasks compared to dense models.",
   "cites": [
    "what-sparsity-does-not-buy"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  }
 ],
 "related": [
  {
   "slug": "the-compressed-photograph",
   "why": "shares ground with § The arithmetic that catches a mixture red-handed · § The same brain at two compressions"
  },
  {
   "slug": "reading-is-fast",
   "why": "shares ground with § The arithmetic that catches a mixture red-handed · § A measurement, from our own card"
  },
  {
   "slug": "two-used-3080s-priced",
   "why": "shares ground with § Measured on our own machines · § How fast does each one write?"
  }
 ],
 "thanks": "",
 "kit": {
  "url": "https://research.strata2signal.com/three-at-the-table/data/",
  "licence": "CC BY 4.0",
  "files": [
   "active-weights-derivation.md",
   "robber-asks.json",
   "STOPWATCH-PROTOCOL.md",
   "STOPWATCH-WINDOW-RUN-2026-08-21.md",
   "stopwatch-window-runs.json",
   "CONTENTION-EVIDENCE.txt"
  ]
 },
 "seat_class": {
  "writer": "gemma-class",
  "judge": null,
  "writer_runtime": "vllm"
 },
 "bench": {
  "bullets_written": 3,
  "bullets_kept": 3,
  "digests_written": 12,
  "digests_kept": 12,
  "chips_written": 7,
  "chips_kept": 4,
  "chips_grounded": 0,
  "chips_published": 0,
  "dropped_by": {
   "new_noun": 0,
   "figure": 0,
   "length": 3,
   "cite": 0,
   "judge": 0,
   "quote": 0,
   "directive": 0,
   "redaction": 0,
   "profanity": 0
  },
  "new_noun_tokens_checked": [
   "0.32",
   "11.9",
   "128",
   "12b",
   "14.4%",
   "15.60",
   "2.0×",
   "2.34",
   "2.99",
   "25.8",
   "26b",
   "27.8",
   "3.70",
   "6.25%",
   "8",
   "B",
   "Bs",
   "GiB",
   "MoE",
   "TB/s",
   "gemma4"
  ],
  "new_noun_tokens_withheld": 0,
  "figure_definition": "v3",
  "drops": [
   {
    "kind": "chip",
    "reason": "length",
    "item": "how does memory usage compare between MoE and dense models?",
    "detail": "q not lowercase"
   },
   {
    "kind": "chip",
    "reason": "length",
    "item": "how do MoE models compare to dense models in decode speed?",
    "detail": "q not lowercase"
   },
   {
    "kind": "chip",
    "reason": "length",
    "item": "how can the true active share of an MoE be calculated?",
    "detail": "q not lowercase"
   }
  ]
 },
 "generated_utc": "2026-09-09T01:28:05Z"
}