{
 "schema_version": "1.0",
 "page_kind": "article",
 "pack_kind": "generate",
 "slug": "draft-heads-and-speculative-decoding",
 "title": "Draft heads and speculative decoding, in plain words",
 "dek": "A language model writes one token for every trip through its weights unless a drafter guesses a few tokens ahead and the model checks the guesses in one trip; this guide says in plain words who does the guessing, why acceptance is three different numbers with one name, why tokens a round is a ceiling and not the speed-up, and why a speed figure that does not say whether a drafter was on cannot be compared with another, on receipts we have already published.",
 "published": "2026-11-12",
 "series": [
  "guide"
 ],
 "licence": "CC BY 4.0",
 "status": "pending-judge",
 "pack_note": "no judge is seated, so every chip is held",
 "notice": "items with published:false are held and are not this page's published words",
 "source": {
  "url": "https://research.strata2signal.com/draft-heads-and-speculative-decoding/",
  "md_url": "https://research.strata2signal.com/draft-heads-and-speculative-decoding/index.md",
  "md_sha": "f5babaf6279d680e1fa764ed1e5a19a2d6c22b4e5e933a640ba43c75b17afadb",
  "html_sha": "6e977a9c72aea06446521ece8e73075e95d76acfadadf93dcc411683fd37b63e",
  "anchors_sha": "50294df262e514e9cbea4b02f821cc2cb9575e150d26ae4e4868bc538ea6efa9"
 },
 "short": {
  "paragraph": "A model writes one token per trip through its weights, and the trip is the slow part. A drafter guesses a few tokens ahead; the model checks all the guesses in one trip and keeps the run of right ones, so one trip can yield two, three or five tokens. Done by the textbook rule, the answer comes from the same distribution as without the drafter, within the card's arithmetic. How much it helps depends on the text, the temperature, the drafter and how many people share the card: on one dense model of ours, switching the draft head off cost 25 to 30 per cent of its writing speed; on another, one engine wrote twice as fast as a second while taking fewer steps a second, because each step carried 2.6 tokens. Our rule from this page on: every speed figure says which drafter was on, how deep it guessed, and what the engine's own log accepted.",
  "from_dek": false,
  "counts": {
   "words": 4651,
   "minutes": 21,
   "tables": 5,
   "kit": false
  },
  "bullets": []
 },
 "sections": [
  {
   "id": "why-writing-is-the-slow-half",
   "heading": "Why writing is the slow half",
   "level": 2,
   "span": [
    2601,
    3751
   ],
   "chunks": [
    [
     2601,
     3751
    ]
   ],
   "chars": 1150,
   "digest": "writing is slower than reading because a model must perform a trip through the weights for every token it produces. on a 27-billion-parameter dense model, reading was forty times faster than writing. writing speed is largely determined by how fast a card can move memory.",
   "digest_skipped": null
  },
  {
   "id": "guess-then-check",
   "heading": "Guess, then check",
   "level": 2,
   "span": [
    3751,
    6296
   ],
   "chunks": [
    [
     3751,
     6296
    ]
   ],
   "chars": 2545,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "who-does-the-guessing",
   "heading": "Who does the guessing",
   "level": 2,
   "span": [
    6296,
    10917
   ],
   "chunks": [
    [
     6296,
     10917
    ]
   ],
   "chars": 4621,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "acceptance-three-numbers-with-one-name",
   "heading": "Acceptance: three numbers with one name",
   "level": 2,
   "span": [
    10917,
    13845
   ],
   "chunks": [
    [
     10917,
     13845
    ]
   ],
   "chars": 2928,
   "digest": "acceptance refers to several quantities, including the ratio of accepted guesses to proposed guesses, tokens a round, and acceptance by slot. a confidence gate can stop a drafter from adding guesses it is unsure of, which raises the share accepted by proposing less.",
   "digest_skipped": null
  },
  {
   "id": "tokens-a-round-is-a-ceiling-not-the-speed-up",
   "heading": "Tokens a round is a ceiling, not the speed-up",
   "level": 2,
   "span": [
    13845,
    17341
   ],
   "chunks": [
    [
     13845,
     17341
    ]
   ],
   "chars": 3496,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "why-every-speed-figure-needs-a-drafter-label",
   "heading": "Why every speed figure needs a drafter label",
   "level": 2,
   "span": [
    17341,
    20612
   ],
   "chunks": [
    [
     17341,
     20612
    ]
   ],
   "chars": 3271,
   "digest": "speed figures require drafter labels because drafting can change a figure by a quarter or more, ties speed to the specific text being written, and can confound comparisons. additionally, the term off must specify what is actually disabled.",
   "digest_skipped": null
  },
  {
   "id": "when-it-helps-and-when-it-does-not",
   "heading": "When it helps, and when it does not",
   "level": 2,
   "span": [
    20612,
    23748
   ],
   "chunks": [
    [
     20612,
     23748
    ]
   ],
   "chars": 3136,
   "digest": "drafting benefits depend on several factors, including how many people share a card, the temperature, the text being written, sampler settings, and prompt length. it can be slower in compute-bound workloads or when the card has no arithmetic to spare.",
   "digest_skipped": null
  },
  {
   "id": "reading-someone-elses-speed-claim",
   "heading": "Reading someone else's speed claim",
   "level": 2,
   "span": [
    23748,
    24783
   ],
   "chunks": [
    [
     23748,
     24783
    ]
   ],
   "chars": 1035,
   "digest": "to read a speed claim, one should ask which drafter was used, how acceptance was measured, the temperature, whether it was one stream or many, the prompt length, and the specific hardware, engine, and version used.",
   "digest_skipped": null
  },
  {
   "id": "what-this-cannot-say",
   "heading": "What this cannot say",
   "level": 2,
   "span": [
    24783,
    26475
   ],
   "chunks": [
    [
     24783,
     26475
    ]
   ],
   "chars": 1692,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "who-ran-this-and-thanks",
   "heading": "Who ran this, and thanks",
   "level": 2,
   "span": [
    26475,
    27632
   ],
   "chunks": [
    [
     26475,
     27632
    ]
   ],
   "chars": 1157,
   "digest": null,
   "digest_skipped": "credits"
  },
  {
   "id": "sources",
   "heading": "Sources",
   "level": 2,
   "span": [
    27632,
    32632
   ],
   "chunks": [
    [
     27632,
     32632
    ]
   ],
   "chars": 5000,
   "digest": null,
   "digest_skipped": null
  }
 ],
 "chips": [
  {
   "id": "c-7cdd5dfb",
   "q": "how does writing speed differ from reading speed in language models?",
   "a": "Reading a prompt requires one trip through the weights for the whole prompt, whereas writing requires one trip per token because the next token does not exist until it is chosen. This asymmetry results in reading being significantly faster than writing.",
   "cites": [
    "why-writing-is-the-slow-half"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-d6bd9f91",
   "q": "how does the verification process work for a drafter?",
   "a": "A model runs one verify round over a proposal, scoring every position at once. The engine keeps guesses that match what the model would have chosen and keeps the model's own token at the first mismatch, throwing the rest away.",
   "cites": [
    "guess-then-check"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-c40879ed",
   "q": "what different kinds of drafters exist for language models?",
   "a": "Drafters include separate small models, extra output layers like Medusa or multi-token prediction heads, feature-reading networks, block drafters, and methods using no model at all, such as n-gram patterns or prompt lookup.",
   "cites": [
    "who-does-the-guessing"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-51d99eaf",
   "q": "what different meanings can the term acceptance have?",
   "a": "Acceptance can refer to the ratio of guesses accepted to guesses proposed, the number of tokens produced per round, the survival rate of guesses by specific slot, or a single implied rate used in textbook formulas.",
   "cites": [
    "acceptance-three-numbers-with-one-name"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-e90d5d79",
   "q": "how does tokens a round relate to actual speed-up?",
   "a": "Tokens a round is a theoretical ceiling rather than the actual speed-up. The drafter adds cost, the verify round is more intensive than a single-token step, and the engine must perform bookkeeping between rounds.",
   "cites": [
    "tokens-a-round-is-a-ceiling-not-the-speed-up"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-6b27f884",
   "q": "what information should accompany every writing-speed figure?",
   "a": "Every figure should include the drafter type, depth and gate settings, acceptance metrics from the engine's log, temperature, and the number of simultaneous streams.",
   "cites": [
    "why-every-speed-figure-needs-a-drafter-label"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-76442266",
   "q": "what factors influence whether drafting provides a speed benefit?",
   "a": "Benefits depend on the number of users sharing a card, the temperature, the nature of the text being written, sampler settings, and prompt length and memory availability.",
   "cites": [
    "when-it-helps-and-when-it-does-not"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-022432d4",
   "q": "what questions help evaluate a speed claim for a model?",
   "a": "Evaluators should ask which drafter was used, what the engine's log reported for acceptance, the temperature, the number of streams, the prompt length, and the specific engine, version, and hardware used.",
   "cites": [
    "reading-someone-elses-speed-claim"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  }
 ],
 "related": [
  {
   "slug": "reading-is-fast",
   "why": "shares ground with § Why writing is the slow half · § Writing is one at a time"
  },
  {
   "slug": "how-the-speed-tool-works",
   "why": "shares ground with § Why writing is the slow half · § What each box means"
  },
  {
   "slug": "the-typist-and-the-developer",
   "why": "shares ground with § Why writing is the slow half · § The typist"
  }
 ],
 "thanks": "## Who ran this, and thanks {#who-ran-this-and-thanks}\n\nThanks to Yaniv Leviathan, Matan Kalman and Yossi Matias, and to Charlie Chen and colleagues at DeepMind, for the two papers that made the method lawful; to the authors of Blockwise Parallel Decoding, Medusa, EAGLE, DFlash, DSpark, MagicDec and TurboSpec, and of Meta's and DeepSeek's multi-token-prediction work, whose figures are quoted here with their dates; to the vLLM team for writing down where drafting hurts; to Qwen and DeepSeek for their models and cards, and to Unsloth for publishing the drafter's file sizes; to the llama.cpp project, the ollama project and the Strata engine's author for engines that print their own acceptance counts, which is the whole reason this page can be written. The August receipts came from ollama's verbose log and a bracketed run in a declared maintenance window; the October receipts from the Strata engine's and llama.cpp's own serve lines, recorded by the full bench. A small human team asked for this page and will approve it; a fleet of AI agents read the sources, re-ran the arithmetic from the public files and drafted it under that team's rulings.",
 "kit": null,
 "seat_class": {
  "writer": "gemma-class",
  "judge": null,
  "writer_runtime": "vllm"
 },
 "bench": {
  "bullets_written": 3,
  "bullets_kept": 0,
  "digests_written": 10,
  "digests_kept": 5,
  "chips_written": 8,
  "chips_kept": 8,
  "chips_grounded": 0,
  "chips_published": 0,
  "dropped_by": {
   "new_noun": 1,
   "figure": 0,
   "length": 0,
   "cite": 0,
   "judge": 0,
   "quote": 0,
   "directive": 0,
   "redaction": 0,
   "profanity": 0,
   "identifier": 0,
   "bare_figure": 0,
   "house_voice": 0,
   "editorial": 0,
   "hand-edit": 7
  },
  "new_noun_tokens_checked": [
   "041",
   "1",
   "1.0",
   "1.34×",
   "1.43×",
   "2026",
   "27-billion-parameter",
   "85%",
   "90%",
   "DSpark",
   "FLy",
   "Medusa",
   "Qwen3.8-Flash-Next",
   "Strata"
  ],
  "new_noun_tokens_withheld": 1,
  "figure_definition": "v3",
  "figure_boundary": "guarded",
  "drops": [
   {
    "kind": "bullet",
    "reason": "new_noun",
    "item": null,
    "item_chars": 3,
    "withheld": "a string naming something the article does not",
    "detail": ""
   },
   {
    "kind": "bullet",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 55,
    "text_sha": "7136a2941484",
    "figure": "1.43",
    "withheld": "one bullet removed from this pack by hand",
    "removed_utc": "2026-11-12",
    "why": "outside the ±3 % tolerance set for the run, so the ratios are the run's reading, not a ratified figure",
    "detail": "removed from this pack by hand on 2026-11-12 (UTC), after the writer ran and after the gates had counted: outside the ±3 % tolerance set for the run, so the ratios are the run's reading, not a ratified figure. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "bullet",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 74,
    "text_sha": "9bab5384427b",
    "figure": "90",
    "withheld": "one bullet removed from this pack by hand",
    "removed_utc": "2026-11-12",
    "why": "Every figure is one we have already published or a maker's own, with its date beside it, or our own arithmetic on those, marked as ours.",
    "detail": "removed from this pack by hand on 2026-11-12 (UTC), after the writer ran and after the gates had counted: Every figure is one we have already published or a maker's own, with its date beside it, or our own arithmetic on those, marked as ours. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 262,
    "text_sha": "71607f37069e",
    "section": "guess-then-check",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-11-12",
    "why": "DeepMind's paper of February 2023 adds the caveat every honest statement carries: the scheme \"preserves the distribution of the target model within hardware numerics\"",
    "detail": "removed from this pack by hand on 2026-11-12 (UTC), after the writer ran and after the gates had counted: DeepMind's paper of February 2023 adds the caveat every honest statement carries: the scheme \"preserves the distribution of the target model within hardware numerics\". Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 278,
    "text_sha": "566853d50b5b",
    "section": "who-does-the-guessing",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-11-12",
    "why": "It couples \"a parallel backbone with a lightweight sequential module\" to \"mitigate suffix decay\", the way a block drafter's later guesses go wrong",
    "detail": "removed from this pack by hand on 2026-11-12 (UTC), after the writer ran and after the gates had counted: It couples \"a parallel backbone with a lightweight sequential module\" to \"mitigate suffix decay\", the way a block drafter's later guesses go wrong. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 276,
    "text_sha": "97c3d89f8bd3",
    "section": "tokens-a-round-is-a-ceiling-not-the-speed-up",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-11-12",
    "why": "outside the ±3 % tolerance set for the run, so the ratios are the run's reading, not a ratified figure",
    "detail": "removed from this pack by hand on 2026-11-12 (UTC), after the writer ran and after the gates had counted: outside the ±3 % tolerance set for the run, so the ratios are the run's reading, not a ratified figure. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 201,
    "text_sha": "3f4fd4b1e5ec",
    "section": "what-this-cannot-say",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-11-12",
    "why": "Every DSpark figure on this page is DeepSeek's own, from its paper, or a file size from Unsloth's listing; none is a reading of ours.",
    "detail": "removed from this pack by hand on 2026-11-12 (UTC), after the writer ran and after the gates had counted: Every DSpark figure on this page is DeepSeek's own, from its paper, or a file size from Unsloth's listing; none is a reading of ours. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 173,
    "text_sha": "ad86ab26e7de",
    "section": "sources",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-11-12",
    "why": "An unrelated project; strata2signal has no affiliation with it.",
    "detail": "removed from this pack by hand on 2026-11-12 (UTC), after the writer ran and after the gates had counted: An unrelated project; strata2signal has no affiliation with it. Not a gate drop, and recorded here so the ledger adds up."
   }
  ]
 },
 "generated_utc": "2026-11-12T00:56:35Z"
}