# Draft heads and speculative decoding, in plain words *A language model writes one token for every trip through its weights, unless something guesses a few tokens ahead and the model checks the guesses in one trip. That trick is speculative decoding, and the guesser is a drafter: a second small model, a head the model ships with, or no model at all. This guide says in plain words how the check works, who does the guessing, why "acceptance" is three different numbers with one name, why tokens a round is a ceiling and not the speed-up, and why a speed figure that does not say whether a drafter was on cannot be compared with any other. Every figure is one we have already published or a maker's own, with its date beside it, or our own arithmetic on those, marked as ours.* *Published 2026-11-12 (UTC) · A small (human) team and a fleet of AI agents.* **the short version:** A model writes one token per trip through its weights, and the trip is the slow part. A drafter guesses a few tokens ahead; the model checks all the guesses in one trip and keeps the run of right ones, so one trip can yield two, three or five tokens. Done by the textbook rule, the answer comes from the same distribution as without the drafter, within the card's arithmetic. How much it helps depends on the text, the temperature, the drafter and how many people share the card: on one dense model of ours, switching the draft head off cost 25 to 30 per cent of its writing speed; on another, one engine wrote twice as fast as a second while taking fewer steps a second, because each step carried 2.6 tokens. Our rule from this page on: every speed figure says which drafter was on, how deep it guessed, and what the engine's own log accepted. 4,651 words · about 21 minutes (at 220 words/min) · 5 tables · data kit: no https://research.strata2signal.com/draft-heads-and-speculative-decoding/ --- *If you're new here: [strata→signal](https://strata2signal.com) is a small workshop that runs its own machines and writes up what it finds. This page follows [Reading is fast, writing is slow](https://research.strata2signal.com/reading-is-fast/), which caught our own model guessing ahead, and sits beside [Mixture of experts, in plain words](https://research.strata2signal.com/mixture-of-experts-in-plain-words/), the other lawful way a model writes faster than its weights allow. Nothing was measured for this page: every number is printed first on a page of ours, in its public data files, or in a maker's own paper or model card, read on the date beside it, or is our own arithmetic on those, marked as ours.* ## Why writing is the slow half {#why-writing-is-the-slow-half} A **token** is the unit a language model reads and writes: a word or a piece of one. The model's **weights** are the billions of numbers that make it up, and to choose one token the card must pull all of them through its arithmetic once: a **trip through the weights**. Reading a prompt is one trip for the whole prompt, because every word of it already exists and can be worked on at once. Writing is one trip per token, because the next token does not exist until the model has chosen it. Our page [Reading is fast, writing is slow](https://research.strata2signal.com/reading-is-fast/) measured that asymmetry on a 27-billion-parameter dense model: reading forty times faster than writing, on the same card, in the same second. Writing speed is mostly set by how fast the card can move memory, and most of the arithmetic the card could do on each trip goes unused. That unused arithmetic is the opening. If the model could be handed several candidate tokens at once, it could score all of them in one trip, the way it scores a prompt. It only needs someone to guess what they are. ## Guess, then check {#guess-then-check} A **drafter** is anything that proposes the next few tokens cheaply. The model then runs one trip over the whole proposal, which is called a **verify round**, and scores every position at once: what it would have chosen after the prompt, what it would have chosen after the first guess, after the second, and so on. The engine walks along the guesses and keeps them while they match what the model would have chosen; at the first mismatch it keeps the model's own token for that position and throws the rest away. Because the model also scored the position after the last guess, a round where every guess is right yields one token more than the number of guesses. | The drafter guesses four | The model's check | Tokens from this one trip | |---|---|---| | All four match | Keep four, plus the model's own fifth | 5 | | The first two match, the third does not | Keep two, plus the model's own third | 3 | | The first guess misses | Keep the model's own first | 1 | A trip never yields fewer tokens than it would have without the drafter, and the check is what makes the speed lawful. The 2022 paper that named the method, by Leviathan, Kalman and Matias at Google, proved that with the right acceptance rule the output is sampled "without changing the distribution" of the big model, and showed "a 2X-3X acceleration" on T5-XXL "with identical outputs". DeepMind's paper of February 2023 adds the caveat every honest statement carries: the scheme "preserves the distribution of the target model within hardware numerics", and the vLLM serving project's documentation (read 2026-11-11) says the same, "theoretically lossless up to the precision limits of hardware numerics". At temperature zero, where the model always takes its single most likely token, our August page called the check exact; the fuller statement is exact in the algorithm, and the same within the hardware's arithmetic. One engine's author says what that can mean: with some quantised files, "the same prompt at temperature 0 can end in a different, equally good answer when drafting, cache state or a resumed conversation differ". Not every scheme keeps the distribution. Medusa (January 2024) offered, in place of the acceptance rule, an optional "typical acceptance scheme", arguing that "it is typically unnecessary to match the distribution of the original model"; vLLM's "FLy" mode "deliberately relaxes verification and is not distribution preserving". A speed figure from a lossy scheme is a figure for a different model. ## Who does the guessing {#who-does-the-guessing} The drafter family has grown since 2018, and the kinds matter because they ship differently, cost different memory and are labelled differently in a speed claim. | Kind | What guesses | Ships with the model? | Example | Keeps the distribution? | Source | |---|---|---|---|---|---| | A separate small model | A second, smaller language model, one token at a time | No, a second file | T5-small drafting for T5-XXL | Yes, by the acceptance rule | Leviathan et al. 2022; Chen et al. 2023 | | Heads the model carries: blockwise, Medusa | Extra output layers that predict several positions at once | Yes, trained onto the model | Medusa heads on Vicuna | Blockwise at its no-loss setting and Medusa-1 yes; blockwise's faster settings, which trade a little quality, and Medusa's typical acceptance no | Stern et al. 2018; Cai et al. 2024 | | Heads the model carries: multi-token prediction (MTP) | One or more small layers trained with the model to predict the next tokens, then reused as a drafter | Yes, inside the model or as its own file | DeepSeek-V3; Qwen3.8-Flash-Next's "4B MTP" | Yes, by the acceptance rule | Gloeckle et al. 2024; DeepSeek-V3 report; the Qwen card | | Feature-reading drafters | A small network that reads the big model's internal state, not just its tokens | A second, model-specific file | EAGLE, EAGLE-3 | Yes | Li et al. 2024, 2025 | | Block drafters | A small model that writes a whole block of guesses in one pass | A second file | DFlash; DSpark (a DFlash-style backbone plus a light sequential head and a confidence gate) | Yes | Chen, Liang and Liu 2026; Cheng et al. 2026 | | No model at all | The text so far: an n-gram that recurs, or a copy of the prompt | Nothing to ship | llama.cpp's `ngram-*` modes; the Strata engine's prompt lookup | Yes | llama.cpp docs; the Strata engine docs | Three of these rows deserve a sentence each. **Multi-token prediction** began as a training trick. Meta's April 2024 paper trained models to predict "the following n tokens using n independent output heads, operating on top of a shared model trunk", for better models, and noted that the heads also make "4-token prediction" models "up to 3 times faster at inference". DeepSeek-V3's report (December 2024) uses it the same way round: "during inference, we can directly discard the MTP modules", or "repurpose these MTP modules for speculative decoding", with "the acceptance rate of the second token prediction" at "between 85% and 90%", "delivering 1.8 times TPS". Qwen3.8-Flash-Next's card (read 2026-10-29) lists "125B with 6B activated, plus 51B n-gram embedding and 4B MTP", "1 layer, trained with multi-steps". Where that head lives on disk, and why the file we benched had none, is one section of [Inside a GGUF file](https://research.strata2signal.com/inside-a-gguf-file/#the-draft-head-that-is-not-there); llama.cpp's support for it merged on 2026-10-01. **DSpark** is DeepSeek's drafter for its V4 family (2026-07-06). It couples "a parallel backbone with a lightweight sequential module" to "mitigate suffix decay", the way a block drafter's later guesses go wrong, and chooses how many guesses to verify per request from "estimated prefix survival probabilities" and the engine's load. Against EAGLE-3 it lengthens the accepted run on Qwen3-4B, 8B and 14B "by 30.9%, 26.7%, and 30.0%", against DFlash "by 16.3%, 18.4%, and 18.3%", measured "with the sampling temperature set to 1.0". In "the DeepSeek-V4 serving system under live user traffic" it replaced a one-token drafter ("MTP-1") and "accelerates per-user generation speeds by 60 to 85 percent at matched throughput levels". DeepSeek-V4.1-Flash's card names it in one line, and Unsloth's GGUF build of the model's 1017 update ships it as its own file: 7.98 GB at Q8_0, 8.51 GB at BF16 (the listing, read 2026-10-29). A speed figure for that model has to say whether that file was loaded. **No model at all.** llama.cpp's server offers five n-gram modes that need no second model (three of them, its documentation says, "rely on patterns that have already appeared in the generated text"). The Strata engine (github.com/Niko1221/Strata; despite the similar name, an unrelated project, with which strata2signal has no affiliation) runs the model's MTP layer by default and, by its documentation at `main` on 2026-10-29 (installers at 0.2.7), "also runs without it: drafts then come from the suffix and prompt-lookup drafter only, every token is still verified". On that engine "MTP off" is not "no drafting", and a figure labelled "MTP off" needs the second half of the label. ## Acceptance: three numbers with one name {#acceptance-three-numbers-with-one-name} When a drafter is on, the engine's log counts what it kept, and the word for the count, **acceptance**, names at least three quantities. Our August run shows all three. On 2026-08-18 we read the runtime's verbose log while `qwen3.8:27b` (dense, 27.3 billion parameters, Q4_K_M, ollama 0.32.13; then the model that screened questions for [RuleSage](https://rulesage.strata2signal.com/), our board-game rules helper) wrote a fixed 128-token reply at temperature zero, its draft head guessing four tokens a round. [The receipt](https://research.strata2signal.com/reading-is-fast/data/drafthead-acceptance-receipt.md) sits beside the August page; the two rows marked ours are our arithmetic on it, re-run for this page. | What "acceptance" can mean | From our August run | What it counts | |---|---|---| | Guesses accepted ÷ guesses proposed | 49.1 % (84 of 171) | Every guess, in every slot, as the engine's log counts them | | Tokens a round | 2.95 | One (the model's own token) plus accepted guesses ÷ rounds: 1 + 84 ÷ 43 | | By slot | 0.860 · 0.558 · 0.372 · 0.163 | The share of rounds in which the first, second, third and fourth guess survived; one plus their sum is 2.95 again | | Each slot given the one before (ours) | 0.860 · 0.649 · 0.667 · 0.438 | How often a guess survived when the guess before it had | | The paper's per-guess rate, α (ours, implied) | About 0.73 | The single rate that would give 2.95 tokens a round at depth four if every guess were an independent coin toss | The textbook formula from the 2022 paper assumes each guess survives with the same probability α, independently, so the expected tokens a round is (1 − α^(γ+1)) ÷ (1 − α) for γ guesses. Our slots are not independent: the fourth guess survived its predecessor far less often (0.438) than the second did (0.649), so the implied 0.73 is a summary, not a measurement, and the log's 49.1 % is lower again because it counts every proposal, those made after a miss included. Two more things move the number without changing the drafter. A **confidence gate** stops the drafter adding a guess it is unsure of: llama.cpp's `--spec-draft-p-min`, for a DSpark draft, "truncates each drafted block at the first position whose predicted acceptance" falls below a threshold; the Strata engine's documentation glosses `--spec-min-p` as "How sure the draft layer must be to add another guess", shipped as `--spec 4 --spec-min-p 0.5`. A gate raises the share accepted by proposing less, so rates under different gates do not compare. And some authors count the model's bonus token inside "accepted length"; DSpark's paper says its figures "include the target-generated bonus token". The thing to look for is the engine's own line. llama.cpp's reads `draft acceptance rate = 0.57576 ( 171 accepted / 297 generated)`, and that line says which of the three it is. ## Tokens a round is a ceiling, not the speed-up {#tokens-a-round-is-a-ceiling-not-the-speed-up} If every round yields 2.95 tokens, the model is not 2.95 times faster. The drafter costs time, the verify round is a bigger trip than a one-token step, and the engine keeps books between them. The 2022 paper's walltime formula divides the tokens-a-round gain by (γc + 1), c being the drafter's cost as a share of the big model's; in their runs c "was always less than 0.05", and the heads and lookup drafters above exist to push it lower. Our August bracket shows what is left after the costs. On 2026-08-21, between 06:35:47Z and 07:09:30Z, in a declared maintenance window on our workstation's card, the same model answered 270 calls at three prompt lengths (1,041, 8,219 and 31,015 tokens) with the draft head on, then off, then on again, a fresh nonce on every call so none was a cache hit. [The results file](https://research.strata2signal.com/reading-is-fast/data/MATRIX-RESULTS.md) is public; its medians: | Prompt | Draft head on (the two on-blocks pooled) | Draft head off | Ratio | Cost of switching it off | |---|---|---|---|---| | 1,041 tokens | 109.08 tok/s | 76.11 tok/s | 1.43× | 30.2 % | | 8,219 tokens | 105.14 tok/s | 74.46 tok/s | 1.41× | 29.2 % | | 31,015 tokens | 92.44 tok/s | 68.94 tok/s | 1.34× | 25.4 % | The bracket logged no acceptance of its own, so the ceiling it was held against is the receipt's 2.95, counted on a different prompt three days earlier: drafting bought 1.34 to 1.43 times the speed, "45–49% of the theoretical maximum" in the file's words. The file carries a label this page carries too: **published as a disagreement**. The two draft-on blocks, eleven minutes apart with the off block between them, differed by 5.72 % at the 8k prompt and 3.67 % at 32k (each gap as a share of the two blocks' mean), outside the ±3 % tolerance set for the run, so the ratios are the run's reading, not a ratified figure. The file also defuses a coincidence: the 49.1 % acceptance above and the "45–49%" here are different quantities that happen to land near each other. One gloss is ours, marked: 2.95 ÷ 1.43 is about 2.06, so if the bracket's rounds carried the receipt's 2.95 tokens, a drafted round cost about twice a plain step, everything included. The October receipt makes the point from the other side. On an NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB) at a 300 W cap, the Strata engine 0.1.39 ran Qwen3.8-Flash-Next at 78.58 verify rounds a second with 2.612 tokens a round: 205.263 tokens a second, drafting from the model's own draft head. llama.cpp at commit `3cf03257`, with no draft head because its file carried none, took 102.9 single-token steps a second: 102.889 tokens a second. Per step the drafted engine was slower; per second it wrote twice as much. The tokens a round are the engine's own counts, derived as generated ÷ (generated − drafts accepted); the three runs read 2.753, 2.612 and 2.813, and the same files gave 2.2 to 2.8 on an NVIDIA GeForce RTX 5090 Laptop GPU (24 GB) and 2.586 on an EVGA GeForce RTX 3090 Ti FTW3 Ultra (24 GB). Two engines, one engine a night on the big card, and the drafter's share of the lead not split by any run: [Qwen3.8-Flash-Next on three cards](https://research.strata2signal.com/flash-next-on-three-cards/) carries the figures with every qualifier, and [Why the first word comes sooner](https://research.strata2signal.com/flash-next-why-the-first-word-comes-sooner/) is the full bench. ## Why every speed figure needs a drafter label {#why-every-speed-figure-needs-a-drafter-label} Four reasons, each with a receipt. **It moves the figure by a quarter or more.** On our August model the draft head was worth 25 to 30 per cent of the writing speed at every prompt length; a figure for that model with no label sits somewhere in a 30 per cent band. **It ties the speed to the text.** With the draft head off, thirty consecutive calls at the 8k prompt spanned 74.27 to 74.70 tokens a second; with it on, 95.82 to 122.57. The August file puts the scatter at 11 to 38 times wider with drafting on, because how many guesses survive depends on what the model happens to be writing; the Strata engine's documentation says of its own output speed that "a different answer to the same prompt moves it by several percent". A single drafted reading carries a variance a tidy median hides. **It confounds a comparison.** [Everyone on the payroll, three at the table](https://research.strata2signal.com/three-at-the-table/) measured several models to show what a mixture of experts buys. One carried a small draft head of its own, so its speed "may be sparsity, sparsity plus speculation", which the page could not separate; the one mixture that, when that page read it in August 2026, had "no draft head anywhere in its file" became "the cleanest sparsity receipt we own". That reading belongs to the build that page read: [Speeds from RAM alone, compared](https://research.strata2signal.com/ram-speeds-compared/), published 2026-11-10, charts a newer build that carries the same name, gemma4:26b, and says every bar from that build includes its draft model. A missing label does not make a figure wrong; it makes it unusable beside another. **"Off" has to say off what.** On the Strata engine at `main` on 2026-10-29, running without the MTP layer still drafts by suffix and prompt lookup, so "MTP off" there means "a weaker drafter on". And a sampler setting can move acceptance without touching the drafter: the Ollama release note that [The free speed wasn't free](https://research.strata2signal.com/the-free-speed-wasnt-free/) traced calls its repeat-penalty change "speeding up speculative decoding", while the same change sped up our own machine, which had "no draft model loaded", by another route. So from this page on, every writing-speed figure we publish carries one line, and we would ask the same of anyone's: > **drafter:** none · MTP head · draft model (name) · EAGLE · DFlash · DSpark · n-gram or prompt lookup **· depth and gate:** guesses a round, and any confidence threshold **· acceptance, from the engine's own log:** accepted of proposed, or tokens a round, saying which **· temperature · streams at once** (one, or how many). The [hardware roster](https://research.strata2signal.com/hardware/rtx-5090-laptop/) prints the drafter's state beside a row only where that row's own page states it, in that page's words, and leaves every other row unlabelled rather than guessed: no label means the roster read no state for that row, not that its drafter was off. On [the speed tool](https://speed.strata2signal.com/), each run you can pick carries "(drafter on)" or "(drafter off)" in its label where the roster rows behind it carry a state. ## When it helps, and when it does not {#when-it-helps-and-when-it-does-not} Drafting is not free speed. It trades arithmetic the card had spare for tokens, and when the card has no arithmetic to spare, or the guesses do not survive, it can cost. Every drafted reading of ours on this page sits in one corner of this space, a single stream at temperature zero, so every row below is cited to others, and none is a rule. | What changes | Which way it cuts | Who says so | |---|---|---| | How many people share the card | vLLM's documentation (page dated 2026-11-07) frames drafting as a way "to reduce inter-token latency under medium-to-low QPS (queries per second), memory-bound workloads", and its 2024 blog reported "1.4x slowdown Llama3-70B on ShareGPT with 4xH100" at high request rates, because the extra work slows a system "already compute-bound". TurboSpec (2024) calls the benefit "often fragile". The other side: MagicDec (2024) shows drafting "can achieve speedup even for a high throughput inference regime for moderate to long sequences", up to 2.51× at batch sizes of 32 to 256; EAGLE-3 reports "1.38x throughput improvement at a batch size of 64"; and under live traffic DSpark's scheduler "automatically restricts the per-request verification length" as the concurrent load rises. | vLLM docs and blog; Liu et al. 2024; Sadhukhan et al. 2024; Li et al. 2025; Cheng et al. 2026 | | Temperature | In the 2022 paper's Table 2, the same T5-small drafter at the same depth gave 3.4× at temperature 0 and 2.6× at temperature 1 (α 0.75 against 0.62), though Table 3 shows the drop is not universal. The Strata engine's documentation says: "Sampled drafting is opt-in and not faster in a way that holds up." | Leviathan et al. 2022; the Strata engine docs | | The text being written | Repeated or structured text suits lookup drafting: llama.cpp documents its n-gram modes for "the rewriting of source code by a LLM", and vLLM's blog reported prompt-lookup "speedups of up to 2.8x" on summarisation. Our August scatter is the same fact from the other side: free prose makes acceptance wander. | llama.cpp docs; vLLM blog; our August file | | Sampler settings | A repeat penalty changes which tokens the model will take, so it changes which guesses survive; Ollama's note tied its default change to "speeding up speculative decoding". | Ollama's release note, via our page | | Prompt length and memory | Our August ratio fell gently with the prompt, 1.43× to 1.34×, as the working memory grew. And a drafter takes memory of its own: DSpark's file is 7.98 GB at Q8_0; the Strata engine's MTP layer takes "~0.7-1 GiB of VRAM", which goes to the expert cache when it is off, while drafting falls back to suffix and prompt lookup. | Our August file; the Unsloth listing; the Strata engine docs | The honest summary is "it depends, and here is what it depends on". The surest gains in the sources above come with one person at a time, a low temperature, text the drafter can predict and memory to spare for it, and some of them report gains under load as well; every receipt of ours on this page was taken in that first setting. ## Reading someone else's speed claim {#reading-someone-elses-speed-claim} Six questions, in the order they usually matter. 1. **Which drafter, if any?** None, the model's own head, a separate draft model (which one), a feature-reading or block drafter, or n-gram lookup. If the claim does not say, the figure cannot be compared. 2. **What did the engine's log say it accepted**, and in which of the three senses: guesses of proposals, tokens a round, or by slot? 3. **What temperature?** In the 2022 paper every drafter it timed was faster at temperature zero than at 1, though acceptance did not fall for every pair of models. 4. **One stream, or many?** A figure from one request on an idle card says nothing about a card serving twenty. 5. **How long was the prompt?** Working memory grows with it, and so does every trip. 6. **Which card, which engine, which version, and when?** The same file on the same card wrote 102.889 and 205.263 tokens a second one night apart, on two engines, one with the draft head and one without. ## What this cannot say {#what-this-cannot-say} - **Anything about DSpark from our own measurements.** An attempt of ours to run a model with it measured nothing, for a reason that had nothing to do with drafting. Every DSpark figure on this page is DeepSeek's own, from its paper, or a file size from Unsloth's listing; none is a reading of ours. - **What Qwen3.8-Flash-Next writes on the Strata engine with its draft head off.** At the build we ran, the server would not start without it, and the bench page records that arm as the arm that would not start. The engine's documentation at `main` on 2026-11-11 (installers at 0.2.13) says running without the head still drafts by prompt lookup, so a run of that arm would be labelled "MTP off, lookup drafting on", not "no drafting". - **What drafting does at temperature 1.0, or with more than one stream, from our own measurements.** Every drafted reading of ours on this page is a single stream at temperature zero, which is why the two biggest "it depends" rows are cited to others. Another page of ours, [Two Hours at 12 tok/s — On Battery](https://research.strata2signal.com/two-hours-on-battery/), does read gemma4:26b writing two streams at once, on a processor with no graphics card in the path, but that run's files record no drafter state and no drafter-off run sits beside it, so it says nothing about what drafting does with two streams. - **How the two August blocks came to disagree.** The file's own reading is sampling variance inside a high-dispersion arm; it is published as a disagreement, and this page repeats it as one. - **Which card ran the August receipts.** That page does not name it, and neither does this one. ## Who ran this, and thanks {#who-ran-this-and-thanks} Thanks to Yaniv Leviathan, Matan Kalman and Yossi Matias, and to Charlie Chen and colleagues at DeepMind, for the two papers that made the method lawful; to the authors of Blockwise Parallel Decoding, Medusa, EAGLE, DFlash, DSpark, MagicDec and TurboSpec, and of Meta's and DeepSeek's multi-token-prediction work, whose figures are quoted here with their dates; to the vLLM team for writing down where drafting hurts; to Qwen and DeepSeek for their models and cards, and to Unsloth for publishing the drafter's file sizes; to the llama.cpp project, the ollama project and the Strata engine's author for engines that print their own acceptance counts, which is the whole reason this page can be written. The August receipts came from ollama's verbose log and a bracketed run in a declared maintenance window; the October receipts from the Strata engine's and llama.cpp's own serve lines, recorded by the full bench. A small human team asked for this page and will approve it; a fleet of AI agents read the sources, re-ran the arithmetic from the public files and drafted it under that team's rulings. ## Sources {#sources} Read 2026-10-29 (UTC) unless dated otherwise; every quote taken from a source in this list was checked against it again on 2026-11-11 (UTC). Papers are quoted in short phrases with attribution; the DSpark paper is under CC BY 4.0. - **Our own pages and files:** [Reading is fast, writing is slow](https://research.strata2signal.com/reading-is-fast/) (2026-08-19, updated 2026-08-21) with its data files [drafthead-acceptance-receipt.md](https://research.strata2signal.com/reading-is-fast/data/drafthead-acceptance-receipt.md), [MATRIX-RESULTS.md](https://research.strata2signal.com/reading-is-fast/data/MATRIX-RESULTS.md) and [matrix-runs.json](https://research.strata2signal.com/reading-is-fast/data/matrix-runs.json), from which this page's August figures were re-computed; [Qwen3.8-Flash-Next on three cards](https://research.strata2signal.com/flash-next-on-three-cards/) and [Why the first word comes sooner](https://research.strata2signal.com/flash-next-why-the-first-word-comes-sooner/) (2026-10-12); [Everyone on the payroll, three at the table](https://research.strata2signal.com/three-at-the-table/); [Inside a GGUF file](https://research.strata2signal.com/inside-a-gguf-file/); [The free speed wasn't free](https://research.strata2signal.com/the-free-speed-wasnt-free/). - **Leviathan, Kalman and Matias**, "Fast Inference from Transformers via Speculative Decoding", arXiv 2211.17192 (v1 2022-11-30, v2 2023-05-18): https://arxiv.org/abs/2211.17192 (Tables 2 and 3, Definition 3.7 and Theorem 3.8). - **Chen et al.**, "Accelerating Large Language Model Decoding with Speculative Sampling", arXiv 2302.01318 (2023-02-02): https://arxiv.org/abs/2302.01318 - **Stern, Shazeer and Uszkoreit**, "Blockwise Parallel Decoding for Deep Autoregressive Models", arXiv 1811.03115 (2018-11-07): https://arxiv.org/abs/1811.03115 - **Gloeckle et al.**, "Better & Faster Large Language Models via Multi-token Prediction", arXiv 2404.19737 (2024-04-30): https://arxiv.org/abs/2404.19737 - **DeepSeek-AI**, "DeepSeek-V3 Technical Report", arXiv 2412.19437 (v1 2024-12-27, v2 2025-02-18), §2.2 and §5.4.3: https://arxiv.org/abs/2412.19437 - **Cai et al.**, "Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads", arXiv 2401.10774 (v1 2024-01-19, v3 2024-06-14): https://arxiv.org/abs/2401.10774 - **Li et al.**, "EAGLE", arXiv 2401.15077 (v1 2024-01-26, v3 2025-03-04): https://arxiv.org/abs/2401.15077 · "EAGLE-3", arXiv 2503.01840 (v1 2025-03-03, v3 2025-04-23): https://arxiv.org/abs/2503.01840 - **Chen, Liang and Liu**, "DFlash: Block Diffusion for Flash Speculative Decoding", arXiv 2602.06036 (v1 2026-02-05, v2 2026-05-28): https://arxiv.org/abs/2602.06036 - **Cheng et al.** (DeepSeek), "DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation", arXiv 2607.05147 (2026-07-06), CC BY 4.0: https://arxiv.org/abs/2607.05147 (§4.1 for the evaluation temperature and the bonus-token note; §5.4 and Figure 8 for load). - **Sadhukhan et al.**, "MagicDec", arXiv 2408.11049 (v1 2024-08-20, v5 2025-04-02): https://arxiv.org/abs/2408.11049 - **Liu et al.**, "TurboSpec", arXiv 2406.14066 (v1 2024-06-20, v3 2025-07-27): https://arxiv.org/abs/2406.14066 - **DeepSeek-V4.1-Flash**, model card (MIT): https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash · **Unsloth**, DeepSeek-V4.1-Flash-1017-GGUF, the repository listing, with the DSpark Q8_0 file at its top level and the BF16 file in `dspark/`: https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF/tree/main - **Qwen3.8-Flash-Next**, model card (weights under the Qwen Community License 1.0): https://huggingface.co/Qwen/Qwen3.8-Flash-Next - **llama.cpp**, `docs/speculative.md` at commit `d0ff6317` (2026-10-29), MIT: https://github.com/ggml-org/llama.cpp/blob/d0ff6317cc28fe3aa0542c21294f7e23a3632241/docs/speculative.md. Our October rows ran commit `3cf03257`. - **The Strata engine**, `docs/DETAILS.md` at `main`, commit `46c152a1` (2026-10-28; installers at 0.2.7), MIT: https://github.com/Niko1221/Strata/blob/46c152a1310f77cd2f0ce2c7cd213962df14fe83/docs/DETAILS.md. Read again at `main` on 2026-11-11 (installers at 0.2.13): every phrase quoted here still stands. An unrelated project; strata2signal has no affiliation with it. Our rows ran 0.1.25 and 0.1.39. - **vLLM**, "Speculative Decoding" documentation (page dated 2026-11-07, read 2026-11-11): https://docs.vllm.ai/en/latest/features/speculative_decoding/ · the vLLM team's blog, "How Speculative Decoding Boosts vLLM Performance by up to 2.8x" (2024-10-17): https://vllm.ai/blog/2024-10-17-spec-decode - **ollama** (MIT): https://github.com/ollama/ollama. The August runtime was 0.32.13. *Every figure on this page was printed first on a page of ours or in a maker's own paper or card, or is our own arithmetic on those, marked as ours; nothing was measured for it. A guide, not a bench. Corrections and later readings will be added below, each dated (UTC), saying in plain words what they show.*