# RECON A — the upstream cause of the ollama 0.32.9 → 0.32.13 speed jump > SANITIZED AT PUBLICATION (2026-08-21): estate-internal box names, home > paths, and private overlay addresses in this working document are replaced > with public aliases ([workspace], the-gpu-box, the-dev-laptop, the-vps, > [private-peer], [agent-memory]); two receipt filenames beginning with a box > name are renamed to speed-forensic-2026-08-16.md / speed-probe-2026-08-16.json. > Quoted clock strings are re-expressed in UTC (same instants) and no longer > byte-match their sources; every load-bearing timestamp is stated in UTC. > Measurements, counts, shas, and quoted third-party text are untouched. Recon lane, 2026-08-21. Scope: find WHY, from upstream sources, with a receipt under every claim. No builds, benches or heavy local processes were run (RAM-pressure constraint) — this report is file reads, `scribe`, and web fetches only. **Tool caveat, stated once, up front.** Every web quotation below was extracted by `WebFetch`, which converts a page to markdown and answers a prompt over it with a small model. Quotes marked **[verbatim, cross-corroborated]** were obtained twice by independent means (my fetch today + an estate document that quoted the same page on 2026-08-16) and match character-for-character. Quotes marked **[verbatim, single extraction]** are one fetch only — spot-check them by eye before they go on a public page. Paraphrases produced by the fetch tool are labelled **[tool paraphrase]** and must never be printed as quotations. > **ERRATUM (added 2026-08-21, post-panel, before publication):** §6's table > gives gemma4:26b as "25.2B total" — the first-party receipt read the same > day says otherwise: /api/show `general.parameter_count = 25,805,936,462` > (25.8B), agreeing with the already-published stopwatch page ("MoE 25.8B"). > Also §8 item 4 ("re-read /api/show for vocab_size") was attempted: the > model_info block on 0.32.13 exposes NO vocab field; the first-party source > is the llama-server load print `n_vocab = 262144` (VOCAB-RECEIPT file). > §9 item 2's ~30-second discrepancy test RAN (TOLL-PROBE-2026-08-21.md): > both candidate causes refuted; the toll is ~2.5 ms/token at both prompt > lengths, and the cross-version A/B's smaller implied toll decomposes into > the default flip net of a ~9% slower penalty path on the newer runtime. --- ## 0. Verdict in one paragraph The gain is **one upstream commit**, and it is not a compiler flag or a kernel: it is a **two-character change to a default value**. Ollama commit `6a261db7d87b13d76c5197cec636a0a3951afb36` — *"api: stop applying repeat_penalty 1.1 to models that don't set one"*, authored by **Jesse Gross (`jessegross`)** on **2026-08-11T18:49:22Z**, `+2 −2` across two files — changed the server-side default `RepeatPenalty` from `1.1` to `1.0` in `api/types.go`. It shipped in **v0.32.10** (published 12 Aug 22:36). `gemma4:26b` publishes no `repeat_penalty` of its own, so it had been silently receiving 1.1 on every request. With the penalty at 1.0 the `penalties` sampler stage becomes a no-op (and, per the-gpu-box's own logs, drops out of the printed chain), and the runtime stops doing a **once-per-token pass over the entire candidate/vocabulary array on the CPU** between the model's logits and the chosen token. The estate has already measured this at the box end, twice, with the runtime held constant — so the mechanism is measured, not merely argued. The legend ("sampling silently off") is **substantially correct**, with two corrections detailed in §7. --- ## 1. The commit — receipts ### 1a. It exists, and it is in the window ``` $ WebFetch https://api.github.com/repos/ollama/ollama/compare/v0.32.9...v0.32.13 ``` Output (**total_commits: 16**), in order: | # | sha | author | first line | |---|---|---|---| | 1 | `96fb6d2fa` | dhiltgen | nemotron_h: support the Nemotron 3.5 prompt layout (#17672) | | 2 | `948f69330` | hoyyeva | docs: fix broken links (#17676) | | 3 | **`6a261db7d`** | **jessegross** | **api: stop applying repeat_penalty 1.1 to models that don't set one** | | 4 | `641df5e5a` | dhiltgen | mlx: enable CUDA backend in CUDA builds (#17688) | | 5 | `4138e853d` | vigneshakaviki | server/images: prevent skipVerify map collision with duplicate digests (#15504) | | 6 | `b6b1b258c` | ParthSareen | openai: support web search in Responses API (#17686) | | 7 | `950dd9ac6` | dhiltgen | MLX update (#17704) | | 8 | `e922bc712` | dhiltgen | llama.cpp bump (#17702) | | 9 | `2b4a99376` | jessegross | nn: speed up prefill on double-scale nvfp4 models | | 10 | `88313499e` | dhiltgen | mlx: avoid pulling MLX models when MLX is missing (#17710) | | 11 | `9a56a0e84` | ParthSareen | agent: allow multiple edits per edit tool call (#17711) | | 12 | `01d04d50f` | dhiltgen | launch: add Muse Code integration (#17594) | | 13 | `7ce88bd68` | dhiltgen | model/renderers: match Muse Glimmer reasoning template (#17732) | | 14 | `39df91c98` | ParthSareen | launch: add DeepSeek Harness integration (#17733) | | 15 | `55127975a` | dhiltgen | qwen3.8: add renderer and MLX import support (#17745) | | 16 | `0f25c31bd` | dhiltgen | qwen3.8: support developer instructions (#17749) | Note commit 3 carries **no `(#NNNN)` suffix** — unlike almost every other commit in the window it has no pull-request number in its subject, and the commit page reports **0 commit comments**. It went in without a PR thread. That is the literal shape of the legend: the change that bought the estate ~32–54% arrived with no discussion attached. ### 1b. Identity, size, and timing ``` $ WebFetch https://api.github.com/repos/ollama/ollama/commits/6a261db7d ``` - **Commit SHA:** `6a261db7d87b13d76c5197cec636a0a3951afb36` - **Author:** `jessegross` — patch header carries his name and public author address (verbatim in COMMIT-6a261db7-RAW-API.json) - **Author date:** `2026-08-11T18:49:22Z` · **Committer date:** `2026-08-12T04:47:51Z` - **Stats:** 2 additions, 2 deletions, 2 files - **Files:** `api/types.go` (+1 −1), `docs/modelfile.mdx` (+1 −1) ### 1c. The diff — [verbatim, single extraction] ``` $ WebFetch https://github.com/ollama/ollama/commit/6a261db7d.patch ``` ```diff --- a/api/types.go - RepeatPenalty: 1.1, + RepeatPenalty: 1.0, ``` ```diff --- a/docs/modelfile.mdx -| repeat_penalty | ... (Default: 1.1) | float | repeat_penalty 1.1 | +| repeat_penalty | ... (Default: 1.0, disabled) | float | repeat_penalty 1.1 | ``` (The doc row's prose is unchanged apart from the parenthetical; the fetch returned the full table row, elided here for width.) ### 1d. The commit message — [verbatim, single extraction] — NEW, not previously in the estate record This text has not been quoted in any estate document before today (the 2026-08-16 forensic quoted only the release notes). It is the single most article-worthy artifact of this recon, because the author explains both the reasoning and the cost — and names gemma4 by name: > Request options are the model's published parameters and the request's own options > layered over the server defaults, so the default repeat_penalty of 1.1 reaches every > model whose parameters leave it unset. No maker of the library's current models > recommends 1.1: their generation configs either omit the penalty, meaning 1.0, or pin > 1.05. llama.cpp dropped the same 1.1 default in 2024; vLLM, SGLang, and transformers > apply no penalty. An always-on penalty also distorts output that legitimately repeats > tokens, such as code, JSON, and long reasoning traces. > > The penalty is especially costly for speculative decoding, where drafts are proposed > without it: the penalized target rejects drafted tokens and the depth controller backs > off. On muse-glimmer 30B (DFlash on M5 Max, HumanEval) the 1.1 default costs 13-16% of > end-to-end throughput at greedy and temperature 1 alike, and drops prose acceptance at > temperature 0.8 from 0.44 to 0.30. On qwen3.6-35B it cuts the mean accepted draft > length from 4.3 to 3.5 tokens and makes the controller stop speculating on prose. > > Defaulting to 1.0 disables the penalty unless a model's parameters or the request set > one. Across the library: > > - qwen3, qwen3.6, and qwen3-coder pin their own values (1.0, 1.0, and Qwen's > recommended 1.05) and are unchanged. > - Everything else local now matches its maker's no-penalty recommendation, including > gemma2 through gemma4, muse-glimmer, both laguna 2.1 models, qwen3.5 (previously 1.1 > stacked on its presence_penalty of 1.5), gpt-oss, deepseek-r1 and v3.1, the nemotron > family, granite4, the mistral and llama3/llama4 families, phi4, glm4, llava, and > devstral. > - qwen2.5 recommends 1.05 but ships no parameters, so it moves from 1.1 to 1.0 and > still needs a parameters layer to conform. > - Cloud models (kimi-k3, deepseek-v4-flash) never receive these defaults. > > Small older models may repeat themselves more without the penalty masking it; the > remedy is a per-model parameter, not a penalty applied to every model. Three things the article can use directly: 1. **The author's own stated cost is 13–16%**, measured on *muse-glimmer 30B, DFlash on M5 Max* — a different model, a different stack, and a *speculative-decoding* mechanism. The estate measured **+31.7%** (this morning's A/B) and **+53.8%** (the 08-16 same-runtime toggle) on a CUDA box with **no draft model in play**. The estate's win is bigger than the vendor's own headline and arrives by a *different route* (§5, §6). 2. **"gemma2 through gemma4"** — the model the estate runs is named in the commit body. 3. The rationale is a *correctness* argument first ("distorts output that legitimately repeats tokens, such as code, JSON, and long reasoning traces"), speed second. The speed was a side effect of fixing a default that was wrong on the merits. --- ## 2. Release notes, v0.32.10 → v0.32.13 — every performance-related line Tag dates from `https://github.com/ollama/ollama/releases` (list page): v0.32.9 — 11 Aug 13:23 · **v0.32.10 — 12 Aug 22:36** · v0.32.11 — 14 Aug 01:22 · v0.32.12 — 14 Aug 16:37 · **v0.32.13 — 14 Aug 19:16** · v0.32.14 — 15 Aug 19:41 · v0.32.15 — 19 Aug 17:25. ### v0.32.10 — [verbatim, cross-corroborated] My fetch of `https://github.com/ollama/ollama/releases/tag/v0.32.10` returned three bullets, character-identical to the ones quoted in the estate's own 2026-08-16 forensic (`[workspace]/rs-thirteen-2026-08/recon/speed-forensic-2026-08-16.md:160-166`). Two independent extractions, five days apart, same strings: > "Models that don't set a `repeat_penalty` now default to 1.0 (off) instead of 1.1, > matching other engines and speeding up speculative decoding; set a per-model parameter > if an older model repeats itself." > "Faster prefill on NVFP4 MLX models with a global scale, about 7–8% on Qwen3.6 and > Muse Glimmer." > "Fixed blob verification being skipped when an OCI manifest's config and layer share a > digest." **Performance-related lines: two.** The first is the cause. The second is an *MLX-path, prefill-only* win (the estate's delta is entirely in `eval_duration`; prefill was 69–76 ms in both arms of the 08-16 probe — `speed-forensic-2026-08-16.md:193`), so it cannot be the cause of a decode-rate change on a CUDA box. ### v0.32.11 — no performance items [tool paraphrase] `ollama launch dsh` (DeepSeek Harness support), `ollama launch muse` (Muse Code, Meta's agentic coding CLI), web search in the OpenAI-compatible Responses API, Muse Glimmer template updates. Nothing performance-related. ### v0.32.12 — no general performance items [verbatim, single extraction] The page is dominated by the Qwen 3.8 27B launch: > "Qwen3.8 delivers substantial gains across coding, professional work, research, and > long-horizon agentic tasks" and (quoted by the 08-16 forensic at `speed-forensic-2026-08-16.md:176-177`): > "For Apple Silicon devices, Ollama has in particular optimized for maximum performance > and output quality suitable for repeated tasks and coding agents." Apple-Silicon-scoped; not applicable to a CUDA box. **UNVERIFIED:** the "What's Changed" PR list on this tag page failed to load in my fetch (GitHub returned error placeholders), so I cannot rule out an unlisted performance item on this tag by reading the page alone — but the 16-commit compare in §1a covers the whole window regardless, and contains everything that shipped between the two tags. ### v0.32.13 — one line [verbatim, single extraction] and matching the forensic at `speed-forensic-2026-08-16.md:181-182`: > "## What's Changed > * qwen3.8: support developer instructions" ### Context beyond the mission window (worth a line in the article's "what's next") - **v0.32.14** [tool paraphrase]: "llm: transcode WebP images for llama-server", "renderers/qwen: tolerate non-leading system messages". No performance items. - **v0.32.15** [verbatim, single extraction]: *"Caches resolved model metadata between requests, cutting time-to-first-token by roughly half (TTFT dropped from ~995 ms to ~524 ms in benchmarks)"*. the-gpu-box is pinned at 0.32.13 — **there is a second, unclaimed dividend sitting two releases ahead**, and it is a *latency* win rather than a decode win, which is exactly what interactive rulesage/amble traffic feels. Sized proposal in §9. --- ## 3. The candidate table — what else was in the window, and why it loses | candidate | what it is | verdict | receipt | |---|---|---|---| | **`6a261db7d` repeat_penalty default 1.1 → 1.0** (v0.32.10) | server-side default flip, `api/types.go` | **THE CAUSE** | §1; §5; estate replay `speed-forensic-2026-08-16.md:36-45`; this morning's A/B `ab-runs.json` | | `e922bc712` llama.cpp bump (#17702) | `LLAMA_CPP_VERSION` **b10353 → b10380**, 27 upstream commits | **bounded ≤2.5%** of the 08-16 jump | See below | | `2b4a99376` nn: speed up prefill on double-scale nvfp4 models | prefill, NVFP4, MLX path | **excluded** | Release note says prefill + NVFP4 MLX; estate delta is decode-only | | `641df5e5a` mlx: enable CUDA backend in CUDA builds (#17688) | MLX engine gains a CUDA backend | **not on this path** | GGUF models do not use the MLX runner — §4 | | `950dd9ac6` MLX update (#17704) | MLX dependency bump | **not on this path** | §4 | | all other 11 commits | renderers, launch integrations, docs, agent edits, OCI digest fix | **not performance** | §1a titles | **Why the llama.cpp bump is bounded, not dismissed.** Two independent bounds: 1. The estate's 08-16 probe re-added `repeat_penalty: 1.1` **on the new binary** and recovered **97.5%** of the historical 0.32.9 rate (136.13 vs 139.55 tok/s) — so at most 2.5% of that jump can live anywhere else, including in llama.cpp (`speed-forensic-2026-08-16.md:44`, `:195-197`). 2. I read the 27-commit llama.cpp window myself: ``` $ WebFetch https://api.github.com/repos/ggml-org/llama.cpp/compare/b10353...b10380 ``` total_commits **27**; the only ones matching CUDA/MoE/decode/perf/sampling keywords were: `f8def7fe` "ggml : require contiguous src for ROLL on CUDA and Metal"; `dd1ea524` gaugarg-nv "llama : support multi-output backend sampling (#25532)"; `0666ad2b` "ci : target ROCm 7.14 for build and release (#25775)"; `030ebb55` "Address review comment of PR 25532 (#26852)"; `6e62ba53` "mtmd: support pocket-tts (#26871)"; `cc078b45` "Dflash support for nemotron-3.5 (#26905)". **No MoE kernel work, no CUDA decode kernel work.** ⚠ This is a *keyword-filtered negative produced by a summarizer over an API response* — by the estate's own absence-claims law it is weaker than a grep, and a lens re-running it should re-derive rather than cite it. `LLAMA_CPP_VERSION` receipts: `$ WebFetch https://raw.githubusercontent.com/ollama/ollama/v0.32.9/LLAMA_CPP_VERSION` → `b10353` `$ WebFetch https://raw.githubusercontent.com/ollama/ollama/v0.32.13/LLAMA_CPP_VERSION` → `b10380` --- ## 4. Which engine actually runs `gemma4:26b` on a CUDA box (this decides the mechanism) This matters because half the window's commits are MLX-only, and because the vendor's stated mechanism (speculative decoding) is not the estate's mechanism. - **GGUF ⇒ llama-server (llama.cpp).** ollama PR **#16031**, dhiltgen, merged 29 May 2026, titled *"runner: Remove CGO engines, use llama-server exclusively for GGML models"* — [verbatim, single extraction] from the body: *"llama-server (built from upstream llama.cpp via FetchContent) is now the sole inference engine for GGUF-based models."* and *"Remove the vendored GGML and llama.cpp backend, CGO runner, Go model implementations"*. - **The Go runner is MLX-only at v0.32.13.** `runner/runner.go` at tag v0.32.13 is ~20 lines and dispatches on a single flag: ```go if len(args) > 0 { switch args[0] { case "--mlx-engine": return mlxrunner.Execute(args[1:]) } } return fmt.Errorf("unknown runner engine, expected --mlx-engine") ``` (`https://raw.githubusercontent.com/ollama/ollama/v0.32.13/runner/runner.go`) - The Go model implementations (including `x/models/gemma4/gemma4.go`, `x/models/gemma4/assistant.go`, `x/models/gemma4/gemma4_moe_test.go`) live under `x/`, the MLX tree, alongside `x/mlxrunner`. **Consequence:** the-gpu-box's `gemma4:26b` (Q4_K_M GGUF, CUDA) runs through **llama.cpp's llama-server**, which is why the-gpu-box's journal prints a llama.cpp-style sampler chain (`speed-forensic-2026-08-16.md:109-135`), and why the MLX/NVFP4 items in the release notes are irrelevant to it. --- ## 5. The mechanism, in plain english — with the upstream source under it **What the runtime does between the model and your screen.** Every generated token, the model produces one score (logit) for every entry in the vocabulary — for a Gemma-class model that is on the order of a quarter of a million numbers. The sampler chain then runs, stage by stage, to turn those scores into one chosen token. `penalties` is the first stage in the chain — it runs *before* `top-k` narrows the field, so it sees the **whole vocabulary**. **What the penalty stage costs.** Upstream source, `src/llama-sampler.cpp` at tag `b10380` — [verbatim, single extraction]: ```c static void llama_sampler_penalties_apply(struct llama_sampler * smpl, llama_token_data_array * cur_p) { auto * ctx = (llama_sampler_penalties *) smpl->ctx; if (ctx->is_disabled()) { return; } // Apply frequency and presence penalties to the cur_p for (size_t i = 0; i < cur_p->size; ++i) { const auto token_iter = ctx->token_count.find(cur_p->data[i].id); if (token_iter == ctx->token_count.end()) { continue; } ... } cur_p->sorted = false; } ``` Read it plainly: when the penalty is on, the CPU walks **the entire candidate array, every token**, doing a hash-map lookup per entry to ask "have we said this word in the last 64 tokens?" — a quarter-million lookups to adjust a handful of numbers — and then sets `cur_p->sorted = false`, which throws away any ordering the next stage would have liked to reuse. When the penalty is off, `is_disabled()` returns at the top and the whole pass evaporates. (the-gpu-box's logs show the stage does not merely short-circuit but leaves the printed chain entirely: `?penalties` vs `penalties`, `speed-forensic-2026-08-16.md:109-135`.) **A second, unconfirmed amplifier worth naming as a hypothesis.** llama.cpp can run sampling *on the GPU* ("backend sampling", PR **#17004**, danbev, merged 4 Jan 2026). That PR's description, [verbatim, single extraction], states the rule: > "If the sampler chain starts with an unsupported sampler (e.g., `penalties`), all > sampling runs on the CPU." and lists `penalties` among the CPU-only samplers (supported on the backend: temp, logit_bias, top_k, greedy, dist, min_p, top_p, temp_ext). PR **#25532** (gaugarg-nv, merged 10 Aug 2026 — inside our llama.cpp window) reports *"~8-12% improvement in tok/s"* for backend sampling on a current consumer flagship card (the hardware model is named in the public PR; generalized here per the hub's no-hardware rule). **If** ollama's llama-server invocation enables backend sampling, then a `penalties` stage at the head of the chain does not just cost its own loop — it forfeits GPU-side sampling for the whole chain and forces a per-token logits round-trip to the host. That would explain a per-token cost of milliseconds rather than microseconds. **UNVERIFIED:** I did not confirm that ollama passes a backend-sampling flag to llama-server (`llm/llama_server.go` is 84 KB and I did not read it end-to-end). Do not print this as the mechanism; print §5's first half, which is measured, and treat this as an open question (§9). **What the estate measured (the numbers that belong in the article).** | experiment | prompt | runtime(s) | penalty on | penalty off | Δ ms/token | speedup | |---|---|---|---|---|---|---| | 08-16 same-runtime toggle (`speed-forensic-2026-08-16.md:36-45`) | sealed 1617-token body | 0.32.13 both arms | 136.13 tok/s = 7.346 ms/tok | 209.37 tok/s = 4.776 ms/tok | **2.570** | **1.538×** | | 08-21 cross-runtime A/B (`ab-runs.json`, `results-window/AB-RUN-2026-08-21.md`) | 68-token robber Q1 | 0.32.9 vs 0.32.13 | 154.09 tok/s = 6.490 ms/tok | 203.00 tok/s = 4.926 ms/tok | **1.564** | **1.317×** | ⚠ **The two estate measurements of the same tax disagree — 2.570 vs 1.564 ms/token — and the article must not print one number as "the" number.** The two runs differ in prompt length (1617 vs 68 tokens), in box load (08-16: three residents + live traffic; 08-21: empty card), and in what varied (08-16 varied only the option on one binary; 08-21 varied the whole binary). Since the penalty pass is **CPU** work while decode is **GPU** work, a busier box plausibly makes the *tax* worse without touching the *decode*; that is a hypothesis with an obvious cheap test (§9), not a finding. Honest article framing: *the tax measured between ~1.6 and ~2.6 ms per generated token across two experiments, worth +32% to +54% depending on prompt length and box load.* **A receipt nobody has pointed at yet: the answers themselves changed.** In this morning's A/B, at `temperature 0, seed 0`, the two arms produced *different text* — `eval_count` 128 (OLD) vs 130 (NEW), and different reply hashes, each perfectly stable within its arm (10/10 byte-identical): - OLD-0.32.9 `reply_sha256 = 5ad07c931cbffda072d56a238df805cc2d74d842d396e5808dd542548fa971be` - NEW-0.32.13 `reply_sha256 = d2db508f5f499e2c4a4b66115e8fc97ad79cc51d89d4728bbb42e9c01a856a25` (`[workspace]/rs-runtime-dividend-2026-08-19/ab-runs.json`) That is the fingerprint of a change that **moves the numbers the model is choosing between**, not a change that merely computes the same choice faster. It is also the cleanest possible statement of the trap in `the-speed-legend-memory`: the upgrade changed what every product on that runtime *says*, not just how fast it says it — and no judged bench has yet compared 1.0 against 1.1 output. **And the request never asked for the penalty.** The A/B harness sends only `{"temperature": 0, "seed": 0, "num_ctx": 32768, "num_predict": 220}` (`[workspace]/rs-runtime-dividend-2026-08-19/leg2b_ab.py:56-58`) — no `repeat_penalty`. The model publishes none either: `https://ollama.com/library/gemma4:26b` shows a parameters block of `temperature 1`, `top_k 64`, `top_p 0.95` (matching the-gpu-box's own `/api/show` at `speed-forensic-2026-08-16.md:146-150`). So the value came from the daemon's defaults struct on both sides of the upgrade — the definition of "silently". --- ## 6. MoE-specific vs general — the answer is arithmetic, not a kernel **No MoE-specific speedup shipped in this window.** Neither ollama's 16 commits (§1a) nor llama.cpp's 27 (§3) contain MoE or expert-routing work. The change is model-agnostic: it applies to every model that publishes no `repeat_penalty`. **But the *benefit* is strongly MoE-shaped, and the estate has the receipt.** The penalty tax is a **fixed per-token cost that does not care how big the model is** — it scales with vocabulary size, not with parameter count. So it is a larger share of a model whose per-token compute is small. Two same-box measurements, same probe session: | model | per-token budget without the stage | tax (ms/token) | tax as % | speedup from dropping it | |---|---|---|---|---| | `gemma4:26b` — 25.2B total / **3.8B active**, 128 experts, 8 used (`ollama.com/library/gemma4:26b`; the-gpu-box `/api/show` at `speed-forensic-2026-08-16.md:138-139`) | 4.776 ms | **2.570** | 53.8% of the new budget | **1.538×** | | `qwen3.6:27b` — 27.8B total (`[workspace]/rs-thirteen-2026-08/recon/perf-probe-qwen-penalty.json`) | 12.844 ms | **2.407** | 18.7% of the new budget | **1.187×** | The two taxes are within **7% of each other in absolute milliseconds**, and the resulting speedups differ by **2.9×**. That is the whole MoE story in one table: a sparse mixture-of-experts activates only ~3.8B of its 25.2B parameters per token, so its matmul work is small enough that a fixed CPU-side sampler pass was eating **more than half** of what remained. The same fix on a model that spends 12.8 ms/token of real compute buys a fifth as much. Two honesty notes on that table: (a) the qwen arm toggled **`presence_penalty` 1.5 → 0**, not `repeat_penalty` — the same `penalties` stage, a different knob keeping it alive (qwen3.6:27b *pins* `repeat_penalty 1` and `presence_penalty 1.5` in its own parameters, per `https://ollama.com/library/qwen3.6:27b`, which is why the v0.32.10 default never touched it); (b) whether qwen3.6:27b is dense is **UNVERIFIED** — its model card lists 27.8B total and does not disclose an active-parameter count, so describe it as "the other 27B seat", not "the dense control". **Upstream corroboration for the qwen anomaly the forensic flagged.** The estate found 88 requests still paying the tax on 08-16 because a caller sends `presence_penalty=1.500` (`speed-forensic-2026-08-16.md:246-252`). There is an **open upstream issue** about exactly that default: ollama issue **#17197**, *"Official `qwen3.6` default `presence_penalty 1.5` conflicts with Qwen's thinking-mode spec (should be `0.0`)"*, author `eaglstun`, opened 15 Jul 2026, still **Open** — [tool paraphrase] it argues the 1.5 belongs to non-thinking/Instruct mode while qwen3.6 thinks by default, quoting the reporter: *"users get thinking-mode generation run with the instruct-mode penalty."* So the estate's "find that caller" follow-up has a vendor-side twin: the 1.5 is **inherited from ollama's own model parameters**, not from the estate's config. --- ## 7. The legend, audited Estate memory `the-speed-legend-memory` says: *"+50% = one commit; sampling silently off."* **Correct:** it is one commit; it is a sampling stage; it was silent (a server default, in neither the request nor the Modelfile); the commit is identified and the mechanism is printed by the daemon rather than inferred. **Correction 1 — "disabled or skipped" is right, but the *vendor's* stated reason is not the estate's mechanism.** The commit message and the release note both frame the win as *speculative decoding* ("the penalized target rejects drafted tokens and the depth controller backs off"; "speeding up speculative decoding"). the-gpu-box has **no draft model in play** — its log prints `draft: 0.000 MiB` (`speed-forensic-2026-08-16.md:192-194`). The estate's win comes from the *plain per-token full-vocab penalty pass* the same change removes. The article should say this explicitly: **the vendor undersold its own change, and the estate got a bigger win by a different route than the one in the release note.** (gemma4 does have MTP/DFlash speculative-decoding paths in the wider ecosystem — ollama PR #15980 "mlx: Gemma4 MTP speculative decoding", vLLM PR #41745 — but those are MLX/vLLM paths, and the-gpu-box's receipt says no drafter was loaded.) **Correction 2 — "+50%" needs a condition attached.** +53.8% is the 08-16 figure at a 1617-token prompt on a loaded box; +31.7% is the 08-21 figure at a 68-token prompt on an empty card. Both are real; neither is "the" number (§5). --- ## 8. What I could NOT verify (say UNVERIFIED, do not narrate) 1. **Whether ollama enables llama.cpp backend (GPU) sampling.** The "penalties-forfeits-GPU-sampling" amplifier in §5 is a hypothesis. `llm/llama_server.go` (84 KB at v0.32.13) was not read end-to-end. 2. **Whether upstream skips adding a disabled `penalties` sampler to the chain.** A fetch over `common/sampling.cpp` at b10380 reported the switch adds samplers unconditionally and that `common_sampler_print` emits `"logits " + "-> " + name` with **no `?` prefix** — which does not match the-gpu-box's `?penalties` log format. So the producer of the-gpu-box's `?`-marked chain line is UNVERIFIED (ollama-side variant, a newer llama.cpp, or a different print). It does not affect the mechanism: `is_disabled()` short-circuits the pass either way. It *is* a negative claim from a summarizer over a large file — treat as weak. 3. **v0.32.12's full "What's Changed" list** — the tag page's PR list failed to load. 4. **gemma4's exact vocabulary size.** Third-party writing describes gemma4's lm_head as "~262K" candidates (jarvislabs/medium/blog results from a web search), but I have no first-party receipt; `/api/show` on the-gpu-box would settle it in one read-only call. 5. **Whether `qwen3.6:27b` is dense.** Card shows 27.8B total, no active count. 6. **Any third-party report of the same jump.** A search of ollama issues for "repeat_penalty performance" surfaced nothing about v0.32.10's default; the estate may be the first to have measured and published this particular consequence. --- ## 9. Follow-ups this recon earned (sized, not scoped by me) 1. **One read-only call closes gap #4:** `/api/show gemma4:26b` on the-gpu-box already returns the architecture block the forensic quoted — re-read it for `vocab_size` so the article can say "a quarter of a million numbers per token" with a first-party receipt. 2. **The 1.6-vs-2.6 ms discrepancy (§5) is testable in ~30 seconds of GPU:** replay the *same* prompt at both context lengths on 0.32.13 with `repeat_penalty` 1.0 vs 1.1, on an idle box. If the tax is ~1.6 ms idle at both lengths, box load explains the gap and the article can say so; if it tracks prompt length, something else is going on. 3. **v0.32.15 is an unclaimed dividend** (§2): metadata caching cutting TTFT ~995 ms → ~524 ms. the-gpu-box is on 0.32.13. Interactive rulesage/amble traffic is TTFT-bound, not decode-bound. Worth a pre-registered upgrade leg — with the standing caution that an ollama upgrade can move a *generation default* silently, which is the whole lesson of this article. 4. **The quality half is still unmeasured** (`the-speed-legend-memory`'s trap): no judged bench has compared 1.0 vs 1.1 output on any product seat, and today's A/B proves the text changes. The article should state that plainly rather than implying the dividend was free of consequence. --- ## 10. Receipts index | what | where | |---|---| | the commit | `https://github.com/ollama/ollama/commit/6a261db7d87b13d76c5197cec636a0a3951afb36` | | its patch | `https://github.com/ollama/ollama/commit/6a261db7d.patch` | | its metadata | `https://api.github.com/repos/ollama/ollama/commits/6a261db7d` | | the 16-commit window | `https://api.github.com/repos/ollama/ollama/compare/v0.32.9...v0.32.13` | | release notes | `https://github.com/ollama/ollama/releases/tag/v0.32.{10,11,12,13,14,15}` | | llama.cpp pin, both tags | `https://raw.githubusercontent.com/ollama/ollama/v0.32.{9,13}/LLAMA_CPP_VERSION` → `b10353` / `b10380` | | llama.cpp window | `https://api.github.com/repos/ggml-org/llama.cpp/compare/b10353...b10380` (27 commits) | | penalties source | `https://raw.githubusercontent.com/ggml-org/llama.cpp/b10380/src/llama-sampler.cpp` | | backend-sampling rule | `https://github.com/ggml-org/llama.cpp/pull/17004` (danbev, merged 4 Jan 2026) | | backend sampling perf figures (hardware named in the PR) | `https://github.com/ggml-org/llama.cpp/pull/25532` (gaugarg-nv, merged 10 Aug 2026) | | GGUF ⇒ llama-server | `https://github.com/ollama/ollama/pull/16031` (dhiltgen, merged 29 May 2026) | | the Go runner is MLX-only | `https://raw.githubusercontent.com/ollama/ollama/v0.32.13/runner/runner.go` | | gemma4:26b parameters (no repeat_penalty) | `https://ollama.com/library/gemma4:26b` | | qwen3.6:27b pins presence_penalty 1.5 | `https://ollama.com/library/qwen3.6:27b` | | upstream issue on that 1.5 | `https://github.com/ollama/ollama/issues/17197` (open) | | estate forensic (08-16) | `[workspace]/rs-thirteen-2026-08/recon/speed-forensic-2026-08-16.md` | | estate qwen penalty probe | `[workspace]/rs-thirteen-2026-08/recon/perf-probe-qwen-penalty.json` | | this morning's A/B rows | `[workspace]/rs-runtime-dividend-2026-08-19/ab-runs.json` | | this morning's A/B writeup | `[workspace]/rs-runtime-dividend-2026-08-19/results-window/AB-RUN-2026-08-21.md` | | the A/B harness (no repeat_penalty sent) | `[workspace]/rs-runtime-dividend-2026-08-19/leg2b_ab.py:56-58` | | estate memory (the legend) | `[agent-memory]/the-speed-legend-memory.md` | | archive search | `scribe search "repeat_penalty"` → 50 matches, top hits in session `65ce6e48` (2026-08-12→17) | **Local commands run for this report** (read-only; no builds, no benches): `ls`/`cat`-free file reads of the five files above · `scribe search "repeat_penalty"` · `scribe search "sampling silently off"` · `scribe sessions "repeat_penalty ollama 0.32.10 gemma4 speed"` · `scribe sessions "Benchmark Glimmer model against golden test sets repeat_penalty sampler chain"` · `ls -la [workspace]/rs-thirteen-2026-08/recon/`.