Speeds from RAM alone

Why this page

The two Ollama models here are two the house keeps resident on its graphics cards: the Mistral model that reads what strangers type at RuleSage, our board-game rules helper, and Gemma 4, the family that speaks for RealKeep, a small game (its 12-billion model is the narrator; this bench ran the 26-billion mixture). Two Hours at 12 tok/s — On Battery measured both with no card at all and concluded "A real emergency tier exists."; memory is dear: The 3080 Rig, Priced read 32 GB of DDR5-5600 at $399.99 on 2026-09-10 and $439.99 on 2026-10-08, more than the used graphics card beside it on every read. This page is that tier on three more machines, with the biggest mixture of experts this shelf has run from memory alone.

A few words this page leans on

• Token — the chunk a model reads and writes in, a little under a word; tokens a second is how fast the answer appears: about 10 or more feels fluid, 4 to 6 comfortable, by the house's rule of thumb.
• Prefill and decode — reading the whole question, then writing one token at a time: "reading the prompt" and "writing" here.
• Memory bandwidth — the bytes a second the processor can pull from memory, in GB/s (a thousand million bytes a second), measured with a STREAM triad, a small program that streams three large arrays and times it.
• Threads — the workers the engine runs at once, one per hardware thread (a core, or half of a core that runs two); Ollama's library picks a default, printed beside one other setting per machine.
• Dense and mixture of experts — a dense model uses every parameter for every token; a mixture wakes a fraction (gemma4:26b about 4 billion of 26; the 180-billion mixture about 6), though the whole model must still sit in memory, all but the big mixture's lookup table. Mixture of experts, in plain words explains the rest.
• Q4_K_M and IQ3_S — the weights stored in about four and about three bits each; the Ollama models are the battery page's files by digest, the big mixture its earlier page's two files.
• NOT WALL — every package watt and joule here is the processor package's own energy counter, not a meter at the wall; the whole computer draws more. The two cards' idle watts are each card's own reading.
• THROTTLED — a run during which the processor's throttle counter rose (the laptop) or its control temperature reached its limit (the small desktop), at any point from sending the request to its last token, the model's load and the prompt's reading included; a label, never a reason to drop the figure. The eight-channel desktop publishes no throttle counter, so the label does not read there.

Memory speed, and the rule this bench registered

The battery page's rule: tokens a second ≈ bandwidth ÷ bytes streamed per token. A dense model streams nearly all its weights every token; a mixture only its active fraction. That page checked itself with gemma4:26b on its three machines: 24.7, 13.3 and 12.7 tokens a second over 100.7, 59.3 and 51.1 GB/s gave 0.245, 0.224 and 0.248 tokens a second per GB/s, "the same ~4 GB of weights-and-cache move per token on all three boxes". This bench registered the check before it ran: for gemma4:26b at the engine's default threads, k is the writing speed over the machine's best triad; the rule holds if k lands between 0.20 and 0.28, the published spread widened by about a tenth each side, else reads above or below. One over k is the GB the rule says moved per token.

The laptop's 0.2445 is 4.1 GB a token, the published figure. The two-channel desktop's 0.6366 is 1.6 GB a token: the model wrote 2.6 times faster than a memory moving 21.4 GB/s can feed the published 4.1 GB a token. The triad is not the noisy part; its nine reps spread 0.3 per cent. Either that memory moves more than the triad read, or the model moves far less than 4 GB a token on that processor; the record holds neither as a reading, nor the sticks' rated speed, so the verdict stands as printed, unexplained. The registration also prints the dense model's ratio at the default threads, with no verdict, because the battery page found that model thread-starved at four threads: 0.0443 on the laptop (2.65 over 59.83) and 0.1199 on the two-channel desktop (2.57 over 21.43). That second ratio leans one way, by arithmetic and not by a reading: 8.3 GB a token at the triad's 21.43 GB/s, against the 15.3 GB the engine reported loaded for a dense model that reads nearly the whole of itself every token; so while it wrote, that memory moved more than the triad read, and why the triad read low is not in the record.

For the big mixture the rule is a ceiling: the best triad over 5.159 GB a token, the whole model's per-token read as its earlier page registered it, 19.03 tokens a second at 98.16 GB/s. The triad scores 24 bytes an element where the hardware moves 32, so the ceiling can read up to a third low; a block above 1.35 times it would have been voided as helped by a card or a cache. None came near.

How it was measured

One registration, frozen 2026-11-01 at 15:09:22Z before any run, amended twice in writing on 2026-11-02 and 2026-11-03, before the small desktop ran (its processor and memory had been recorded as another machine's; a build folder on the eight-channel desktop had been renamed). Each machine ran its triad once after a quiet minute (ten reps, the first discarded, the best and worst of nine printed), then its model arms inside a sandbox with no graphics device files, the card libraries' device variables emptied, and no network but its own loopback. No card was used, proven on each machine by checks that all had to pass: the sandbox's own probe; the engine's own evidence (on the two Ollama machines, zero bytes on any card after every load, 16 reads each; on the eight-channel desktop, start logs listing only processor buffers and finding no device); the host's card monitor outside the sandbox never listing a process of the run; no run voided; and, on the eight-channel desktop, the writing speed under the ceiling. Every machine read PROVEN.

The two Ollama machines ran Ollama 0.32.15, the battery page's version, with its two model files by digest (the registry's gemma4:26b tag has since moved), its frozen prompt (536 tokens as Gemma reads it, 1,023 as Mistral does, its default system prompt included), 256 tokens written, a 4,096-token window, temperature 0. Each arm, one model at one thread setting, ran a warm-up and three scored runs, the model unloaded and the engine read back empty before each, so no prompt-reading figure here can come from a cached prefix, the battery page's trap. Writing speed is the engine's token count over its writing time, prompt-reading likewise, the first word the client's clock less the load; each printed figure is the median of three.

The big mixture ran through llama.cpp build 11514 (commit de7fa0a3), the binary its earlier page ran on this same desktop with an EVGA GeForce RTX 3090 Ti FTW3 Ultra (24 GB) helping, now with every layer on the processor and an 8,192-token window instead of 131,072, the one registered difference, to leave memory for the live services. Its 54.8 GB first part was read into memory before the first block (16.9 s); the second, a lookup table, is read row by row as tokens need it. Three prompts of 4,038 tokens, the earlier page's own, 256 tokens each, the first request after each load included as that page counted it; writing speed there is the client's clock, the tokens after the first over the time after the first; three blocks at 8, 32 and 16 threads, each a fresh server. Package energy came from the processor's own counter where the run could read one; the small desktop's could not be read. A request's energy runs from sending it to its last token: on the laptop that includes the model's fresh load from disk, 13 to 18 s of every scored request; on the eight-channel desktop, whose model stayed loaded through each block, it does not, so the two columns are not one measure.

The laptop: Intel Core Ultra 9 290HX Plus, 64 GB of DDR5-5600

Measured 2026-11-05, 16:10:36Z to 16:36:34Z, on mains with the fan on and the one timer that uses the card paused. The processor has 8 performance cores without hyperthreading and 16 efficiency cores, 24 threads; the library beneath Ollama counted 4, the halving the battery page found on its laptop under this same Ollama. The card in it, an NVIDIA GeForce RTX 5090 Laptop GPU 24 GB, idled at about 12 W outside the sandbox.

Idle, the package drew 13.5 W (NOT WALL).

At the default the mixture wrote 5.5 times faster than the dense model, and 1.8 times at twenty. Twenty threads made the dense model 2.08 times faster and cut its first word from 83 s to 28 s; the same setting made the mixture a third slower, 0.67 of its speed, while it read the prompt 2.5 times faster and its CPU package drew 32 W more while writing (NOT WALL). What the record shows about why: the THROTTLED labels were earned while the prompt was read, not while the answer was written. In the mixture's four-thread runs the throttle counter rose by about 200 to 800 during the prompt, the package touching 102 to 105 °C, and by at most 24 during the writing, the package peaking at 82 to 88 °C; at twenty threads it rose by four in one run, during the model's load, and never during writing, the package peaking at 75 to 76 °C with about 21 of the 24 threads busy (about 5 at four threads). The twenty-thread runs were slower all the same (10.01, 9.80 and 8.74). So the record does not show heat as the cause: the mixture wrote slower at twenty threads while running cooler. The dense model's THROTTLED labels came mostly the same way, its counter rising by about 860 to 8,300 while its prompt was read and by at most 252 while it wrote, and twenty threads still doubled its writing. The battery page's gaming laptop read the same cliff, 13.3 to 8.4 at four and twenty threads (its dense model 2.7 and 5.1), over a 59.3 GB/s triad, and read it as a mixture with so little work per token that "the coordination overhead — threads fighting over the same starved memory bus, plus the small cores dragging a static schedule — swamps it"; this page did not test that and prints it as that page's reading. This laptop's rows sit close to those, and this page does not say the two are one machine.

The two-channel desktop: AMD Ryzen 7 3700X, 32 GB of DDR4

Measured 2026-11-04, 15:39:29Z to 16:16:21Z, by day with nothing else running (other work under 0.02 of a core). Eight cores, 16 threads, no efficiency cores; the library counted 8, the physical cores, as it did on the battery page's server. Two 16 GB sticks, 30.3 GiB as the system reports it. Its card, an EVGA GeForce RTX 3090 XC3 Ultra 24 GB, idled at about 25 W outside the sandbox.

The control temperature peaked between 65 and 76 °C in each run, under its 95 °C limit. Halving the threads cost the mixture 13 per cent and bought the dense model 4; both prompt-reading rates roughly halved; the three runs of each arm sit within 1.1 per cent of each other. The mixture wrote 5.3 times the dense model's speed at 8 threads and 4.5 at 4. At each machine's default threads (8 here, 4 on the laptop) this desktop wrote within a token a second of the laptop on both models, from memory that read at a third of the laptop's bandwidth: the verdict above in one sentence. Given 20 threads, the laptop wrote the dense model at 5.51, twice this desktop's best.

The eight-channel desktop: AMD EPYC 7532, 128 GB of DDR4-3200, the big mixture

Measured 2026-11-03, 07:30:01Z to 07:59:09Z, in the computer's nightly bench window beside its live services, which stayed up and read unchanged at the end. Thirty-two cores, 64 threads; eight 16 GB dual-rank sticks, one in each channel. Every card in the machine, an NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB) among them, was hidden from the run and proven unused while it kept its own work (one run saw a card at 75 per cent). The model is Qwen3.8-Flash-Next at IQ3_S, 54.8 GB of weights and a 28.8 GB lookup table: 125 billion parameters in its experts and layers, about 6 billion awake per token, and 51 billion in that table, about 180 billion by its model card, which also counts a 4-billion draft head these files do not carry.

Idle, the package drew 54.2 W (NOT WALL). This processor publishes no throttle counter, so THROTTLED cannot read here; its hottest sensor read 47 to 54 °C. In no run did other work average more than about half a core (0.28 to 0.53; one core would have labelled it BOX SHARED), and none faulted more than 1,000 of its model's pages back from disk.

Two expectations were registered. First, that at 8 threads memory alone writes slower than the published pair on this binary, 22.657 tokens a second with that EVGA GeForce RTX 3090 Ti FTW3 Ultra (24 GB) helping and the experts of 34 of the 48 layers left in memory: it holds, 6.51, a 3.5-times gap the card had been closing. Second, that 32 threads beat 8: it holds, by 1.279 times, with 16 threads between at 1.19. The earlier page read its 8-thread row as thread-bound ("7.63 to 7.90 of the 8 busy"); four times the threads bought 28 per cent, and the record does not say what binds at 32. The share of the ceiling, 34 to 44 per cent, is what memory alone bought this model on this one desktop: comfortable by the rule of thumb, never fluent, and the first word of a 4,038-token question is the real cost, 98 seconds at best, about three and a half minutes at the pair's thread count. More threads also cost less energy a request, 19.3 kJ against 26.6 (NOT WALL). What the memory bought is the row itself: 128 GB held the 54.8 GB part whole beside the live services; a 32 GB machine could not have held it in memory.

What the numbers do not say

• They do not join these machines to the battery page's three (its server's triad 100.7 GB/s against this eight-channel desktop's 98.16); that page names by class, this one by parts.
• The two-channel desktop's rows are not joined to any desktop row this shelf published earlier.
• No wall watts: the bench's one meter, the eight-channel desktop's UPS, carries other devices, so its reading is not that computer's draw.
• Three scored runs per arm, not the battery page's five, and fresh loads where that page's writing speeds came from warm runs.
• One prompt length per model, one window each, one quantization each, two or three thread settings: no sweep, no long context, no quality reading (the replies were hashed, not kept).
• The two-channel desktop's "above" has no explanation here, and the big mixture's ceiling is conservative by up to a third.

Who ran this, and thanks

The registration, the kit that ran it and the three result folders are the record; every figure above is read from them, and the data kit beside this page carries each leg's summary, triad and receipt line. The writing stood on Ollama 0.32.15 and llama.cpp beneath it, the big mixture on llama.cpp build 11514. The models are Google's Gemma 4 at 26 billion, Mistral Small 3.2 from Mistral AI, and Qwen3.8-Flash-Next; each earlier page names the licence it found. A small human team pasted the stages, armed the night window and signed the amendments; a fleet of AI agents built the kit, read the files and wrote this page under that team's rulings.

Sources

• The bench's own result files for the three runs (2026-11-03, 2026-11-04 and 2026-11-05 UTC), its registration (frozen 2026-11-01T15:09:22Z) and its two signed amendments; the data kit beside this page carries each leg's summary, triad and receipt line.
• Two Hours at 12 tok/s — On Battery: "Read this before the table", "The one table to remember", "The wait", "Set the thread count", "What this buys the house".
• Why the first word comes sooner: "The 24 GB desktop card: both engines in one window", and its registration's bytes-a-token figure.
• The 3080 Rig, Priced: "The bill, priced 8 October 2026".
• Mixture of experts, in plain words.