Qwen3.8-Flash-Next on an RTX 5090 laptop, an RTX PRO 6000 Blackwell and an RTX 3090 Ti

Writing speed on a 4,038-token prompt, in tokens a second, each card under its own rule for what may be compared:

• The laptop: an NVIDIA GeForce RTX 5090 Laptop GPU (24 GB) in a laptop with 64 GB of memory. llama.cpp at commit 3cf03257, no draft head: 24.4 tokens a second. The Strata engine 0.1.25, drafting from the model's draft head (a small extra network the model ships to guess tokens ahead): 85.6 tokens a second, 3.5 times as fast; the two ran back to back on the same laptop, about an hour apart (2026-10-02 and 2026-10-03, UTC), which is the rule that admits a ratio here; llama.cpp read the prompt at its default 512-token micro-batch (512 tokens at a time) and ran on 8 of the laptop's 24 threads, with no draft head because the file we benched carries no draft-head block (llama.cpp at more than 8 threads was not measured); the Strata engine ran 23 expert workers (its own CPU threads for expert work, as its log counts them) plus its host thread (one more thread, by the same log), and its run shared the laptop with another job's dry run (other work of ours on the same laptop at the same time; which way that cuts is not established).
• The desktop's 96 GB card: an NVIDIA RTX PRO 6000 Blackwell Workstation Edition at a 300 W cap, in a desktop with 123.39 GiB of memory, its services stopped. llama.cpp at 3cf03257, no draft head: 102.889 tokens a second. The Strata engine 0.1.39, drafting from the model's draft head: 205.263 tokens a second, 2.0 times as fast; one engine a night (2026-10-06 and 2026-10-07) at the 300 W cap, a gap beyond the wide margin we set for readings taken on different nights, which is the rule that admits a ratio here; every expert on the card for both engines; llama.cpp read the prompt at its default 512-token micro-batch and ran on 8 of the desktop's 64 threads, with no draft head because the file we benched carries no draft-head block (llama.cpp at more than 8 threads was not measured); the Strata engine ran 31 expert workers plus its host thread.
• The same desktop's 24 GB card: an EVGA GeForce RTX 3090 Ti FTW3 Ultra at a 300 W cap, in the same desktop with 123.39 GiB of memory, both engines in one window beside live work on the computer's other card. llama.cpp at 3cf03257, no draft head: 22.490 tokens a second. The Strata engine 0.1.39, drafting from the model's draft head: 98.291 tokens a second, 4.4 times as fast, beyond the registered margin (the margin we wrote down for this pair before it ran), which is the rule that admits a ratio here; llama.cpp read the prompt at its default 512-token micro-batch, kept 34 of 48 layers' experts in system memory (the laptop's fit, the layer split found on the laptop, not refitted on this card) and ran on 8 of the computer's 64 threads (the laptop's line, its launch line as the laptop ran it, not refitted for this computer), with no draft head because the file we benched carries no draft-head block (llama.cpp at more than 8 threads was not measured); the Strata engine ran 31 expert workers plus its host thread, with 8,144 of 24,576 experts on the card; those eight threads, not the card, set llama.cpp's writing speed.

On the laptop, where the card cannot hold the model, the first word of the answer also came much sooner on the Strata engine 0.1.25 (23 expert workers, drafting from the model's draft head; its run shared the laptop with another job's dry run): 47.6 s after a 128K-token prompt, where llama.cpp at 3cf03257, no draft head, reading the prompt at its default 512-token micro-batch on 8 threads (llama.cpp at more than 8 threads was not measured), took 545.1 s, on the same laptop and under the same conditions as the laptop's row above (204.4 s at a 4,096-token micro-batch, its logical batch raised to match, a control run on 2026-10-05, three days later, beside two of our audit jobs (other work of ours on the same laptop at the same time), with the experts of 42 of 48 layers in system memory). Every number here is printed first, with its file and field, on Why the first word comes sooner: the Strata engine and llama.cpp on Qwen3.8-Flash-Next, the full bench. No ratio is drawn between cards: the computers, the builds and the placement of the model differ.

Qwen3.8-Flash-Next has about 180 billion parameters in all. By its model card, 125 billion are in its main network, of which about 6 billion are used for each token, plus a 51-billion-parameter lookup table and a 4-billion-parameter draft head. It is built as a mixture of experts: its 48 layers hold many small sub-networks ("experts", 24,576 slots in all), and for each token it uses only a few of them, which is why a model this large runs at all on a card with 24 GB of memory, in a computer with the memory to hold every expert (46.84 GiB pinned in system memory for the file we benched; our laptop's 64 GB was just enough). We compared the two engines on the same compressed files. llama.cpp, pinned at one commit (3cf03257), is the engine the Strata engine builds on. The Strata engine pins the experts in the computer's own memory with the most-used ones copied to the card, and drafts several tokens a step from the model's own draft head (the small extra network the model ships, "multi-token prediction"), which it reads from files in its own format beside the compressed files. llama.cpp at that commit has no support for this model's draft head; llama.cpp added support on 2026-10-01, but the compressed file we benched carries no draft-head block (its header, read on 2026-10-09), so llama.cpp's draft head (merged upstream in pull request #29761, and present in b11514, the current build we ran) was not run on it, and no llama.cpp row on this page has the draft head, the RTX 3090 Ti's row at a current build included.

Every figure here was measured by us between 2026-10-02 and 2026-10-09 (UTC) and is printed first on the full bench, Why the first word comes sooner: the Strata engine and llama.cpp on Qwen3.8-Flash-Next, the full bench, with every row's file and field, the three runs behind every median, the quality tests and everything this page leaves out. This page adds no figure the full bench does not print; nothing here speaks for a later build of either engine.

The three cards, and how they were run

The same method on all three: the same compressed files (pinned by hash) in the IQ3_S compression (a 3-bit-class copy, 54.8 GB plus a 28.8 GB lookup table), which carry no draft-head block (the Strata engine adds the draft head from files in its own format); prompts of 4,038 and 32,038 tokens ("4K" and "32K"); 256 tokens written, greedy (the engine always takes its most likely next token), one request at a time; three runs at each length, the median of three with the first request after load included, on our own client's clock, every reply's token count checked. The full bench also prints 128,037-token rows ("128K") for every card whose run reached that length. llama.cpp: its default 512-token micro-batch (how many prompt tokens it reads at a time), 8 threads (the laptop's line, raised there from llama.cpp's own default of 4 before the runs, which moved its writing 4 to 21 %, and kept on every card; more were not measured), flash attention on, an 8-bit KV cache. The Strata engine: the line its own setup writes, prompts read 8,192 tokens at a time, the draft head in use, its own pool of expert workers (it takes no thread setting). A registered reading had its range and tests written down and dated before it ran; an exploratory one had none written down, so it confirms or refutes no prediction; the comparison rules below apply to it all the same, and every such figure carries the label.

Words used here. The draft head is a small extra network the model ships to guess a few tokens ahead; an engine checks the guesses in one step of the model, a verify round, and keeps the right ones, so one round can yield more than one token. Expert-pool workers are the Strata engine's own CPU threads for expert work. The KV cache is an engine's working memory for the text so far (8-bit says how compactly it is stored). Flash attention is a faster way to compute attention. The hit rate is the share of expert lookups the Strata engine served from the card. A band is the range we wrote down for a 4K writing speed before the run, from published bandwidths (the card's, the computer's memory's and the link's, as their makers state them, not measured) over the bytes each token reads; a reading is "inside" or "outside" it. Dynamic Boost is the laptop moving power between its processor and its card. AVX-512 is a set of wide vector instructions the desktop's processor lacks; there the Strata engine's expert kernels run on the older AVX-2. The full bench defines each at more length: The words this page leans on.

Three rules decide what may be compared. On the laptop the two engines ran back to back, about an hour apart (llama.cpp 2026-10-02 23:03:15Z to 23:40:55Z, the Strata engine 2026-10-03 00:46:41Z to 00:50:45Z, UTC; the date changes only because the hour crossed midnight), so their ratio is printed. On the 96 GB card they ran one engine a night, so a gap is reported only when it is beyond unmeasured night-to-night variation: larger than 10 % of the larger reading and than three times the wider row's run-to-run spread, a wide margin on purpose, because we have never measured how much a reading drifts from night to night. On the RTX 3090 Ti both engines ran in one window beside changing live work, under the same margin, registered before the pair ran ("the registered margin"); a ratio prints only when a gap is beyond it. Between cards nothing is compared as a ratio: the laptop and the desktop are two computers; the two desktop cards share a computer and the same 300 W cap but not a card, a build of each engine, a placement of the model or a night. Board watts are the card's own readings, never wall power.

Writing the answer

Tokens a second after the first word, on our client's clock, the median of three; one decimal on the laptop, three on the desktop cards, as the full bench prints them. Every row's conditions are in the table above. llama.cpp at commit 3cf03257 has no support for the model's draft head, and the file we benched carries no draft-head block for any llama.cpp build to read; the Strata engine (github.com/Niko1221/Strata) drafted from the draft head, read from files in its own format. The desktop cards' 32K rows are exploratory (no band was drawn at 32K). The last row is llama.cpp at a current build, b11514 (commit de7fa0a3), 452 commits after 3cf03257, on the RTX 3090 Ti only: the same file, line and layer split, the build the only change in the line; it ran last in the window and found the model's weights file already in the computer's file cache from the arms before it (0 major page faults on its nine runs), a condition its figures carry; and no draft head, because the file we benched carries none.

On the laptop the Strata engine 0.1.25 wrote 3.5 times as fast as llama.cpp at 3cf03257 at 4K and 4.2 times at 32K (85.6 against 24.4, 87.1 against 20.7 tokens a second), the two engines back to back, both 4K medians inside the bands we wrote down before the run; it drafted from the model's draft head, which llama.cpp at that commit lacks, and ran 23 expert workers against llama.cpp's 8 threads at its default 512-token micro-batch (llama.cpp at more than 8 threads was not measured), and its run shared the laptop with another job's dry run. On the 96 GB card the Strata engine 0.1.39 wrote 2.0 times as fast at 4K and 2.5 times at 32K (205.263 against 102.889; 189.155 against 76.337, exploratory), one engine a night at 300 W, both gaps beyond unmeasured night-to-night variation, every expert on the card for both engines, llama.cpp at its default 512-token micro-batch on 8 of the desktop's 64 threads against the Strata engine's 31 expert workers; per step of the model it was slower, 78.58 verify rounds a second against 102.9 single-token steps, and each round yielded 2.612 tokens, so on our reading of its own counts the lead on this card is the draft head, which llama.cpp at 3cf03257 did not run. On the RTX 3090 Ti, at its 300 W cap in a computer with 123.39 GiB of memory, both engines in one window beside live work on the computer's other card: the Strata engine 0.1.39 wrote faster at 4K and at 32K (98.291 against 22.490 tokens a second, 4.4×; 92.571 against 19.620, 4.7×, exploratory), both beyond the registered margin, with 8,144 of 24,576 experts on the card (a hit rate of 0.921 at 4K), 2.586 tokens a verify round from the model's draft head and 31 expert-pool workers, against llama.cpp at 3cf03257, no draft head, at its default 512-token micro-batch with 34 of 48 layers' experts in system memory (the laptop's fit, not refitted on this card) on 8 of the computer's 64 threads (the laptop's line, not refitted for this computer; llama.cpp at more than 8 threads was not measured); those eight threads, not the card, set llama.cpp's writing speed. llama.cpp at a current build on the same card (b11514, 452 commits later; the same file, line, layer split and window; no draft head): it wrote 22.657 tokens a second at 4K against the pinned commit's 22.490, within the registered margin, so no ratio is printed: two llama.cpp builds 452 commits apart on the same file, line, layer split and card, the later one run last in the window with the model's weights file already in the computer's file cache; neither build ran the draft head. Nothing above is a ratio between cards. The rows behind these figures, with the three runs behind every median, are on the full bench: Writing the answer (the laptop) · The desktop card: one engine a night · The 24 GB desktop card.

The first word

Seconds from sending the prompt to the first token of the answer, on our client's clock, the median of three with the first request after load included; every row's conditions are the two tables' above. llama.cpp reads the prompt 512 tokens at a time (its default) and, on a card that cannot hold the model, ships the experts it keeps on the CPU across to the card once per chunk; the Strata engine (github.com/Niko1221/Strata) reads 8,192 at a time, sixteen times as many tokens for each trip (our reading of both engines' logs; neither project says so). A 4,096-token micro-batch was run only on the laptop, never on the desktop cards. The desktop cards' 32K rows are exploratory. One chart per prompt length follows, because on one scale the 4K bars would vanish.

At 4K the wait is short on the 96 GB card, which holds the whole model: its two readings, 2.52 s on llama.cpp at 3cf03257 (no draft head) and 1.06 s on the Strata engine 0.1.39 (drafting from the model's draft head), one engine a night at 300 W, sit within unmeasured night-to-night variation under our rule, so no ratio is printed. On the laptop's 24 GB card, which cannot hold it, the wait was 19.1 s on llama.cpp at 3cf03257 (its default 512-token micro-batch, 8 threads, no draft head) against 1.9 s on the Strata engine 0.1.25 (23 expert workers, drafting from the model's draft head), a tenfold gap, back to back (llama.cpp at more than 8 threads was not measured; the Strata engine's run shared the laptop with another job's dry run; 7.6 s with llama.cpp reading 4,096 tokens at a time, its logical batch raised to match, a control run on 2026-10-05, three days later, beside two of our audit jobs, where the bigger batch's compute buffer on the card pushed 8 more layers of experts, 42 of 48, into system memory). On the RTX 3090 Ti, at its 300 W cap in a computer with 123.39 GiB of memory, beside live work on the computer's other card: the Strata engine 0.1.39's first word came sooner (1.92 against 16.87 s, 8.8×), beyond the registered margin, with 8,144 of 24,576 experts on the card, 8,192-token prompt reads and 31 expert workers against llama.cpp at 3cf03257 with its 512-token micro-batch and 34 of 48 layers' experts in system memory (the laptop's fit, not refitted on this card) on 8 of the computer's 64 threads (the laptop's line, not refitted for this computer; llama.cpp at more than 8 threads was not measured); llama.cpp b11514, no draft head, read 15.84 s.

At 32K the laptop's gap is 125.6 s on llama.cpp at 3cf03257 (its default 512-token micro-batch, 8 threads, no draft head) against 12.1 s on the Strata engine 0.1.25 (23 expert workers, drafting from the model's draft head), 10.4 times, back to back (50.0 s with the 4,096-token control run of 2026-10-05, its differences named at 128K below; llama.cpp at more than 8 threads was not measured; the Strata engine's run shared the laptop with another job's dry run); the 96 GB card's is 20.91 s on llama.cpp at 3cf03257 against 6.37 s on the Strata engine 0.1.39, 3.3 times, one engine a night at 300 W, beyond unmeasured night-to-night variation (exploratory), where every layer sits on the card for both engines and the difference is the prompt path: llama.cpp's default 512-token micro-batch on 8 of the desktop's 64 threads against the engine's 8,192-token reads on 31 expert workers (a bigger llama.cpp batch was not run on this card). On the laptop, one length further (on the full bench): at 128,037 tokens llama.cpp at 3cf03257 waited 545.1 s at its default micro-batch, against the Strata engine 0.1.25's 47.6 s, on the same laptop and under the same labels as its rows above; and 204.4 s at a 4,096-token micro-batch, its logical batch raised to match (a control run on 2026-10-05, three days after the 545.1 s run, beside two of our audit jobs, with the experts of 42 of 48 layers in system memory), printed beside that 47.6 s with no ratio drawn. On the 96 GB card at that length: 94.22 s on llama.cpp at 3cf03257 against 26.63 s on the Strata engine 0.1.39, 3.5 times, one engine a night at 300 W, beyond unmeasured night-to-night variation, against llama.cpp's default micro-batch on 8 of the desktop's 64 threads, the Strata engine on its 31 expert workers. On the RTX 3090 Ti, at its 300 W cap in a computer with 123.39 GiB of memory, beside live work on the computer's other card: the Strata engine 0.1.39's first word came sooner (12.14 against 138.02 s, 11.4×, exploratory), beyond the registered margin, with 8,144 of 24,576 experts on the card, 8,192-token prompt reads and 31 expert workers against llama.cpp at 3cf03257 with its 512-token micro-batch and 34 of 48 layers' experts in system memory (the laptop's fit, not refitted on this card) on 8 of the computer's 64 threads (the laptop's line, not refitted for this computer; llama.cpp at more than 8 threads was not measured); llama.cpp b11514, no draft head, read 130.90 s. The rows behind these figures, with the three runs behind every median, are on the full bench: The first word (the laptop) · The desktop card: one engine a night · The 24 GB desktop card.

What each card drew

Board power, the card's own one-second readings over each request, the median of three at each prompt length; never wall power (no memory, fans, screen or drive are counted, and both engines compute part of the model on the CPU on the 24 GB cards). The desktop cards' caps are set limits; the laptop's floating 135-150 W limit was reached by the Strata engine at 128K. The 96 GB card's 4K requests last a few seconds, so those medians rest on two to six samples each; every row's sample counts are in the last column (the laptop's 32K counts are not printed on the full bench). Recorded beside the speed rows, claiming nothing; nothing here compares one computer with another.

The 96 GB card sat at its 300 W cap under both engines (297.8 and 290.6 W at 4K), so there the watts say what the limit was, not what the engine did; the RTX 3090 Ti's medians are in the table beside its 300 W cap, and llama.cpp's lower draw there (168.5 against the Strata engine's 281.1 W at 4K) came over a request that ran several times longer, the card waiting on llama.cpp's eight CPU threads, so it is not lower energy; no energy figure is printed for the desktop cards. On the laptop the Strata engine 0.1.25 drew about twice the board power of llama.cpp at 3cf03257 (126.3 against 61.1 W at 4K; its run shared the laptop with another job's dry run) for a much shorter time per request. Energy per request is printed on the full bench for the laptop only (the desktop rows record watts and claim nothing about energy): on the registered rows (llama.cpp at 3cf03257, 2026-10-02; the Strata engine 0.1.25, 2026-10-03, its run sharing the laptop with another job's dry run) the card drew 1.77 against 0.51 kJ per request at 4K, 9.43 against 1.96 at 32K and 38.26 against 7.28 at 128K, llama.cpp 3.5 to 5.3 times the Strata engine's, because each of the Strata engine's requests finished so much sooner; the CPU package's share is on the full bench: What the card drew, and what the CPU package drew. No watt or joule here is compared across computers.

Each card, in one paragraph

The laptop: an RTX 5090 laptop card, 24 GB, in a laptop with 64 GB of memory. The card cannot hold the model, so each engine keeps part of it in the computer's own memory: llama.cpp the experts of 34 of the 48 layers, computed on the CPU while writing and shipped to the card once per 512-token chunk while reading; the Strata engine 0.1.25 all 24,576 experts pinned in system memory (46.84 GiB), the 8,120 its shipped profile ranks hottest copied to the card, prompts read 8,192 tokens at a time, several tokens a step drafted from the model's draft head. Three things explain the lead in our reading of the logs, the batch, the cache (about 92 % of expert lookups found the expert on the card) and the draft head (2.2 to 2.8 tokens a verify round), not split here. The cost is memory: at 128K the laptop had 2.66 GiB of its 61.23 GiB left under the Strata engine 0.1.25, against our stop line of 2.0, its run sharing the laptop with another job's dry run. A re-read at 0.1.39 on 2026-10-09 produced no figure, so this page draws the 0.1.25 rows for the laptop, labelled.

The desktop's 96 GB card: an RTX PRO 6000, at a 300 W cap. Both engines put every layer and every expert on the card (only the model's two input lookup tables stay in host memory); the computer's other services were stopped, its second card kept its own work, and each engine had its own night, because the desktop does other work. The first word at 4K came within unmeasured night-to-night variation of llama.cpp's; at 32K and 128K it came 3.3 and 3.5 times sooner on the Strata engine 0.1.39 (20.91 against 6.37 s; 94.22 against 26.63 s; exploratory), one engine a night at 300 W, beyond unmeasured night-to-night variation, against llama.cpp at 3cf03257, no draft head, at its default 512-token micro-batch on 8 of the desktop's 64 threads (a bigger batch was not run on this card), the Strata engine on 31 expert workers, drafting from the model's draft head. Writing, the lead has one source on our reading of the engine's own counts, the draft head: per step the Strata engine 0.1.39 was slower (78.58 verify rounds a second against llama.cpp's 102.9 steps) and each round yielded 2.612 tokens. llama.cpp at 3cf03257 has no support for it. Upstream added support on 2026-10-01, but the file we benched carries no draft-head block, so llama.cpp's draft head (#29761, b11514) was not run on it. Even with every expert on the card the engine held 49.60 GiB of the desktop's memory: the memory condition does not go away with a bigger card.

The desktop's 24 GB card: an RTX 3090 Ti, at a 300 W cap, in the same computer, with 123.39 GiB of memory. The laptop's question on a desktop: a card the size of the laptop's (24 GB) in a computer with an AMD EPYC 7532 (32 cores, Zen 2, no AVX-512), 128 GB of DDR4-3200 in eight channels (123.39 GiB reported) and a PCIe 4.0 x16 link, where the laptop has an Intel Core Ultra 9 290HX Plus, 64 GB of DDR5-5600 in two channels and a card link the Strata engine's own probe read at 28.7 GB/s. llama.cpp again kept the experts of 34 of the 48 layers in system memory (the laptop's fit, not refitted on this card), and the Strata engine 0.1.39 again ran with part of its cache on the card (8,144 of 24,576 slots, a hit rate of 0.921 at 4K, 2.586 tokens a verify round). Both engines ran in one window on 2026-10-09 (09:32Z to 10:51Z), each built for this card's architecture, beside the computer's live services, which kept running on the 96 GB card (this card's own usual work was moved there for the window) and shared the processor and memory with both engines; the 4K writing medians were read against bands drawn before the run (llama.cpp's 4K median below it (opened, a reading outside its band that we then examined, as our registration requires: its eight CPU threads, the laptop's setting kept on a computer with many more, set its writing speed; the card did not); the Strata engine's 4K median inside the band we wrote down before the run; llama.cpp b11514's 4K median below it (opened: its eight CPU threads, the laptop's setting kept on a computer with many more, set its writing speed; the card did not)). The Strata engine 0.1.39, drafting from the model's draft head with its 31 expert workers, wrote faster at 4K and at 32K (98.291 against 22.490 tokens a second, 4.4×; 92.571 against 19.620, 4.7×, exploratory) and its first word came sooner at 32K (12.14 against 138.02 s, 11.4×, exploratory), each beyond the registered margin, with 8,144 of 24,576 experts on the card against llama.cpp at 3cf03257, no draft head, at its default 512-token micro-batch with 34 of 48 layers' experts in system memory (the laptop's fit, not refitted on this card) on 8 of the computer's 64 threads (the laptop's line, not refitted for this computer; llama.cpp at more than 8 threads was not measured; those eight threads, not the card, set llama.cpp's writing speed); at 4K the first word (1.92 against 16.87 s) read beyond the registered margin. A third arm ran on this card only: llama.cpp at a current build, b11514 (commit de7fa0a3, 2026-10-08), 452 commits after 3cf03257, with the same file, line and layer split. It ran without the draft head: on 2026-10-01 llama.cpp added support for this model's draft head, but the file we benched carries no draft-head block, and b11514 refuses to start the head on it; it wrote 22.657 tokens a second at 4K against the pinned commit's 22.490 (within the registered margin; no ratio) and 21.257 against 19.620 at 32K, exploratory (within the registered margin; no ratio), its first word 15.84 and 130.90 s against 16.87 and 138.02: two llama.cpp builds 452 commits apart, the same file, line, layer split and card, the later one run last in the window with the model's weights file already in the computer's file cache; neither build ran the draft head. Nothing here is a ratio against the laptop's card (the same 24 GB; a different computer, build and placement) or the 96 GB card (the same computer and the same 300 W cap; a different card, build and night). Each card's own section on the full bench: The first word (the laptop) · Writing the answer (the laptop) · The desktop card: one engine a night · The 24 GB desktop card.

What we found

1. On a 24 GB card that cannot hold the model, the first word came about ten times sooner on the Strata engine (llama.cpp 545.1 s, the Strata engine 47.6 s at 128K; 19.1 against 1.9 s at 4K): llama.cpp at 3cf03257, no draft head, its default 512-token micro-batch, 8 threads (llama.cpp at more than 8 threads was not measured), the experts of 34 of 48 layers in system memory; the Strata engine 0.1.25 at 8,192 tokens a read on 23 expert workers, drafting from the model's draft head, its run shared with another job's dry run; back to back on 2026-10-02 and 2026-10-03. A 4,096-token micro-batch, llama.cpp's logical batch raised to match, cut its wait by 60.3, 60.2 and 62.5 % at 4K, 32K and 128K (204.4 s at 128K, a control run on 2026-10-05, three days later, beside two of our audit jobs, with 8 more layers of experts, 42 of 48, in system memory); the Strata engine's 47.6 s stands beside that 204.4 s with no ratio drawn here: the control was not back to back with the Strata engine's run, so the full bench prints only the arithmetic between them, beside the five things that differ. The first word (the laptop) · What a 4,096-token micro-batch does
2. On the same laptop the answer was written 3.5 to 4.9 times faster on the Strata engine 0.1.25 (llama.cpp 24.4, the Strata engine 85.6 tokens a second at 4K; 14.2 against 69.8 at 128K), 23 expert workers against 8 threads at llama.cpp's default 512-token micro-batch (llama.cpp at more than 8 threads was not measured), drafting from the model's draft head, which llama.cpp at 3cf03257 lacks; both banded 4K medians inside their bands; its run shared the laptop with another job's dry run, back to back with llama.cpp's. Writing the answer (the laptop)
3. On a 96 GB card with room for every expert, the lead comes from the draft head, on our reading of its own counts: the Strata engine 0.1.39 wrote 2.0 times llama.cpp at 3cf03257 at 4K (205.263 against 102.889 tokens a second), one engine a night at 300 W in a computer with 123.39 GiB of memory, its services stopped, every expert on the card for both engines, llama.cpp at its default 512-token micro-batch on 8 of the desktop's 64 threads against the Strata engine's 31 expert workers, beyond unmeasured night-to-night variation, while making fewer model steps a second (78.58 rounds against 102.9) at 2.612 tokens a round; llama.cpp did not run the draft head (the file we benched carries no draft-head block). We never ran the Strata engine without its draft head; the arm that would have done so, on the laptop, would not start. The first word at 4K was within night-to-night variation, and at 32K came 3.3 times sooner (20.91 against 6.37 s, exploratory), one engine a night at 300 W, against llama.cpp at 3cf03257 at its default 512-token micro-batch; a bigger batch was not run on this card. The desktop card: one engine a night
4. On a 24 GB desktop card in a computer with 123.39 GiB of memory, both engines in one window at 300 W beside live work on the computer's other card: the Strata engine 0.1.39, drafting from the model's draft head, wrote faster at 4K (98.291 against 22.490 tokens a second, 4.4×) and gave the first word sooner at 32K (12.14 against 138.02 s, 11.4×, exploratory), both beyond the registered margin, with 8,144 of 24,576 experts on the card and 31 expert-pool workers against llama.cpp at 3cf03257, no draft head, at its default 512-token micro-batch with 34 of 48 layers' experts in system memory (the laptop's fit, not refitted on this card) on 8 of the computer's 64 threads (the laptop's line, not refitted for this computer; llama.cpp at more than 8 threads was not measured); those eight threads, not the card, set llama.cpp's writing speed. llama.cpp at a current build (b11514), no draft head, on the same card: llama.cpp b11514 (commit de7fa0a3) wrote 22.657 tokens a second at 4K against 22.490 at 3cf03257, within the registered margin; no ratio; its first word at 4K 15.84 against 16.87 s. Two builds 452 commits apart, the same file, line, layer split and card, the later one run last in the window with the model's weights file already in the computer's file cache (0 major page faults on its nine runs); neither build ran the draft head, because the file we benched carries none. The other two cards' llama.cpp rows are at 3cf03257. The 24 GB desktop card
5. For the Strata engine, system memory is the gate, not the card. It pins every expert in the computer's memory on every card, 46.84 GiB for this file; the 64 GB laptop had 2.66 GiB left at 128K under the Strata engine 0.1.25, its run sharing the laptop with another job's dry run, and even with every expert on the 96 GB card the engine held 49.60 GiB of the desktop's memory. llama.cpp on the laptop mapped about 61.8 GiB against 61.23 reported (a memory map can exceed the memory installed; the system pages the file in and out) and peaked at 27.46 GiB of host memory on the 96 GB card. Who the Strata engine is for
6. Quality on the laptop: the same six-way score through either engine, a doorman gap that is reply format, and both below the bar we set (the Strata engine 0.1.25; llama.cpp at 3cf03257 with flash attention off, its V cache in 16-bit, at an 8K context, because its flash-attention build stopped on one quality prompt): 91 of 108 on our six-way reading test (108 lines, each asking which of six dinner guests said it), within noise of two 27-billion-parameter models on the same laptop (93 and 96, provisional rows on another engine); on our 48-item doorman test, which asks for a one-letter verdict, most replies came back as a small JSON object our strict scorer does not read, 36 of 48 through the Strata engine (11 right) and 40 of 48 through llama.cpp (7 right), so the gap is reply format, not judgment (the two engines' doorman answers matched on 44 of 48 items, under the 47 our agreement gate asked); the bar we set before the run, the two comparators' own scores (93 of 108 and 36 of 48), was not met on either engine; no "better" anywhere. Is it any good?

Which card and engine for which case

Each row is read off the full bench's rows; nothing was measured for this table. "Built from source" is our own rule: we never ran either project's installer, setup script or prebuilt binary. The Strata engine builds for NVIDIA's CUDA or AMD's HIP; we tested NVIDIA only, on Linux.

What it does not say

• No ratio between cards: the two 24 GB cards share a size, not a computer (an Intel Core Ultra 9 290HX Plus with 64 GB of DDR5-5600 in two channels, against an AMD EPYC 7532 with 128 GB of DDR4-3200 in eight channels and a PCIe 4.0 x16 link), a build or a placement; the two desktop cards share a computer and a 300 W cap, not a card, a build or a night.
• Not an RTX 3090: the 24 GB desktop card here is an RTX 3090 Ti; nothing about two cards working together.
• One request at a time; the versions benched only (llama.cpp at 3cf03257 on all three cards and, on the RTX 3090 Ti only, at b11514, both without the draft head, which the file we benched does not carry; the Strata engine at 0.1.25 and 0.1.39); the IQ3_S file.
• Not wall power; no energy figure for the desktop cards.
• The 96 GB card is one engine a night, compared only under the cross-night rule; the laptop's 0.1.39 re-read produced no figure, so no 0.1.39 laptop row is here; the laptop's registered Strata rows shared the laptop with another job's dry run; the RTX 3090 Ti's rows ran beside the computer's live services on its other card.
• llama.cpp ran near its defaults, not at its tuned best: the 512-token micro-batch (4,096 tried once, on the laptop), 8 threads, the memory map its own log says costs speed, and, on every build, no draft head (the file carries none) and no separate draft model.
• Not code, not agentic work, not long answers, not sampled output; quality on two short tests only, on the laptop, and the doorman reads reply format as much as judgment.
• The data kit has not shipped; it will sit beside the full bench, not this page.
• The full bench's own list of what it does not say is longer: What it does not say.

Who ran this, and where the files are

The Strata engine's author (github.com/Niko1221/Strata; MIT); Georgi Gerganov and the ggml authors (llama.cpp, MIT); ISTA-DASLab (the GGUF files, the compressed copies of the weights in the file format llama.cpp reads); Qwen (the model and its draft head, under the Qwen Community License 1.0); NVIDIA (CUDA). None of them owed us anything. A small (human) team wrote the questions, pressed the keys and read the pages; a fleet of AI agents built the harness, ran the audits and drafted the prose. Every figure on this page is printed first on Why the first word comes sooner: the Strata engine and llama.cpp on Qwen3.8-Flash-Next, the full bench, with its row, file and field; this page adds no figure the full bench does not print. Despite the similar name, Strata (github.com/Niko1221/Strata) is an unrelated project; strata2signal has no affiliation with it.