{
 "schema_version": "1.0",
 "page_kind": "article",
 "pack_kind": "generate",
 "slug": "flash-next-why-the-first-word-comes-sooner",
 "title": "Why the first word comes sooner: the Strata engine and llama.cpp on Qwen3.8-Flash-Next, the full bench",
 "dek": "Why the Strata engine (github.com/Niko1221/Strata, an unrelated project) gives the first word of an answer sooner than llama.cpp on the same Qwen3.8-Flash-Next files, and what it costs, measured one request at a time on an RTX 5090 laptop card (24 GB, in a laptop with 64 GB of memory, no settable cap), an RTX PRO 6000 Blackwell (96 GB, at a 300 W cap, one engine a night) and an EVGA GeForce RTX 3090 Ti FTW3 Ultra (24 GB, at a 300 W cap, both engines in one window beside live work on the computer's other card): the wait for the first word, writing speed, board power, quality and agreement, with llama.cpp at commit 3cf03257 at its default 512-token micro-batch and at a 4,096-token one, and on the RTX 3090 Ti also at a current build (b11514), every llama.cpp row without the draft head, which the file we benched does not carry.",
 "published": "2026-10-12",
 "series": [
  "bench"
 ],
 "licence": "CC BY 4.0",
 "status": "pending-judge",
 "pack_note": "no judge is seated, so every chip is held",
 "notice": "items with published:false are held and are not this page's published words",
 "source": {
  "url": "https://research.strata2signal.com/flash-next-why-the-first-word-comes-sooner/",
  "md_url": "https://research.strata2signal.com/flash-next-why-the-first-word-comes-sooner/index.md",
  "md_sha": "7aa985c9754164274e914872f452496b4b6b36608f857e2ce380c651e09367c2",
  "html_sha": "aa805dc7ababc60bf6b659afebe8f4ee34a478a307ceda4451ed4915f5fdd983",
  "anchors_sha": "1fdb404522d08a7d90a23237e41f639a764e57df53297c9c858e19775e6e287a"
 },
 "short": {
  "paragraph": "The Strata engine (github.com/Niko1221/Strata, an unrelated project) is a program that runs one AI model, Qwen3.8-Flash-Next, on one graphics card plus the computer's own memory. The model is built from many small parts, called experts, and uses only a few of them for each token (a word or a piece of one: the unit an engine reads and writes). We ran the Strata engine beside llama.cpp, the open-source engine it builds on, with llama.cpp pinned at one commit (3cf03257). Both used the same model files (the Strata engine also read the model's draft head from files in its own format, which the GGUF files do not carry) and the same prompts, one request at a time, on two computers. The headline, on the laptop card that cannot hold the model: the first word at a 128,037-token prompt came 545.1 s after sending on llama.cpp at commit 3cf03257 (its default 512-token micro-batch, 8 of the laptop's 24 threads, the experts of 34 of 48 layers in system memory, no draft head because the file we benched carries no draft-head block; llama.cpp at more than 8 threads was not measured) against 47.6 s on the Strata engine 0.1.25 (8,192 tokens a read, 23 expert workers plus its host thread, drafting from the model's draft head, its run beside another job's dry run), the two back to back on 2026-10-02 and 2026-10-03 (UTC), which is the rule that admits a comparison there; the other two cards' figures are below, each under its own rule.",
  "from_dek": false,
  "counts": {
   "words": 20010,
   "minutes": 91,
   "tables": 16,
   "kit": false
  },
  "bullets": []
 },
 "sections": [
  {
   "id": "what-this-page-is",
   "heading": "What this page is",
   "level": 2,
   "span": [
    15711,
    16319
   ],
   "chunks": [
    [
     15711,
     16319
    ]
   ],
   "chars": 608,
   "digest": "a small workshop measures why the Strata engine gives the first word of an answer sooner than llama.cpp on the same files and what it costs, testing on a 24 GB laptop card, a 96 GB desktop card, and a 24 GB desktop card.",
   "digest_skipped": null
  },
  {
   "id": "the-words-this-page-leans-on",
   "heading": "The words this page leans on",
   "level": 2,
   "span": [
    16319,
    23815
   ],
   "chunks": [
    [
     16319,
     23815
    ]
   ],
   "chars": 7496,
   "digest": "this section defines terms used in the article, including token, model, engine, mixture of experts, room on the card, GB and GiB, threads and workers, prefill, decode, micro-batch, context, KV cache, expert cache, draft head, flash attention, quantization formats, and various measurement rules.",
   "digest_skipped": null
  },
  {
   "id": "which-cards-this-page-covers",
   "heading": "Which cards this page covers",
   "level": 2,
   "span": [
    23815,
    25474
   ],
   "chunks": [
    [
     23815,
     25474
    ]
   ],
   "chars": 1659,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "one-model-two-engines",
   "heading": "One model, two engines",
   "level": 2,
   "span": [
    25474,
    28862
   ],
   "chunks": [
    [
     25474,
     28862
    ]
   ],
   "chars": 3388,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "the-question-and-the-instruments",
   "heading": "The question, and the instruments",
   "level": 2,
   "span": [
    28862,
    36924
   ],
   "chunks": [
    [
     28862,
     36924
    ]
   ],
   "chars": 8062,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "why-the-first-word-comes-sooner-in-three-sentences",
   "heading": "Why the first word comes sooner, in three sentences",
   "level": 2,
   "span": [
    36924,
    38462
   ],
   "chunks": [
    [
     36924,
     38462
    ]
   ],
   "chars": 1538,
   "digest": "the Strata engine's lead is attributed to how it handles expert trips during prompt reading, its use of the model's draft head during writing, and its larger micro-batch size compared to llama.cpp.",
   "digest_skipped": null
  },
  {
   "id": "the-first-word-the-laptop-the-iq3-s-file",
   "heading": "The first word (the laptop, the IQ3_S file)",
   "level": 2,
   "span": [
    38462,
    44479
   ],
   "chunks": [
    [
     38462,
     44479
    ]
   ],
   "chars": 6017,
   "digest": "this section provides first-word and prompt rate measurements for llama.cpp and the Strata engine on a laptop using the IQ3_S file, including various prompt lengths and thread counts.",
   "digest_skipped": null
  },
  {
   "id": "writing-the-answer-the-laptop-the-iq3-s-file",
   "heading": "Writing the answer (the laptop, the IQ3_S file)",
   "level": 2,
   "span": [
    44479,
    49566
   ],
   "chunks": [
    [
     44479,
     49566
    ]
   ],
   "chars": 5087,
   "digest": "this section details writing speeds for the laptop using the IQ3_S file, including expert cache hit rates, tokens a verify round, and comparisons between different engine versions and thread counts.",
   "digest_skipped": null
  },
  {
   "id": "the-q2-0-pair-the-laptop",
   "heading": "The Q2_0 pair (the laptop)",
   "level": 2,
   "span": [
    49566,
    53088
   ],
   "chunks": [
    [
     49566,
     53088
    ]
   ],
   "chars": 3522,
   "digest": "this section compares first-word and writing speeds for the Q2_0 file on a laptop, noting how llama.cpp's memory limit was reached during its clean re-read.",
   "digest_skipped": null
  },
  {
   "id": "what-the-card-drew-and-what-the-cpu-package-drew-the-laptop",
   "heading": "What the card drew, and what the CPU package drew (the laptop)",
   "level": 2,
   "span": [
    53088,
    56483
   ],
   "chunks": [
    [
     53088,
     56483
    ]
   ],
   "chars": 3395,
   "digest": "this section reports energy consumption for the graphics card and CPU package on the laptop for both engines using the IQ3_S and Q2_0 files.",
   "digest_skipped": null
  },
  {
   "id": "the-desktop-card-one-engine-a-night",
   "heading": "The desktop card: one engine a night",
   "level": 2,
   "span": [
    56483,
    67235
   ],
   "chunks": [
    [
     56483,
     67235
    ]
   ],
   "chars": 10752,
   "digest": "this section compares the two engines on a 96 GB card at a 300 W cap, providing writing speeds and first-word measurements under the cross-night rule.",
   "digest_skipped": null
  },
  {
   "id": "the-24-gb-desktop-card-both-engines-in-one-window",
   "heading": "The 24 GB desktop card: both engines in one window",
   "level": 2,
   "span": [
    67235,
    82478
   ],
   "chunks": [
    [
     67235,
     82478
    ]
   ],
   "chars": 15243,
   "digest": "this section compares the engines on a 24 GB desktop card in one window, including writing speeds, first-word times, and prompt rates for llama.cpp and the Strata engine.",
   "digest_skipped": null
  },
  {
   "id": "are-the-answers-the-same",
   "heading": "Are the answers the same?",
   "level": 2,
   "span": [
    82478,
    84064
   ],
   "chunks": [
    [
     82478,
     84064
    ]
   ],
   "chars": 1586,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "is-it-any-good",
   "heading": "Is it any good?",
   "level": 2,
   "span": [
    84064,
    87291
   ],
   "chunks": [
    [
     84064,
     87291
    ]
   ],
   "chars": 3227,
   "digest": "the article evaluates the model on six-way and doorman tests, noting that it scored within noise of 27B models on the six-way but was behind on the doorman test due to reply format.",
   "digest_skipped": null
  },
  {
   "id": "who-the-strata-engine-is-for-and-who-it-is-not-for",
   "heading": "Who the Strata engine is for, and who it is not for",
   "level": 2,
   "span": [
    87291,
    89109
   ],
   "chunks": [
    [
     87291,
     89109
    ]
   ],
   "chars": 1818,
   "digest": "the Strata engine is for users wanting Qwen3.8-Flash-Next on a Linux computer with an NVIDIA card and sufficient memory, but it is not for users needing other models, Mac users, or those with 32 GB of RAM.",
   "digest_skipped": null
  },
  {
   "id": "what-was-written-down-before-the-first-run",
   "heading": "What was written down before the first run",
   "level": 2,
   "span": [
    89109,
    93170
   ],
   "chunks": [
    [
     89109,
     93170
    ]
   ],
   "chars": 4061,
   "digest": "this section details the registration of bands, gates, and predictions made before the measurements were taken, including the desktop registration and the laptop's ninth amendment.",
   "digest_skipped": null
  },
  {
   "id": "what-else-the-bench-read",
   "heading": "What else the bench read",
   "level": 2,
   "span": [
    93170,
    94728
   ],
   "chunks": [
    [
     93170,
     94728
    ]
   ],
   "chars": 1558,
   "digest": "this section notes a llama.cpp GPU memory fault, the speed cost of llama.cpp's memory map, the Strata engine's memory sizing on the laptop, and the engine's network surface.",
   "digest_skipped": null
  },
  {
   "id": "still-pending-and-when-it-lands",
   "heading": "Still pending, and when it lands",
   "level": 2,
   "span": [
    94728,
    96393
   ],
   "chunks": [
    [
     94728,
     96393
    ]
   ],
   "chars": 1665,
   "digest": "a table lists planned updates, such as the Strata engine 0.1.39 laptop rows, quality runs, and desktop same-night pairs, with target dates.",
   "digest_skipped": null
  },
  {
   "id": "what-it-does-not-say",
   "heading": "What it does not say",
   "level": 2,
   "span": [
    96393,
    99302
   ],
   "chunks": [
    [
     96393,
     99302
    ]
   ],
   "chars": 2909,
   "digest": "this section clarifies the limits of the article, stating it does not cover other models, Mac, agentic work, wall power, or comparisons between different cards or computers.",
   "digest_skipped": null
  },
  {
   "id": "what-comes-next-on-this-page",
   "heading": "What comes next on this page",
   "level": 2,
   "span": [
    99302,
    99823
   ],
   "chunks": [
    [
     99302,
     99823
    ]
   ],
   "chars": 521,
   "digest": "the article lists upcoming updates, including the same-night pair on the 96 GB card and the Strata engine 0.1.39 quality run on the laptop.",
   "digest_skipped": null
  },
  {
   "id": "what-to-take-with-you",
   "heading": "What to take with you",
   "level": 2,
   "span": [
    99823,
    104618
   ],
   "chunks": [
    [
     99823,
     104618
    ]
   ],
   "chars": 4795,
   "digest": "this section summarizes the key findings regarding first-word speed, writing speed, energy, and quality for the laptop and desktop environments.",
   "digest_skipped": null
  },
  {
   "id": "who-ran-this-and-thanks",
   "heading": "Who ran this, and thanks",
   "level": 2,
   "span": [
    104618,
    105229
   ],
   "chunks": [
    [
     104618,
     105229
    ]
   ],
   "chars": 611,
   "digest": null,
   "digest_skipped": "credits"
  },
  {
   "id": "appendix-sources-reproduce-per-run-telemetry",
   "heading": "Appendix: sources, reproduce, per-run telemetry",
   "level": 2,
   "span": [
    105229,
    121467
   ],
   "chunks": [
    [
     105229,
     121467
    ]
   ],
   "chars": 16238,
   "digest": "this appendix provides instructions for reproducing the results from source, lists the build lines, and provides a table of sources for every figure in the article.",
   "digest_skipped": null
  }
 ],
 "chips": [
  {
   "id": "c-065679b0",
   "q": "what model and engine types are compared?",
   "a": "The article compares the Strata engine and llama.cpp using the Qwen3.8-Flash-Next mixture-of-experts model. It examines different compressed weight formats, specifically IQ3_S and Q2_0, across various hardware configurations.",
   "cites": [
    "one-model-two-engines"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-f75cdabb",
   "q": "how does the first word timing differ on a laptop?",
   "a": "On a laptop using the IQ3_S file, the Strata engine provides the first word significantly sooner than llama.cpp. At a 128K prompt length, the Strata engine takes 47.6 seconds compared to llama.cpp's 545.1 seconds at a 512-token micro-batch.",
   "cites": [
    "the-first-word-the-laptop-the-iq3-s-file"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-46bf1e7f",
   "q": "how do the engines compare on a 96 gb card?",
   "a": "On a 96 GB card, the Strata engine writes 205.263 tokens a second at 4K, while llama.cpp writes 102.889 tokens a second. The Strata engine's lead is considered beyond unmeasured night-to-night variation.",
   "cites": [
    "the-desktop-card-one-engine-a-night"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  }
 ],
 "related": [
  {
   "slug": "flash-next-on-three-cards",
   "why": "shares ground with § What this page is · § Writing the answer"
  },
  {
   "slug": "ollama-and-vllm-on-one-rtx-3090",
   "why": "shares ground with § The desktop card: one engine a night · § Requests at once"
  },
  {
   "slug": "six-cards-the-same-questions",
   "why": "shares ground with § Writing the answer (the laptop, the IQ3_S file) · § Writing: how fast each one answers"
  }
 ],
 "thanks": "## Who ran this, and thanks {#who-ran-this-and-thanks}\n\nThe Strata engine's author (github.com/Niko1221/Strata; MIT); Georgi Gerganov and the ggml authors (llama.cpp, MIT); ISTA-DASLab (the GGUF files); Qwen (the model and its draft head, under the Qwen Community License 1.0); NVIDIA (CUDA). None of them owed us anything. A small (human) team wrote the questions, pressed the keys and read the pages; a fleet of AI agents built the harness, ran the audits and drafted the prose. Despite the similar name, Strata (github.com/Niko1221/Strata) is an unrelated project; strata2signal has no affiliation with it.",
 "kit": null,
 "seat_class": {
  "writer": "gemma-class",
  "judge": null,
  "writer_runtime": "vllm"
 },
 "bench": {
  "bullets_written": 3,
  "bullets_kept": 0,
  "digests_written": 22,
  "digests_kept": 18,
  "chips_written": 8,
  "chips_kept": 3,
  "chips_grounded": 0,
  "chips_published": 0,
  "dropped_by": {
   "new_noun": 2,
   "figure": 1,
   "length": 1,
   "cite": 0,
   "judge": 0,
   "quote": 0,
   "directive": 0,
   "redaction": 1,
   "profanity": 0,
   "identifier": 0,
   "bare_figure": 0,
   "house_voice": 0,
   "editorial": 0,
   "hand-edit": 7
  },
  "new_noun_tokens_checked": [
   "0.1.39",
   "096",
   "102.889",
   "108",
   "11",
   "125",
   "128K",
   "14.2",
   "2.2",
   "2.5",
   "2.7",
   "2.8",
   "205.263",
   "24",
   "27B",
   "300",
   "3090",
   "32",
   "4",
   "40",
   "40-90",
   "46.84",
   "47.6",
   "48",
   "4K",
   "5090",
   "512",
   "512-token",
   "545.1",
   "6000",
   "69.8",
   "91",
   "948",
   "96",
   "CPU",
   "EVGA",
   "Edition",
   "FTW3",
   "GB",
   "GPU",
   "GeForce",
   "GiB",
   "IQ3S",
   "KV",
   "Laptop",
   "Linux",
   "Mac",
   "MiB",
   "NVIDIA",
   "PRO",
   "Q20",
   "Qwen3.8-Flash-Next",
   "RAM",
   "RTX",
   "Strata",
   "Ti",
   "Ultra",
   "W",
   "Workstation"
  ],
  "new_noun_tokens_withheld": 3,
  "figure_definition": "v3",
  "figure_boundary": "guarded",
  "drops": [
   {
    "kind": "digest",
    "reason": "redaction",
    "item": null,
    "item_chars": 238,
    "withheld": "a string carrying a fenced literal",
    "detail": ""
   },
   {
    "kind": "digest",
    "reason": "new_noun",
    "item": null,
    "item_chars": 4,
    "withheld": "a string naming something the article does not",
    "detail": ""
   },
   {
    "kind": "bullet",
    "reason": "figure",
    "item": "the Strata engine 0.1.39 on the 96 GB card used 40,948 MiB",
    "detail": "figures absent from span: ['96']"
   },
   {
    "kind": "chip",
    "reason": "new_noun",
    "item": null,
    "item_chars": 3,
    "withheld": "a string naming something the article does not",
    "detail": ""
   },
   {
    "kind": "chip",
    "reason": "length",
    "item": "how does writing speed compare on a laptop with IQ3_S?",
    "detail": "q not lowercase"
   },
   {
    "kind": "bullet",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 48,
    "text_sha": "509765c019b6",
    "figure": "205.263",
    "withheld": "one bullet removed from this pack by hand",
    "removed_utc": "2026-10-12",
    "why": "Bare figure: the section gives 205.263 in tokens a second and the bullet lost the unit, the loss gate 8 dropped on the twin page; it also names no card, and beside bullet 1 the same subject reads as two speeds.",
    "detail": "removed from this pack by hand on 2026-10-12 (UTC), after the writer ran and after the gates had counted: Bare figure: the section gives 205.263 in tokens a second and the bullet lost the unit, the loss gate 8 dropped on the twin page; it also names no card, and beside bullet 1 the same subject reads as two speeds. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "bullet",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 53,
    "text_sha": "d87003b5d342",
    "figure": "40-90",
    "withheld": "one bullet removed from this pack by hand",
    "removed_utc": "2026-10-12",
    "why": "Ends in a band, 40-90, with no unit and no word saying it is a band; the speed was 85.6 tokens a second and one run sat outside; beside bullet 2 the same subject reads as two speeds.",
    "detail": "removed from this pack by hand on 2026-10-12 (UTC), after the writer ran and after the gates had counted: Ends in a band, 40-90, with no unit and no word saying it is a band; the speed was 85.6 tokens a second and one run sat outside; beside bullet 2 the same subject reads as two speeds. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 226,
    "text_sha": "b13977a4e26a",
    "section": "one-model-two-engines",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-10-12",
    "why": "Gives 125 billion as the whole model and drops the section's plus a 51-billion-parameter lookup table and a 4-billion-parameter draft head; the section's three figures add to 180 billion.",
    "detail": "removed from this pack by hand on 2026-10-12 (UTC), after the writer ran and after the gates had counted: Gives 125 billion as the whole model and drops the section's plus a 51-billion-parameter lookup table and a 4-billion-parameter draft head; the section's three figures add to 180 billion. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 134,
    "text_sha": "4d29d5f37252",
    "section": "are-the-answers-the-same",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-10-12",
    "why": "Reads as both pairs missing both halves; the section says the IQ3_S pair agreed on 108 of 108 six-way items and missed only the doorman half, the Q2_0 pair only the six-way half.",
    "detail": "removed from this pack by hand on 2026-10-12 (UTC), after the writer ran and after the gates had counted: Reads as both pairs missing both halves; the section says the IQ3_S pair agreed on 108 of 108 six-way items and missed only the doorman half, the Q2_0 pair only the six-way half. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 259,
    "text_sha": "934c9a16bf4f",
    "chip_id": "c-6839e433",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-12",
    "why": "Drops the section's pin at commit 3cf03257, so it reads as llama.cpp writing one token a step in general; the page says upstream llama.cpp added this model's draft head on 2026-10-01.",
    "detail": "removed from this pack by hand on 2026-10-12 (UTC), after the writer ran and after the gates had counted: Drops the section's pin at commit 3cf03257, so it reads as llama.cpp writing one token a step in general; the page says upstream llama.cpp added this model's draft head on 2026-10-01. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 255,
    "text_sha": "ad79952c49cb",
    "chip_id": "c-f3e149b4",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-12",
    "why": "Gives the doorman's 11 of 48 as a judgment score without the section's reason: 36 of 48 replies came in a form the strict scorer does not read, and the section says it reads format more than judgment.",
    "detail": "removed from this pack by hand on 2026-10-12 (UTC), after the writer ran and after the gates had counted: Gives the doorman's 11 of 48 as a judgment score without the section's reason: 36 of 48 replies came in a form the strict scorer does not read, and the section says it reads format more than judgment. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 268,
    "text_sha": "ad3cd0d9ea49",
    "chip_id": "c-94063e07",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-12",
    "why": "Turns the section's fit list into the engine's design intent; the section says 0.1.39 adds opt-in batch slots and a multi-conversation cache, and that the engine also builds for AMD's HIP.",
    "detail": "removed from this pack by hand on 2026-10-12 (UTC), after the writer ran and after the gates had counted: Turns the section's fit list into the engine's design intent; the section says 0.1.39 adds opt-in batch slots and a multi-conversation cache, and that the engine also builds for AMD's HIP. Not a gate drop, and recorded here so the ledger adds up."
   }
  ]
 },
 "generated_utc": "2026-10-12T14:38:41Z"
}