# PREREG-W0 · R-LB-IQ3, llama.cpp 3cf03257 whole on the desktop's larger card, llama-bench (exploratory) *Internal: names the desktop, the desktop's larger card, the desktop's smaller card and local paths; not for shipping. Times are UTC from `date -u`.* | | | |---|---| | **Registered** | 2026-10-05T00:03:53Z (`date -u` on the laptop, read just before this file was written), by the lane `desktop-w0-stage` | | **Before any reading** | Yes. No R-LB-IQ3 reading exists, and none can exist before the window opens at 2026-10-05 07:30Z. The engine was still building when this was written; nothing has run on any card | | **Preamble** | `~/bench-archive/LANE-PREAMBLE.md` v3.1, sha256 `07986860dd5975c70aaf062677a8ef0b4051a2ad0ae2e89c26346f75ae8ce017` (matched) | | **Authority** | an operator, 2026-10-05 ~00:2xZ (as the brief quotes it): "let's designate [a local clock, withheld] for the desktop bench windows, and start running whatever benches we need to do on the desktop in that window each night" / "so let's roll with all your recs on this" / "and please, do bench something in the 2.5 hours you have tonite!". Ledger D-20261005-096, D-20261005-097, and D-20261005-098 (tonight's shape) | | **Status** | **Exploratory.** It is not a row of the Strata bench PREREG (`plans-2026-10-02/strata-bench/PREREG.md`). It adds no arm there and changes none of that file's bands or gates | | **Companion** | `STAGE.md` in this folder: every sha256 and size, the build, the copy, and the disk readings | --- ## 1 · What the row measures How fast llama.cpp at commit 3cf03257 runs Qwen3.8-Flash-Next (ISTA-DASLab GSQ-RCO IQ3_S, the same two shard files the laptop's L arms use) when the model sits **whole on the desktop's larger card**, the desktop's NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB), with nothing else on that card. It reads llama.cpp's own `llama-bench`: - **prompt processing:** a 512-token synthetic prompt (`pp512`), in tokens per second; - **generation:** 128 generated tokens (`tg128`), in tokens per second; - each at three context depths already in the KV cache: **0, 32,768 and 131,072 tokens** (`-d`, which llama-bench supports at 3cf03257: `tools/llama-bench/llama-bench.cpp:462` prints `-d, --n-depth `, and `:611` parses it); - **3 repetitions** per test, after llama-bench's own warmup run, reported as mean and standard deviation in its JSON. llama-bench runs the tests in this order, with depth as the outer loop (`llama-bench.cpp:1313-1338`): pp512@d0, tg128@d0, pp512@d32768, tg128@d32768, pp512@d131072, tg128@d131072. It fills each non-zero depth once per test and then restores the saved state for the other repetitions (`:2404-2440`). The model loads once for all six tests. If the runtime cap cuts the row, the deepest tests are the ones lost, and the banded reading (tg128@d0) is the second test run. ## 2 · The row's identity | Item | Value | |---|---| | Label | **R-LB-IQ3** | | Box and card | the desktop; **the desktop's larger card only** (PCI bus withheld-2026-10-13, CUDA index 0 in PCI order) | | Engine | llama.cpp `3cf03257f219afbe7334045ff7c6a06ac68c627d` ("CUDA: enable sparse fa for qwen4 (#28770)"), built **natively on the desktop** by this lane in `~/bench-flashnext/llama.cpp-3cf03257/build`; the `llama-bench` binary's sha256, the cache flags and the toolkit are in `STAGE.md` | | Build flags | `-G Ninja -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120 -DLLAMA_USE_PREBUILT_UI=OFF`, `GGML_NATIVE` at its default (ON, so the CPU code targets the desktop's own EPYC 7532: AVX2, no AVX-512, no AVX-VNNI) | | File | `~/bench-flashnext/models/IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-00001-of-00002.gguf` (54,817,524,224 B, sha256 `4c1eb2ceb4915e1192f4f386021897bde56a97f40a0bb78bb86465e0f7d2aca3`) and `…-00002-of-00002.gguf` (28,800,138,432 B, sha256 `316b46f3a2dbd68c900f43136ab9449f9dcc3725dfd8c794847c204bc161e113`), ISTA-DASLab revision `ed59f920`; the pins are the laptop's fetch receipts (`plans-2026-10-02/strata-bench/receipts/fetch-ista.log` lines 15-16), and `STAGE.md` records the sha256 read on the desktop | | Placement | every layer on the desktop's larger card: `-ngl 99`, `-ncmoe 0` (no expert on the CPU), `-sm none -mg 0`. The two input lookup tables, `token_embd` (337,715,200 B) and `per_layer_token_embd` (the n-gram table, 28,800,138,240 B, shard 2), stay in host memory, read row by row through the file mapping: that is llama.cpp's own placement for input embeddings, as on any single-card run, and it is not a split across cards. The desktop's smaller card (the 3090 Ti) is hidden from the process and is never used | | Attention and cache | flash attention **on** (`-fa on`) and an 8-bit KV cache (`-ctk q8_0 -ctv q8_0`), as the laptop's speed rows (`strata-bench/harness/engines.py:116-118`) | | Loading | `-lm mmap -lzm on`, as the laptop's L line (`engines.py:119`); at 3cf03257 lazy mode "on" reads the rows of the tensors the architecture marks on demand (`include/llama.h:221`) | | Threads, batches | `-t 8` (the laptop line's 8, D-20261002-048); `-b 2048 -ub 512` (llama-bench's defaults, named so the row does not lean on them) | | Conditions | the desktop's seats down per D-098 (docent-vllm, ollama, ollama-11435, and comfyui if active; the warm-seats and power-reconcile timers paused; the desktop's larger card read near-empty before the row); the desktop's larger card at its in-service cap, **420 W**, read back before the row. D-098 lets the desktop's smaller card's seat (`ollama-11435`) stay up if the row leaves the desktop's smaller card untouched, and this row does: the desktop's smaller card is hidden by `CUDA_VISIBLE_DEVICES`, `-sm none` keeps the model on one device, and the build carries no Vulkan backend (`STAGE.md` reads `GGML_VULKAN` from the cache). Whether it stays up is the window kit's call, and the row's receipt says which | | Power | not banded. If the window kit samples watts, they are recorded beside the row and claim nothing | ## 3 · The exact command Run as an operator on the desktop, inside the window kit's transient user unit (its runtime cap, below), with the JSON on stdout and the load log and progress lines on stderr: ``` CUDA_DEVICE_ORDER=PCI_BUS_ID CUDA_VISIBLE_DEVICES=0 /workshop/bench-flashnext/llama.cpp-3cf03257/build/bin/llama-bench -m /workshop/bench-flashnext/models/IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-00001-of-00002.gguf -ngl 99 -ncmoe 0 -sm none -mg 0 -fa on -ctk q8_0 -ctv q8_0 -lm mmap -lzm on -t 8 -b 2048 -ub 512 -p 512 -n 128 -d 0,32768,131072 -r 3 -o json -v --progress > R-LB-IQ3.json 2> R-LB-IQ3.stderr.log ``` - `-v` turns llama.cpp's own log on (llama-bench silences it otherwise, `llama-bench.cpp:2246-2247`). It is the placement receipt: the offload line, and the CUDA0 and CPU buffer sizes. - `-o json` prints each test's object as the test finishes, so a row cut by the cap keeps every finished test. - The binary is checked against its sha256 in `STAGE.md` before the row, and the two shards against theirs. ## 4 · The prediction (tg128 at depth 0) **The arithmetic.** - **The desktop's larger card's memory bandwidth:** 1,792 GB/s (NVIDIA's published figure for the RTX PRO 6000 Blackwell Workstation Edition: 512-bit GDDR7). It is a published spec, not measured on the desktop. - **Bytes read per generated token, from the strata recon:** each token routes to 10 of 512 experts in each of 48 layers, so 480 experts. Each expert holds 3 × 2,560 × 640 = 4,915,200 weights, and IQ3_S's routed experts average 3.331 bits per weight (`plans-2026-09-29/strata-recon/RECON-v1.md`, the parameter arithmetic and the ISTA table). So 480 × 4,915,200 × 3.331 / 8 = **982.4 MB of routed experts per token.** The file's own header gives the same figure: 10/512 of its 50,292,326,400 B arena is 982,272,000 B (0.01 % apart). - **The rest of each token's read** is the dense path, which the recon names ("the rest of the 6B active is the dense path and the shared expert") but does not size for IQ3_S. This lane read it from shard 1's tensor table (headers only, `STAGE.md` §5): every non-expert tensor in shard 1 except `token_embd`, which is 3,589,245,440 B of attention, DeltaNet, shared-expert, router and hyper-connection weights; 65,740,800 B of the n-gram projection (`ple_*`); and 521,472,000 B of output head. That is **4,176,458,240 B.** The two lookup tables add about 1,360 B (one `token_embd` row) and 1,440 B (16 n-gram rows of 90 B, the recon's figure) per token, which is negligible. - **Total: about 5.159 GB per token.** So the bandwidth ceiling is 1,792 / 5.159 ≈ **347 tok/s.** **Why the reading sits well under the ceiling** [inferred, not measured]. At one stream, this architecture issues many small kernels per token: 30 expert matrices of 2,560 × 640 per layer, 36 Gated DeltaNet layers with a recurrent state update each, four BF16 hyper-connection matrices per layer, the sparse-attention indexer on 12 layers, and a host-side read of the n-gram rows before each token. Launch and synchronisation costs, not bandwidth, bind much of each token. The nearest anchor on this card is gpt-oss:120b (about 5B active) at 161 tok/s through ollama at 300 W (CB-5; PLAN-v2 Appendix A, R0-B2). Appendix A's own bet for Flash-Next on the desktop's larger card, R0-B2, is 60-120 tok/s at 300 W beside a live seat, with a 40 tok/s screen floor. **The bands, for tg128 at depth 0 (mean of the 3 repetitions):** | Band | tok/s | Share of the 347 ceiling | What it means | |---|---:|---:|---| | **Predicted** | **85-160** | 24-46 % | the prediction holds | | Outside the prediction | 40-85 or 160-347 | 12-24 % or 46-100 % | the row stands as measured; the prediction missed, and the miss is reported | | Under the floor | below 40 | below 12 % | the row stands, but no figure is quoted until the placement receipt is read (every layer on CUDA0, no expert on the CPU, the cap 420 W, the card the desktop's larger card) | | Impossible | above 347 | above the ceiling | **VOID**: it points at a wrong file, a wrong flag or a wrong reading, never at the card | **No band** is registered for prompt processing at any depth, or for generation at 32,768 or 131,072. Those readings are exploratory and are reported as read, with no claim attached. ## 5 · The stop rules 1. **No drill, no row.** The row runs only if the window's restore drill read GREEN first (D-098, step 2). 2. **The desktop's larger card must be near-empty at the start.** The window kit reads the desktop's larger card's used memory and its compute processes after the seats go down. Any tenant still on the desktop's larger card means no row. 3. **The binary and the files are checked first.** `llama-bench`'s sha256 equals `STAGE.md`'s, and both shards equal their pins. A mismatch means no row. 4. **The card is checked in the result.** The JSON's `gpu_info` must name the RTX PRO 6000 Blackwell Workstation Edition and one device only. Anything else (the 3090 Ti, two devices, or none) makes the row **VOID**, and it is not re-run tonight. 5. **The placement is checked in the log.** The stderr log must show every layer offloaded to the GPU and no expert buffer on the CPU. Anything else makes the row **VOID**. 6. **The hard runtime cap.** The row runs in a transient user unit whose `RuntimeMaxSec` ends it by **09:35:00Z**, whatever it is doing. A cut row keeps its finished tests, marked "cut at the cap". The expected run is about 10-20 minutes (one load of about 55 GB onto the card, two fills each at 32,768 and 131,072 tokens), so the cap only matters if something stalls. 7. **The UPS.** If the desktop's UPS reports OB, LB or FSD, or `battery.runtime` under 300 s on 3 consecutive 1-second samples, the window kit stops the row at once, and the restore runs (memory `desktop-ups-low-battery-sheds-live-seats`). 8. **Heat.** The desktop's larger card above 85 C stops the row (the guard's I-5 limit). 9. **A crash is a finding.** SIGILL, a CUDA error, an out-of-memory error or an abort ends the row. It is recorded with its log and is never re-run within the window to get a pass. 10. **One run.** No test is repeated to move a reading into a band. 11. **The restore runs regardless.** The row's end or death fires the restore (a watcher), the deadline timer at 09:45Z restores no matter what, and an operator's paste is the backup only (D-098). ## 6 · What the row does NOT claim - **It is not a registered Strata-bench row.** It is not under that PREREG's arms, bands or gates (G0, G1, G2, "Better"), and it is not the "Addendum R" the 10-03 NO-GO drafted. - **It is not the harness's measurement.** llama-bench's synthetic `pp512` and `tg128` are not the harness's SLICES.json prompts at 4K, 32K and 128K, not its client clock, and not its token-count check. There is no quality block. - **The laptop's harness rows stay the page's comparison.** This row uses a different card, the whole model on the card instead of `--n-cpu-moe`, and llama-bench instead of llama-server. It is not a like-for-like column beside them. - **It is not "beside the live seats".** The seats are down, so it says nothing about a window shaped like shape S (300 W, the docent stopped, gemma4 still on the desktop's larger card). - **It is not a Strata row, a COI row, or an energy or cost figure.** - **One box, one night, one build.** It speaks for llama.cpp 3cf03257 as built here, nothing newer. ## 7 · What follows **A Strata-on-the desktop's larger card row follows on a later night**, under its own registration, written before that row: Strata's own engine at its pinned commit, the same file, whole on the desktop's larger card where the engine allows it. This exploratory row measures llama.cpp's side first, so the later comparison has a same-card llama.cpp reference. ## 8 · What the row writes - `R-LB-IQ3.json` (llama-bench's JSON) and `R-LB-IQ3.stderr.log` (the load log and progress lines), with the unit's start and end `date -u` stamps and its exit status; - the pre-row receipt: the binary's and shards' sha256 read again, the desktop's larger card's used memory and compute processes, the cap read back, and which seats were down (the desktop's smaller card's seat up or down); - the post-row receipt from the restore (the drill's receipt shape: units, timers, warm seats, the assistant's `/health`, both Ollama instances' residents, both caps 420 and 350 W); - after the row, the IQ3_S files are removed (LA-2's window-scoped exception, D-097), and the free space is read back. --- ## 9 · Confirmation after the build (appended 2026-10-05T00:37:31Z, still before any reading) Appended under the registration above. It changes no prediction, band, stop rule or command flag. It fills in what was unknown at 00:03:53Z and tightens one check. - **The engine exists as registered.** `STAGE.md` §4 has the facts: - llama.cpp 3cf03257, built natively on the desktop, Release, `GGML_NATIVE=ON`, sm_120a only, `GGML_VULKAN=OFF`; - objdump finds 0 zmm, 0 VNNI and 0 AVX-512 mask instructions in every library; - `llama-bench --help` from the built binary lists `-d, --n-depth`, `-fa `, `-lm`, `-lzm`, `-ncmoe`, `-sm ` and `-o <…json…>`, so §3's command parses at this commit. - **One deviation in how it was built:** CUDA **13.4** (V13.4.92, the laptop engine's own toolkit), staged in user space at `~/bench-flashnext/cuda-13.4`, compiled the engine instead of the desktop's system CUDA 13.1. - The reason: 13.1's headers do not compile against the desktop's glibc 2.43. - The engine's RUNPATH loads the 13.4 runtime libraries from that folder, on the desktop's driver 595.84 (CUDA 13.2), through CUDA's minor-version compatibility. The row's own load is the first proof of that pairing on the desktop's larger card. - If the load fails on it, that is stop rule 9's "a crash is a finding", not a band miss. - **Stop rule 3, tightened:** `llama-bench` is a 17,872 B launcher; its code lives in `libllama-bench-impl.so`. So "the binary is checked" means **every file in `STAGE.md` §4's binaries table that the row loads**: `llama-bench`, `libllama-bench-impl.so`, `libllama.so.0.4.1`, `libllama-common.so.0.4.1`, `libggml.so.0.24.0`, `libggml-base.so.0.24.0`, `libggml-cpu.so.0.24.0` and `libggml-cuda.so.0.24.0`. Each must read its listed sha256. The `llama-bench` sha256 to match is `1961b21521a998ae332463ea784e4e1b239cfaf919e58322ff6445e1af8a7cca`, the relinked binary's; the v2 binary's `e0fdc4b7…` is superseded. - **The shards on the desktop** read their pins (`STAGE.md` §3): `4c1eb2ce…` and `316b46f3…`, at 00:18:51Z and 00:25:26Z. ## 10 · Amendments before any reading (appended 2026-10-05T03:01:33Z) 1. **The cap.** The row's condition changes from 420 W to **300 W** (D-20260927-008's posture; PLAN-v2 §3 D-7; the UPS arithmetic in PLAN-v2 §2.1). The band of §4 is unchanged: it is bandwidth arithmetic for decode. The 420 W reading is a later posture row, `R-LB-IQ3-420`, under PREREG-BANDS' cross-night rule. 2. **Stop rule 5, the test.** The stderr log must read offloaded 49/49; `CUDA0 model buffer size` within ± 2 % of 52,844 MiB (18,870.50 + 34 × 999.22, from the laptop's L-IQ3-t8 `server.log:182-184` and 50,292,326,400 B ÷ 48); `CPU_Mapped model buffer size` ≤ 2,400 MiB (35,835.49 − 34 × 999.22 = 1,862 MiB). Otherwise VOID. If the line names differ on the desktop, the lead reads them and the row prints "placement unread", never "whole". 3. **The attempt rule.** An a1 with a finished tg128@d0 test is never re-run, whatever its band and whatever ended the row later; the finished tests stand, labelled with the end reason. a2 runs only if a1 has no finished tg128@d0 (VOID, a crash, or a cut before it); a2 is then the figure of record whatever its direction, a1 printed beside it. If a1 crashed in pp512@d0, the follow-up is R-LB-IQ3-tg (the same argv with `-p 0`), its band this row's; a new label, not a2. A new build sha is a new label, never an attempt. 4. **Out of band.** An outside-band reading prints `OUT-OF-BAND (opened: no)` (the laptop's A1-4), and nothing quotes it until the lead has read the engine log, the rows and the box's state (`gpu-1hz.csv`, `ups.csv`, `row-meta.txt`, the desktop's smaller card's busy share per test). 5. **§7, corrected.** R-LB-IQ3 is not the Strata rows' reference: it is a llama-bench reading (synthetic tokens, no template, 128 tokens, depth 0, the engine's clock, the mean). The same-instrument reference is R-LS-IQ3 (llama-server 3cf03257 with the harness's client), registered in PREREG-BANDS. Until R-LS-IQ3 reads, no sentence compares Strata with llama.cpp on the desktop. 6. **R-LB-IQ3-n2** is the same argv on a later night, a registered noise read for the cross-night rule, never an attempt and never a replacement for a1's figure. 7. **The stamps.** §5 rule 6's hard end moves from 09:35:00Z to **09:10:00Z**, and rule 11's deadline restore from 09:45Z to **09:20:00Z** (PLAN-v2 Appendix A.1, fold S-I-2; `kit/window.conf`): the restore's own step timeouts sum to about 2,390 s, so a deadline restore must start at least 2,400 s before 10:00Z to end inside the window. The row is expected to take 10-20 min from about 07:45Z, so the earlier hard end costs no reading. 8. **The files after the row.** §8's last line ("after the row, the IQ3_S files are removed") is replaced by LA-2b, ruled keep in D-20261005-105 (`kit/row.conf SHARDS_AFTER_ROW=keep`): the IQ3_S pair, the CUDA 13.4 subset and the build tree stay on the desktop for the week's rows that need them, and are removed by their exact paths after the last of those rows, at the latest Sat 2026-10-10. The restore's R10 still reads the bench store's free GiB against the 400 GiB floor, and tonight's receipt prints `UNDER` while the files stay.