# Arm 3's substrate, measured — inferencebox, 2026-09-21 *What this file is, for a reader who scrolled straight here: the receipts behind the PREPARATION for arm 3 of the Jev bench (README §Arm 3) — the vLLM install on inferencebox, the two checkpoints' revisions and sizes, the live seat map read off the box, and the two things inferencebox turned out to have that benchbox did not. Every line is output from the box, not a recollection. Times are UTC, 2026-09-21. **No GPU work was run: arm 3 has not started.*** --- ## 1 — The install | | | |---|---| | started / finished | **19:15:36Z → 19:15:47Z (11 s)**, against a 45-minute refusal ceiling | | venv | `/workshop/bench-vllm-venv`, **7.7 GB**, built by `uv 0.12.7` | | interpreter | **CPython 3.12.14** (uv-managed, already on the box) — **not** inferencebox's system 3.14.4 | | command | `uv venv --python 3.12 ~/bench-vllm-venv` then `uv pip install "vllm==0.29.0"` | | log | `~/bench-arm3/logs/install-vllm.log` on inferencebox | | why 11 s and not benchbox's 4 m 42 s | `Resolved 196 packages in 639ms · Prepared 56 packages in 9.49s · Installed 196 packages in 423ms` — inferencebox's uv cache is **27 GB** warm, so 140 of the 196 packages never left the disk | **The seat's venv was not touched and is not touched by anything here.** `~/vllm-env/bin/python -c "import vllm"` still prints **0.27.1** — that venv serves `docent-vllm.service`, the estate's 24/7 assistant seat, and arm 3 runs entirely out of the new one. ### Versions, as installed — the same pin benchbox's arm 2 ran | package | inferencebox (arm 3) | benchbox (arm 2) | |---|---|---| | vllm | **0.29.0** | 0.29.0 | | torch | **2.13.0+cu130** | 2.13.0+cu130 | | NCCL (in torch) | 2.29.7 | 2.29.7 | | triton | 3.7.1 | 3.7.1 | | transformers | 5.17.0 | 5.17.0 | | flashinfer-python | 0.6.18 | 0.6.18 | | huggingface-hub | 1.32.0 | 1.32.0 | | numpy | 2.3.5 | 2.3.5 | ### The driver check, done before the install was trusted ``` $ nvidia-smi --query-gpu=driver_version --format=csv,noheader 595.84 $ nvidia-smi | head -4 | NVIDIA-SMI 595.84 Driver Version: 595.84 CUDA Version: 13.2 | ``` The wheel is built against **CUDA 13.0**; the driver supports **13.2**. A CUDA runtime is forward-compatible within a major version against a driver at least as new, so **13.0 on a 13.2 driver is in bounds** — checked, not assumed, because the reverse is the ordinary way this fails. ``` $ ~/bench-vllm-venv/bin/python -c "import vllm, torch; ..." vllm 0.29.0 torch 2.13.0+cu130 torch.version.cuda 13.0 cuda avail True nccl (2, 29, 7) 0 NVIDIA RTX PRO 6000 Blackwell Workstation Edition (12, 0) 97250 MiB 1 NVIDIA GeForce RTX 3090 (8, 6) 24124 MiB ``` **`(12, 0)` is the whole reason arm 3 exists**: sm_120 has native FP8 arithmetic, and arm 2's sm86 did not (it fell to Marlin weight-only, §R2.2 / §A3.4). **One warning worth carrying into the window**, printed by vLLM on import because the two boards differ: *"Detected different devices in the system … Please make sure to set `CUDA_DEVICE_ORDER=PCI_BUS_ID`."* Arm 3 pins by **UUID** rather than by index, so ordering cannot pick the wrong board — and `CUDA_DEVICE_ORDER=PCI_BUS_ID` is exported anyway in `arm3_window.sh`, because a belt that costs nothing is not worth arguing about. --- ## 2 — Two things inferencebox HAS that benchbox did not Arm 2 spent real time closing two gaps on benchbox (README §R2.8): no C compiler at all, and no CUDA toolkit. **Neither exists here**, and the absence of the workaround is itself a receipt: ``` $ command -v gcc cc -> /usr/bin/gcc, /usr/bin/cc $ gcc --version | head -1 -> gcc (Ubuntu 15.2.0-16ubuntu1) 15.2.0 $ /usr/local/cuda/bin/nvcc --version | tail -2 Cuda compilation tools, release 13.1, V13.1.115 ``` So triton's per-kernel C launcher stub compiles with the system gcc and benchbox's `ziglang` shim is **not used**, and flashinfer's JIT sampler *could* build. **`VLLM_USE_FLASHINFER_SAMPLER=0` is set anyway and deliberately** (README §A3.3.5): it keeps the sampling path byte-identical to arm 2, and a cold flashinfer JIT inside a 45-minute window is minutes this arm does not have. It cannot move a number — every arm is greedy at temperature 0 and the readout reads raw logprobs. --- ## 3 — The two checkpoints Downloaded **19:16:38Z → 19:27:19Z**, `hf download` pinned to each revision, both exit 0, into `HF_HOME=~/hf-cache`. Sizes are `du -Lsb` on the snapshot, and they match the hub's own file-metadata totals to the byte. | | BF16 arm | FP8 arm | |---|---|---| | repository | `openjev/openjev` | `openjev/openjev-FP8` | | **revision** | **`5ec9e5fd2f80a6fff386779b1e5ac7e389971889`** | **`4ec320f267401e67c9be04d5df1be4d2b6b64f10`** | | bytes on disk | **54,734,528,013** (54.73 GB) | **30,409,874,987** (30.41 GB) | | files / shards | 28 / **12** | 21 / **12** | | `config.json` | `Qwen3_5ForConditionalGeneration`, `qwen3_5`, 64 layers, hidden 5,120, **dtype bfloat16, no `quantization_config`** | the same, plus **`quant_method: fp8`, `weight_block_size: [128, 128]`** | | snapshot | `~/hf-cache/hub/models--openjev--openjev/snapshots/5ec9e5fd…` | `~/hf-cache/hub/models--openjev--openjev-FP8/snapshots/4ec320f2…` | **The FP8 revision is the one arm 2 measured** (`4ec320f2…`), so the FP8 arm is the same bytes on a different card rather than a different checkpoint. **The tokenizer is byte-identical across the two repositories** — `sha256(tokenizer.json)` begins `06b9509352d2af50` on both, 19,989,325 bytes — so the two arms tokenize every item the same way and the 16,384-token coverage table (53 / 35 / 108) is one table, not two. **Disk:** inferencebox was at 917 GB free on `/` before the preparation and **838 GB free** after the 85.14 GB of weights and the 7.7 GB venv. Nothing was deleted. **LICENCE, restated where it will be read:** both repositories are **CC BY-NC 4.0 — research and non-commercial only. BENCH-ONLY. These weights never become a seat.** --- ## 4 — The harness, installed and sha-verified on both boxes ``` $ sha256sum run.py (laptop and inferencebox, both) 372c9ae632ffbc18299f14b290bbda5f0fe4ec458cbbbee89e4838245c86b7be $ (cd kit && sha256sum *.json | sort) | sha256sum (laptop and inferencebox, both) 7c1f2c5a83f92f52b854ab0d24f3ae40f3a22a13b9d7216e8bfe75a1eec79b97 $ ~/bench-vllm-venv/bin/python run.py --help | grep -o "openjev-[a-z0-9-]*largecard" openjev-bf16-largecard openjev-fp8-largecard ``` **Arm 1's and arm 2's twelve summaries re-score byte-for-byte under the extended harness** — `run.py --score rows/` run from `HEAD`'s copy and from the arm-3 copy produced identical output, checked after the patch and not asserted. Three literals became arm facts, because a number that is right on one box is a lie on the next: `Arm.box` (the announcement said `benchbox` unconditionally), `Arm.power_cap_w` (every row said `250`; **largecard caps at 420 W**), and a per-arm `EXPECT_MINUTES_BY_ARM` so the announcement's "expect ~N min" is arm 3's estimate rather than benchbox's. --- ## 5 — The live seat map, read off the box (read-only) `nvidia-smi --query-compute-apps` joined to `systemctl cat`, 2026-09-21T19:2xZ, with nothing stopped, started or changed: | pid | MiB | board | unit | port | what it carries | in the window | |---|---|---|---|---|---|---| | 1989604 | **45,620** | largecard | `docent-vllm.service` | 8001 | `gemma-4-26b-a4b-fp8`, 131,072 window — the hub's assistant **and** the long table's GEMMA_CHAIRS / DOORMAN / CHECKS / MC | **STOP** | | 3637998 | **18,846** | largecard | `ollama.service` | its default port | `gemma4:26b` Q4_K_M — rulesage's answerer, amble's guide | **STOP** | | 3634815 | **996** | largecard | `ollama.service` | its default port | `nomic-embed-text` F16 | **STOP** | | 3430131 | **954** | largecard | `rerank-seat-7997.service` (**user**) | 7997 | MiniLM cross-encoder, fp32 CUDA | **STAYS** | | 2965595 | 17,464 | the 3090 board | `ollama-11435.service` | 11435 | `mistral-small3.2:24b` — rulesage's classifier, the beat lab's genie-mod, the long table's MISTRAL_CHAIRS | **STAYS** | | 2967293 | 608 | the 3090 board | `ollama-11435.service` | 11435 | `nomic-embed-text` F16 | **STAYS** | Boards: largecard `GPU-25bc3288-319a-790c-a14e-82acdb8d15b8`, **66,444 / 97,887 MiB**, cap **420 W**. the 3090 board `GPU-fde82ed0-be70-eb1f-2d4d-5538c5b82166`, 18,087 / 24,576 MiB, cap 300 W. **Two units that carry no model and still decide the window:** * **`inferencebox-warm-seats.timer`** fires every 5 minutes and its service carries **`Wants=ollama.service`** — so a tick at 04:05Z would **start the seat underneath the bench**. Its own description says *"stop during bench windows"*. **It is stopped first and started last.** * **`estate-inference-fence.service`** is a `oneshot` nft table keeping the inference ports private-network-only. It touches no board and **stays up**; dropping a fence for a bench is not a trade. **The start order is forced, and this is the single most useful thing in this file.** `docent-vllm.service` carries `ExecStartPre=/usr/local/bin/docent-wait-for-residents`, which polls `/api/ps` on **both** its default port and :11435 and **blocks up to 600 s** until `gemma4:26b` and `mistral-small3.2:24b` are resident. So the way back is: **ollama → re-pin → the seat**, never the reverse. Its own unit header records the boot that follows: **`Application startup complete` 4 m 09 s after launch** (engine init 140.37 s, of which 77.21 s compile; model load 7.7 s), measured 2026-09-08. **`journalctl -u docent-vllm.service` as `an operator` shows only systemd's own lines** — the seat's stdout is not readable without root on this box, which is why every journal paste in the runbook is a `sudo` paste and says so.