# The box, before the first measured row — benchbox, 2026-09-21

Every reading below was taken on benchbox between 16:40Z and 16:56Z, before the driver
started. It is here so a reader can check the bench's claims about its own substrate
without re-running anything.

## The pin — one CUDA device, no Vulkan device (D-20260920-133)

The bench unit's own discovery log, after the store merge and restart (16:54:30Z):

```
WARN  msg="user overrode visible devices" CUDA_VISIBLE_DEVICES=GPU-46890836-d8f7-e868-1977-c1ce28db0d7a
WARN  msg="user overrode visible devices" GGML_VK_VISIBLE_DEVICES=-1
INFO  msg="inference compute" id=0 filter_id=GPU-46890836-d8f7-e868-1977-c1ce28db0d7a
      library=CUDA compute=8.6 name=CUDA0 description="NVIDIA GeForce RTX 3090"
      pci_id=0000:04:00.0 type=discrete total="23.6 GiB" available="23.3 GiB"
```

One device, and it is the named one. No Vulkan line — which is the whole point of the
memory this pin comes from: ollama enumerates the *other* board through its Vulkan backend
and the scheduler takes the biggest one, so an instance whose CUDA filter is perfect can
still run on its neighbour.

The unit, as started (transient, user scope, no sudo):

```
systemd-run --user --unit=ollama-bench-c0 \
  -E OLLAMA_HOST=127.0.0.1:11436 \
  -E OLLAMA_MODELS=/workshop/bench-store/ollama \
  -E CUDA_VISIBLE_DEVICES=GPU-46890836-d8f7-e868-1977-c1ce28db0d7a \
  -E OLLAMA_VULKAN=0 -E GGML_VK_VISIBLE_DEVICES=-1 \
  -E HIP_VISIBLE_DEVICES= -E ROCR_VISIBLE_DEVICES= \
  -E OLLAMA_KEEP_ALIVE=-1 -E OLLAMA_FLASH_ATTENTION=1 \
  -E OLLAMA_CONTEXT_LENGTH=131072 \
  /usr/local/bin/ollama serve
```

## Residency, proved by growth on the named board

| moment | card 0 `…46890836` | card 1 `…7aa0be10` |
|---|---|---|
| idle, nothing loaded (16:44Z) | 1 MiB, 22.78 W | 1 MiB, 17.81 W |
| adapter loaded, first call (16:45Z) | **18,020 MiB**, 105.12 W | **1 MiB**, 18.42 W |
| evicted again before the driver (16:56Z) | 1 MiB, 26.39 W | 1 MiB, 18.06 W |

Refusing in both directions: the named board grew, and no other board did.

## The context measurement that set `num_ctx`

The docent's own bank does not fit the unit's 8,192. Measured, page by page, by the
server's own `prompt_eval_count`:

| num_ctx | card 0 after load | card 1 | verdict |
|---|---|---|---|
| 16,384 | 18,178 MiB | 1 MiB | fits, covers 37 of 42 pages |
| 32,768 | 18,530 MiB | 1 MiB | fits, covers 39 of 42 |
| 65,536 | 18,970 MiB | 1 MiB | fits |
| 81,920 | 19,306 MiB | 1 MiB | fits — **and still refuses the longest page** |
| 98,304 | 19,642 MiB | 1 MiB | fits |
| **131,072** | **20,314 MiB** | **1 MiB** | **chosen** |

The refusal at 81,920 is the server's own sentence, and it is the reason the number moved:

```
{"error":{"code":400,"type":"exceed_context_size_error",
  "message":"request (82488 tokens) exceeds the available context size (81920 tokens)",
  "n_prompt_tokens":82488,"n_ctx":81920}}
```

`three-new-voices-at-the-narrators-chair` is 82,488 tokens as the docent fences it. At
131,072 the model is still whole on one card at 20,314 MiB of 24,576, with card 1 at 1 MiB.

## The trap of the day — the first output token is not a letter

The adapter ships the Gemma 4 canonical chat template, whose assistant turn opens on a
thinking channel. Asked the OpenJev way with nothing else set, position 0 is `<|channel>`
at logprob −0.0000026, and the option letters sit far below it:

```
<|channel>  -0.0000026      C  -13.455      D  -14.207      A  -15.091      B  -18.091
```

The template's own tail closes the channel inside the generation prompt when thinking is off:

```jinja
{{- '<|turn>model\n' -}}
{%- if not enable_thinking -%}{{- '<|channel>thought\n<channel|>' -}}{%- endif -%}
```

ollama's `think: false` sends exactly that. The same question, one flag later:

```
A  -0.2858      [  -1.3950      B  -7.3127      C  -10.479      D  -11.429
```

Position 0 is now the answer. Every arm in this bench sets it, the generating controls
included, so the readout arms and the generating arms differ in the readout and in nothing
else. `chat_template_kwargs` is *ignored* by ollama (measured: the channel token came back);
`reasoning_effort: "none"` on `/v1` works and is equivalent.

## The two models, and one asymmetry that must be stated

| | adapter | base |
|---|---|---|
| tag | `hf.co/mradermacher/jevify-gemma4-26b-a4b-GGUF:Q4_K_M` | `gemma4:26b` |
| ollama id | `6c304955191f` | `08ae7ec1744b` |
| params / quant | 25.2B, **Q4_K_M** | 25.2B, **Q4_K_M** (`file_type` in its own config blob) |
| model layer | 16.8 GB | 16.9 GB |
| vision projector | none | `mmproj-BF16.gguf`, 1.19 GB |
| **draft model** | **none** | **`gemma4-assistant` (MTP), 462 MB, `draft_num_predict: 3`** |

Same parameter count, same quantisation class — the comparison the gate is written on is
like-for-like. **The asymmetry is the draft model**: the base tag ships a speculative-decoding
draft and the adapter's GGUF does not. It cannot touch a readout (one token, one forward
pass) but it can flatter the base's *generating* arm, so `base-generate`'s latency is read
as a floor for the base and not as a like-for-like against `jev-generate`.

The base was already on the box in `~/bench-store/ollama` (the coordinator's note, 16:53Z);
the adapter's blobs and manifest were moved into that same store rather than a second 17 GB
being pulled onto a disk at 97 %. Same filesystem, content-addressed, `mv`, instant.