# The Jev bench — the three gaps, 2026-09-22

*Drawn by `tables_gaps.py` from row files written on 2026-09-21 plus one cross-encoder run on a laptop CPU. No GPU second was spent and no box in the estate was called. Every pre-registration these tables answer is in `GAPS.md`, committed before its first number existed.*

## GAP 1 — one decision, shown whole

Both decisions below are the same six-way question — *which of the six guests said this line?* — put to two models in exactly the same bytes: the rebuilt prompt hashes to the sha256 both rows carry. The two ran on **different hardware and different serving postures** (the decision model whole on one large card; the chat model split across two consumer cards), so nothing about *time* may be read across these two columns — only the answers and the letter scores, which is all this section uses them for. The like-for-like control exists and is quoted under each table. The lines are quoted from the long table's public wall, whose address the frozen kit records as `https://longtable.strata2signal.com/api/wall`; each line carries the `(guest, edition, course, line index)` tuple that wall entry gave it. That tuple is the one the bench later found is **not unique** across dinners (README R6), so the row's position in the file is quoted beside it as the address that is.

**The arithmetic, reproduced from the stored scores.** Every probability in this section falls out of the six raw letter scores and two published constants — a temperature of **0.85** and a floor of **−30** for a letter that never appeared — by `run.py::decide`'s readout branch:

```python
z = [(FLOOR if v is None else v) / READOUT_T for v in raw]   # FLOOR = -30.0, READOUT_T = 0.85
p = softmax(z)
```

Checked against all 216 stored rows of the two arms, the largest disagreement between the reproduced probability and the one written down on the night is **0.0e+00** — which is to say the rows are auditable with a calculator and nothing here is taken on trust.

### The one the decision model got right — row 14 of 108

*The line: the long table's public wall, edition 1, starter course, line 1 (kit row 14 of 108). True speaker: **Charles Darwin**. The speaker's name is never in the prompt, and any line naming its own speaker was dropped from the set before sampling.*

**The prompt, exactly as sent** — one user turn, no system turn; the bench sends `messages=[{"role": "user", …}]` and nothing else (`run.py::call`). Its sha256 is `ef694d81223f69efcb6bd3ffa1a36a8d80da830bea500d5ac814dc611d85d860`, the same sha both arms' rows carry.

```
State:
A dinner table with six guests: Charles Darwin, Albert Einstein, Hypatia of Alexandria, Ibn Sina, Carl Sagan, Socrates of Athens.

Question: Six people are at a dinner table. Which of them said the line below?
Line: Host, you pose a most profound question, one that strikes at the very heart of how we perceive the world. I should think a child possesses a most remarkable, unclouded receptivity to the natural world, observing the minute movements of a beetle or the structure of a leaf without the heavy shroud of preconceived theories. As we age, we often become so encumbered by our own deductions and the rigid structures of our learning that we lose that primal, instinctive wonder which is so essential to true observation.
Options:
[A] darwin: Charles Darwin
[B] einstein: Albert Einstein
[C] hypatia: Hypatia of Alexandria
[D] ibn_sina: Ibn Sina
[E] sagan: Carl Sagan
[F] socrates: Socrates of Athens

Answer with the letter of the best option only.
```

| | the decision model, 8-bit, one card | the chat model, 8-bit, two cards |
|---|---|---|
| first token at position 0 | `A` | `[` |
| option letters found in the top 20 | **6 of 6** | **5 of 6** |
| chose | **Charles Darwin** | **Carl Sagan** |
| probability it gave that choice | **0.7240** | **0.8288** |
| correct? | **yes** | **no** |
| seconds *(different cards — not comparable, and not compared)* | 0.085 | 0.109 |

| letter | guest | raw score, decision model | p | raw score, chat model | p |
|---|---|---|---|---|---|
| A | Charles Darwin ← | -0.4224 | 0.7240 | -9.2536 | 0.1058 |
| B | Albert Einstein | -1.6724 | 0.1664 | -9.7536 | 0.0587 |
| C | Hypatia of Alexandria | -5.2349 | 0.0025 | -11.8786 | 0.0048 |
| D | Ibn Sina | -3.7974 | 0.0137 | -12.6911 | 0.0019 |
| E | Carl Sagan | -3.7974 | 0.0137 | -7.5036 | 0.8288 |
| F | Socrates of Athens | -2.2974 | 0.0798 | **−30, floored** | 0.0000 |

*← marks the true speaker. A **floored** letter is one that never appeared in the server's raw top-20 at the first output position, so the protocol gives it −30 and the softmax gives it, in effect, zero. A floored letter is not a low opinion; it is an absent one.*

*The like-for-like control: the decision model also ran on the chat model's own server and cards, and on this row it chose **Charles Darwin** at **0.7053** — the same answer. Across all 108 items the two postures score identically (93.5 % each), so the difference between the two columns above is the model and not the machine.*

### The one the decision model got wrong — row 7 of 108

*The line: the long table's public wall, edition 1, main course, line 2 (kit row 7 of 108). True speaker: **Charles Darwin**. The speaker's name is never in the prompt, and any line naming its own speaker was dropped from the set before sampling.*

**The prompt, exactly as sent** — one user turn, no system turn; the bench sends `messages=[{"role": "user", …}]` and nothing else (`run.py::call`). Its sha256 is `e3b79e40f9996ce2bd4d674814cbfc0013d3e6bbf022b549d661259700672136`, the same sha both arms' rows carry.

```
State:
A dinner table with six guests: Charles Darwin, Albert Einstein, Hypatia of Alexandria, Ibn Sina, Carl Sagan, Socrates of Athens.

Question: Six people are at a dinner table. Which of them said the line below?
Line: Socrates, Hypatia raises a most pertinent point. I have observed in my travels that the conditions of a man's life can greatly influence his capacity for contemplation and growth. A man who is preoccupied with the securing of his next meal or the safety of his family may have little time or energy to devote to the pursuit of knowledge or the cultivation of his soul. It is for this reason that I have argued that the progress of the mind and the prosperity of the body are not entirely distinct, but rather closely intertwined.
Options:
[A] darwin: Charles Darwin
[B] einstein: Albert Einstein
[C] hypatia: Hypatia of Alexandria
[D] ibn_sina: Ibn Sina
[E] sagan: Carl Sagan
[F] socrates: Socrates of Athens

Answer with the letter of the best option only.
```

| | the decision model, 8-bit, one card | the chat model, 8-bit, two cards |
|---|---|---|
| first token at position 0 | `D` | `[` |
| option letters found in the top 20 | **6 of 6** | **5 of 6** |
| chose | **Ibn Sina** | **Ibn Sina** |
| probability it gave that choice | **0.8963** | **0.9123** |
| correct? | **no** | **no** |
| seconds *(different cards — not comparable, and not compared)* | 0.086 | 0.116 |

| letter | guest | raw score, decision model | p | raw score, chat model | p |
|---|---|---|---|---|---|
| A | Charles Darwin ← | -3.2970 | 0.0227 | -9.0959 | 0.0199 |
| B | Albert Einstein | -5.5470 | 0.0016 | -9.5334 | 0.0119 |
| C | Hypatia of Alexandria | -4.6720 | 0.0045 | -9.5959 | 0.0111 |
| D | Ibn Sina | -0.1720 | 0.8963 | -5.8459 | 0.9123 |
| E | Carl Sagan | -5.6720 | 0.0014 | -8.4084 | 0.0448 |
| F | Socrates of Athens | -2.2970 | 0.0736 | **−30, floored** | 0.0000 |

*← marks the true speaker. A **floored** letter is one that never appeared in the server's raw top-20 at the first output position, so the protocol gives it −30 and the softmax gives it, in effect, zero. A floored letter is not a low opinion; it is an absent one.*

*The like-for-like control: the decision model also ran on the chat model's own server and cards, and on this row it chose **Ibn Sina** at **0.8770** — the same answer. Across all 108 items the two postures score identically (93.5 % each), so the difference between the two columns above is the model and not the machine.*

## GAP 2 — the temperature refit

*Every number here comes out of `letter_logprobs`, a field every readout row kept on the night precisely so this could be done later for nothing. No model was called.*

**The published temperature on task (a) is not 0.85.** The yes/no readout applies `softmax(raw / 0.85)` and then pushes `logit(p_yes)` through `NOUL_T = 1.829074`. For two options those compose exactly, so the published calibration on (a) is a single temperature of **0.85 × 1.829074 = 1.554713** — checked against every stored row, max error **1.8e-15**. A refit on (a) is read against that, not against 0.85.

### Task (c) — which of the six guests said this line?

*Variant **A** is the floor as it ran: a letter the server never surfaced enters at −30, which is what the deployed protocol does. Variant **B** drops those letters and renormalises over the ones that did surface — a diagnostic, not something a caller could run, because a caller cannot know in advance which letters will appear. **A|B** is variant A read on variant B's smaller denominator, so the two can be compared without changing two things at once.*

Split: seed 20260921, stratified 50/50 — **54 items fit, 54 held**, and it is the same items for every arm — the split is a function of the label sequence alone, and all six arms scored all 108. Published temperature on this task: **0.850000**.

| arm | floored rows (all items) | fitted T, variant A | held acc | Brier before → after | rank-blind Brier at this accuracy | ECE before → after | NLL before → after |
|---|---|---|---|---|---|---|---|
| `jev-readout` | 0 of 108 | **0.711** | 88.9 % | 0.2458 → 0.2316 | 0.2074 | 0.1459 → 0.1035 | 0.5389 → 0.5174 |
| `base-readout` | 101 of 108 | **5.301** | 81.5 % | 0.3457 → 0.2484 | 0.3292 | 0.1769 → 0.1016 | 1.6351 → 0.5324 |
| `openjev-fp8-readout` | 0 of 108 | **1.152** | 94.4 % | 0.1144 → 0.1166 | 0.1074 | 0.0451 → 0.0316 | 0.2626 → 0.2534 |
| `openjev-bf16-largecard` | 0 of 108 | **1.168** | 94.4 % | 0.1132 → 0.1152 | 0.1074 | 0.0466 → 0.0393 | 0.2664 → 0.2547 |
| `openjev-fp8-largecard` | 0 of 108 | **1.145** | 94.4 % | 0.1211 → 0.1228 | 0.1074 | 0.0622 → 0.0459 | 0.2687 → 0.2600 |
| `gemma4-fp8-readout` | 103 of 108 | **5.433** | 85.2 % | 0.2754 → 0.2375 | 0.2700 | 0.1408 → 0.0851 | 1.2989 → 0.5097 |

*The **rank-blind Brier** column is what a perfectly calibrated predictor at that arm's own accuracy would score if it had no opinion about which of the losing options was the runner-up. It is an anchor, not a floor — an arm that ranks the losers well can beat it — and its job is to put an accuracy difference on the Brier scale, so the part of a Brier gap that is accuracy rather than calibration is visible.*


**The floored letters, taken out of the fit — variant B, beside variant A on B's own denominator.**

| arm | rows B must drop (held) | A\|B: Brier before → after | A\|B: ECE before → after | fitted T, variant B | B: Brier before → after | B: ECE before → after |
|---|---|---|---|---|---|---|
| `jev-readout` | 0 of 54 | 0.2458 → 0.2316 | 0.1459 → 0.1035 | **0.711** | 0.2458 → 0.2316 | 0.1459 → 0.1035 |
| `base-readout` | 2 of 54 | 0.2820 → 0.1884 | 0.1452 → 0.1096 | **3.690** | 0.2820 → 0.1805 | 0.1452 → 0.0701 |
| `openjev-fp8-readout` | 0 of 54 | 0.1144 → 0.1166 | 0.0451 → 0.0316 | **1.152** | 0.1144 → 0.1166 | 0.0451 → 0.0316 |
| `openjev-bf16-largecard` | 0 of 54 | 0.1132 → 0.1152 | 0.0466 → 0.0393 | **1.168** | 0.1132 → 0.1152 | 0.0466 → 0.0393 |
| `openjev-fp8-largecard` | 0 of 54 | 0.1211 → 0.1228 | 0.0622 → 0.0459 | **1.145** | 0.1211 → 0.1228 | 0.0622 → 0.0459 |
| `gemma4-fp8-readout` | 2 of 54 | 0.2090 → 0.1803 | 0.1077 → 0.1221 | **3.011** | 0.2090 → 0.1537 | 0.1077 → 0.0803 |

*Variant B's Brier is summed over a **reduced** option set and is not the same quantity as variant A's — it is read down its own column, never across.*

### Task (a) — does the article answer this question?

*Variant **A** is the floor as it ran: a letter the server never surfaced enters at −30, which is what the deployed protocol does. Variant **B** drops those letters and renormalises over the ones that did surface — a diagnostic, not something a caller could run, because a caller cannot know in advance which letters will appear. **A|B** is variant A read on variant B's smaller denominator, so the two can be compared without changing two things at once.*

Split: seed 20260921, stratified 50/50 — **26 fit / 27 held** for `openjev-fp8-readout`, `openjev-bf16-largecard`, `openjev-fp8-largecard`, `gemma4-fp8-readout` and **31 fit / 32 held** for `jev-readout`, `base-readout`. **The split is NOT the same items across those two groups**: the arms served on a 16,384-token window refused the same ten overlong pages, so they score 53 items where arm 1 scores 63, and a stratified half of 53 is not a subset of a stratified half of 63. Within each group it IS the same items, and a cross-group comparison on this task is read as two sets, not one. Published temperature on this task: **1.554713**.

| arm | floored rows (all items) | fitted T, variant A | held acc | Brier before → after | rank-blind Brier at this accuracy | ECE before → after | NLL before → after |
|---|---|---|---|---|---|---|---|
| `jev-readout` | 5 of 63 | **degenerate** (ran to 0.06) | 100.0 % | 0.0038 → 0.0000 | 0.0000 | 0.0232 → 0.0000 | 0.0243 → 0.0000 |
| `base-readout` | 19 of 63 | **degenerate** (ran to 0.27) | 100.0 % | 0.0000 → 0.0000 | 0.0000 | 0.0003 → 0.0000 | 0.0003 → 0.0000 |
| `openjev-fp8-readout` | 0 of 53 | **degenerate** (ran to 0.09) | 100.0 % | 0.0054 → 0.0000 | 0.0000 | 0.0445 → 0.0000 | 0.0459 → 0.0000 |
| `openjev-bf16-largecard` | 0 of 53 | **degenerate** (ran to 0.09) | 100.0 % | 0.0050 → 0.0000 | 0.0000 | 0.0434 → 0.0000 | 0.0447 → 0.0000 |
| `openjev-fp8-largecard` | 0 of 53 | **degenerate** (ran to 0.09) | 100.0 % | 0.0054 → 0.0000 | 0.0000 | 0.0445 → 0.0000 | 0.0459 → 0.0000 |
| `gemma4-fp8-readout` | 16 of 53 | **degenerate** (ran to 0.14) | 100.0 % | 0.0000 → 0.0000 | 0.0000 | 0.0002 → 0.0000 | 0.0002 → 0.0000 |

*The **rank-blind Brier** column is what a perfectly calibrated predictor at that arm's own accuracy would score if it had no opinion about which of the losing options was the runner-up. It is an anchor, not a floor — an arm that ranks the losers well can beat it — and its job is to put an accuracy difference on the Brier scale, so the part of a Brier gap that is accuracy rather than calibration is visible.*


**The floored letters, taken out of the fit — variant B, beside variant A on B's own denominator.**

| arm | rows B must drop (held) | A\|B: Brier before → after | A\|B: ECE before → after | fitted T, variant B | B: Brier before → after | B: ECE before → after |
|---|---|---|---|---|---|---|
| `jev-readout` | 0 of 32 | 0.0038 → 0.0000 | 0.0232 → 0.0000 | **degenerate** (ran to 0.06) | 0.0038 → 0.0000 | 0.0232 → 0.0000 |
| `base-readout` | 0 of 32 | 0.0000 → 0.0000 | 0.0003 → 0.0000 | **degenerate** (ran to 0.27) | 0.0000 → 0.0000 | 0.0003 → 0.0000 |
| `openjev-fp8-readout` | 0 of 27 | 0.0054 → 0.0000 | 0.0445 → 0.0000 | **degenerate** (ran to 0.09) | 0.0054 → 0.0000 | 0.0445 → 0.0000 |
| `openjev-bf16-largecard` | 0 of 27 | 0.0050 → 0.0000 | 0.0434 → 0.0000 | **degenerate** (ran to 0.09) | 0.0050 → 0.0000 | 0.0434 → 0.0000 |
| `openjev-fp8-largecard` | 0 of 27 | 0.0054 → 0.0000 | 0.0445 → 0.0000 | **degenerate** (ran to 0.09) | 0.0054 → 0.0000 | 0.0445 → 0.0000 |
| `gemma4-fp8-readout` | 0 of 27 | 0.0000 → 0.0000 | 0.0002 → 0.0000 | **degenerate** (ran to 0.14) | 0.0000 → 0.0000 | 0.0002 → 0.0000 |

*Variant B's Brier is summed over a **reduced** option set and is not the same quantity as variant A's — it is read down its own column, never across.*

### The finding, against the rule written before the fit

*Pre-registered rule (`GAPS.md`, gap 2): the gap is **CLOSED** if the refit brings `gemma4-fp8-readout`'s held-half ECE within 0.02 and Brier within 0.02 of `openjev-fp8-largecard`'s refitted values.*

| | the chat model, refitted | the decision model, refitted | distance | rule |
|---|---|---|---|---|
| ECE | 0.0851 | 0.0459 | **0.0392** | within 0.02? **no** |
| Brier | 0.2375 | 0.1228 | **0.1147** | within 0.02? **no** |

**Verdict: NOT CLOSED.**

A single temperature does a great deal and does not do the job. It cuts the chat model's held-half calibration error from 0.1408 to 0.0851 — **0.0557 of ECE, 40 per cent of it, for no card time at all** — and it cuts the chat model's mean NLL from 1.2989 to 0.5097, which is the floor talking: on the untargeted path four of six letters arrive at −30, the softmax reports near-certainty, and only a large temperature — near **5.3**, which flattens rather than sharpens — can spread that back out. That is why the fitted temperature for the two chat-model arms is **5.301** and **5.433** where the two OpenJev checkpoints sit at **1.168** and **1.145**: the chat model's readout has to be flattened by a factor of five before its numbers mean anything, and the decision model's barely needs touching.

**What no temperature reaches.** After the refit the chat model still posts a Brier of 0.2375 against the decision model's 0.1228, a residual of **0.1147**. That residual is accuracy, not calibration. The two arms are **9.3 accuracy points apart** on the held half — 94.4 % against 85.2 %, which on 54 items is **51 correct answers against 46** — and put on the Brier scale that difference alone is worth **0.1626**, the distance between the rank-blind anchors for the two accuracies (0.1074 and 0.2700). **The accuracy difference is larger than the whole residual**, which is to say the refitted chat model is already beating its own anchor and has no calibration debt left worth naming. What it has is 5 fewer right answers. So the article's question gets a two-part answer, and both parts are in the tables: a refit closes most of the chat model's **calibration** gap for free, and none of its **accuracy** gap — and the accuracy gap is the one README R4 is about, the letters that were never on the table.

**The two checkpoints agree about their own temperature.** The same OpenJev weights at 16-bit and at 8-bit, on the same card and the same items, fit to **1.168** and **1.145** — 0.023 apart, or 2.0 %. The quantisation does not move the calibration, which is one more reason to serve the 8-bit one.

**And the published 0.85 is not the best temperature for anything measured here.** Five of the six arms fit above it — the three OpenJev arms by about a third, the two chat-model readouts by a factor of six. `READOUT_T = 0.85` was fitted for OpenJev on the *targeted* extraction path, where no letter is ever floored; on the untargeted path every one of those arms wants a flatter distribution than the card gives it. The single exception is `jev-readout` at **0.711**, which wants to be **sharper** than 0.85 — the community adapter under the everyday server, and the only arm here whose tuning was somebody else's.

### Task (a) says nothing, and that is the finding

Every arm reads task (a) at **100 per cent**. On a set an arm never gets wrong, the likelihood is maximised by infinite confidence: the NLL falls monotonically as T → 0 and reaches floating-point zero well inside the search interval. So all twelve task-(a) fits are **degenerate** — the number the search stops at is where the arithmetic ran out of mantissa, not a temperature anyone measured, and the tables label it so rather than printing it as a result. **A saturated task cannot calibrate anything**, which is worth knowing before anyone fits a threshold on an easy set.

## GAP 3 — the reranker baseline

**The oldest instrument in the building, on the newest question.** `cross-encoder/ms-marco-MiniLM-L6-v2` is 22 million parameters and six transformer layers, trained in 2020 to score a (query, passage) pair for passage ranking — the same kind of model this workshop already runs as a rerank seat. It scores an option without writing a word, which is the whole mechanism a decision model is sold on, and it predates the phrase by five years. Here it is asked the bench's task (a): **does the article answer this question?**

It ran **on a laptop CPU**, from the copy already in the local model cache. The 42 pages cut into **2363 chunks** (per page: 8 smallest, 37 median, 290 largest), and the 63 items took **42.2 s** in total, a median of **0.41 s** per item. No GPU second and no card was involved in any line of this section.

| | the reranker | what it is being read against |
|---|---|---|
| ROC AUC, all 63 items | **0.982** | — (a readout that is right every time has no ROC to draw) |
| ROC AUC, fit half (31 items) | 0.952 | — |
| ROC AUC, held half (32 items) | **1.000** | — |
| accuracy, fit half (31 items, at its own best threshold) | **93.5 %** | — |
| accuracy, **held half** (32 items, threshold carried over) | **90.6 %** | **100.0 %** for every model arm |
| always-yes floor | 66.7 % | the only floor this task has |
| *held half at its OWN best threshold — an upper bound, not a result* | *100.0 %* | *what the ranking alone is worth* |

The threshold fitted on the fit half is **-5.565** and is the only number carried across; nothing else about the held half was looked at first. The last row is **not** an accuracy anybody could have had — it is the held half graded with a threshold chosen on the held half, printed only so the distance between *ranking* and *deciding* is visible.

### On the 53 items the short-window arms reached

Four of the six model arms ran under a 16,384-token window and refused ten overlong rows. The reranker has no window to refuse with — it chunks — so it is also read on their denominator:

| set | n | AUC | accuracy at the same threshold |
|---|---|---|---|
| all items | 63 | 0.982 | 92.1 % |
| the 53 the short-window arms reached | 53 | 0.979 | 92.5 % |
| held half ∩ those 53 | 24 | 1.000 | 91.7 % |

### Beside the model arms, none of which was re-run

| arm | items scored | accuracy on task (a) |
|---|---|---|
| `jev-readout` — the community adapter, readout | 63 | **100.0 %** |
| `base-readout` — the chat model, readout | 63 | **100.0 %** |
| `openjev-fp8-readout` — OpenJev 8-bit, two cards, readout | 53 | **100.0 %** |
| `openjev-bf16-largecard` — OpenJev 16-bit, one card, readout | 53 | **100.0 %** |
| `openjev-fp8-largecard` — OpenJev 8-bit, one card, readout | 53 | **100.0 %** |
| `gemma4-fp8-readout` — the chat model on OpenJev's server, readout | 53 | **100.0 %** |
| `cross-encoder/ms-marco-MiniLM-L6-v2` — cross-encoder reranker, laptop CPU | 32 | **90.6 %** (held half) |
| always-yes | 63 | 66.7 % |

*The model arms' figures are read out of their own report files and were not re-run for this table.*

### The finding, against the rule written before the model was loaded

*Pre-registered rule (`GAPS.md`, gap 3): **MATCHES** at 100.0 % held-half accuracy, **CLOSE** at ≥ 95 %, **DOES NOT MATCH** below.*

**Verdict: DOES NOT MATCH** — 90.6 % on the held half against 100.0 % for every one of the six model arms, and 66.7 % for always answering yes. With an AUC of 0.982 over all 63 items, this is a **threshold** problem, not a ranking one — the scores order the two classes well and the cut between them is what will not hold.

**And the sharpest way to say it.** On the held half the reranker's AUC is **1.000** — it ranks those 32 items *perfectly*, every answerable question above every unanswerable one. It still gets **3 of them wrong**, because the cut it was given came from the other half and did not transfer. A model that ranks perfectly and decides at 90.6 % is the difference between a score and a decision, in one line. That difference is exactly what a decision model is sold as removing — it is asked a question and answers it, with no threshold to carry and none to fit.

**What this licenses, and it was written down before the number existed.** The comparison is unfair in both directions and the pre-registration says so. Unfair to the reranker: it was trained for passage ranking, not for a plain-language yes/no question; it has no notion of a question being *unanswerable*, which is exactly what the 21 off-page items are; and it needs a threshold **fitted on labels**, where every model arm needed none and was simply asked. Unfair to the model arms: 22 million parameters on a laptop CPU against 26–27 billion on a card. So the only strong reading available was a reranker **win** — that would have said this gate is a retrieval-similarity task in disguise and should be served by a 22 M-parameter model for nothing. That reading is not available.

**And the denominator is small, which the table says in place.** 63 items, a held half of 32, where **one flip is 3.1 points**. Nothing here should be read to three significant figures.

**The honest summary of the old instrument.** A cross-encoder answers *“how well does this passage match this query”*, and the docent's abstain gate asks *“is the answer in this page at all”*. Those are close enough that a retrieval score carries real signal, and far enough apart that the signal does not become a decision without labels to cut it with. The bench's own small-encoder arm (the article's *A model of our own*) is the same lesson from the other side: the ceiling is made of labels.

### Task (c) — pre-registered as SKIPPED, and skipped

A six-way *which guest said this?* would have to be cast as the line scored against six **voice cards**. The frozen kit has no such text: a guest is an `id` and a display name, and the six options as every arm saw them were literally `[A] darwin: Charles Darwin`. Scoring a line against the string "Charles Darwin" with a passage-ranking cross-encoder would measure nothing, and writing voice cards for the six guests would be inventing an instrument mid-bench and handing the baseline text no arm ever saw. This paragraph is the report, and it was written before the model was loaded.

### How to re-run it

```
# on the laptop, no card, no box
~/bench-rerank-venv/bin/python rerank_baseline.py     # ~45 s, writes rows-gaps/rerank.a.jsonl
python3 tables_gaps.py                                # redraws this file
```