RUN START 2026-09-21T18:36:43Z census: longest page 80515 tokens, num_ctx 16384, fits=False, overlong pages 7 ANNOUNCE box benchbox · cards GPU-46890836-d8f, GPU-7aa0be10-974 · arm openjev-fp8-readout · task a · start 2026-09-21T18:36:45Z · expect ~20 min on-01 yes -> REFUSED: overlong — (80515 tokens) on-02 yes -> REFUSED: overlong — (26442 tokens) on-03 yes -> yes 3.34s on-04 yes -> yes 5.09s on-05 yes -> yes 14.20s on-06 yes -> yes 16.00s on-07 yes -> yes 6.00s on-08 yes -> yes 3.91s on-09 yes -> yes 6.90s on-10 yes -> yes 11.95s on-11 yes -> yes 9.44s on-12 yes -> yes 1.99s on-13 yes -> yes 7.70s on-14 yes -> REFUSED: overlong — (31120 tokens) on-15 yes -> yes 5.77s on-16 yes -> yes 4.90s on-17 yes -> yes 11.10s on-18 yes -> yes 5.64s on-19 yes -> yes 6.46s on-20 yes -> yes 5.81s on-21 yes -> yes 5.74s on-22 yes -> yes 7.99s on-23 yes -> yes 9.10s on-24 yes -> yes 8.12s on-25 yes -> yes 5.78s on-26 yes -> yes 6.22s on-27 yes -> yes 16.44s on-28 yes -> yes 13.50s on-29 yes -> REFUSED: overlong — (61727 tokens) on-30 yes -> yes 5.20s on-31 yes -> yes 5.45s on-32 yes -> yes 6.47s on-33 yes -> REFUSED: overlong — (29677 tokens) on-34 yes -> REFUSED: overlong — (16138 tokens) on-35 yes -> yes 11.25s on-36 yes -> yes 12.31s on-37 yes -> yes 7.77s on-38 yes -> yes 11.19s on-39 yes -> yes 9.12s on-40 yes -> REFUSED: overlong — (18796 tokens) on-41 yes -> yes 10.07s on-42 yes -> yes 7.37s off-01 no -> no 7.72s off-02 no -> no 0.23s off-03 no -> no 0.28s off-04 no -> REFUSED: overlong — (80515 tokens) off-05 no -> no 11.99s off-06 no -> no 5.77s off-07 no -> no 4.92s off-08 no -> no 0.36s off-09 no -> no 5.66s off-10 no -> no 3.20s off-11 no -> no 0.72s off-12 no -> no 5.79s off-13 no -> no 5.74s off-14 no -> REFUSED: overlong — (61727 tokens) off-15 no -> no 16.44s off-16 no -> no 5.19s off-17 no -> no 1.99s off-18 no -> REFUSED: overlong — (31120 tokens) off-19 no -> no 8.12s off-20 no -> no 6.94s off-21 no -> no 7.77s {"n": 63, "n_reached": 53, "n_labelled": 53, "refused": 10, "refused_by_reason": {"overlong": 10}, "correct": 53, "errors": 0, "brier": 0.005899739134370508, "reliability": [{"lo": 0.0, "hi": 0.1, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.1, "hi": 0.2, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.2, "hi": 0.3, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.3, "hi": 0.4, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null} ANNOUNCE box benchbox · cards GPU-46890836-d8f, GPU-7aa0be10-974 · arm openjev-fp8-readout · task b · start 2026-09-21T18:43:19Z · expect ~15 min on-01 - -> REFUSED: overlong — (80515 tokens) on-02 - -> REFUSED: overlong — (26442 tokens) on-03 - -> for-the-technically-curious 0.69s on-04 - -> where-these-answers-come-from 5.70s on-05 - -> top 14.55s on-06 - -> how-to-listen 16.26s on-07 - -> the-bill 6.32s on-08 - -> five-minutes-of-training-honestly- 4.11s on-09 - -> sixteen-of-nineteen 1.08s on-10 - -> two-hours-at-12-tok-s-the-test-tha 12.33s on-11 - -> what-the-battery-was-for-the-instr 9.81s on-12 - -> top 0.27s on-13 - -> the-answer-in-one-paragraph 8.04s on-14 - -> REFUSED: overlong — (31120 tokens) on-15 - -> the-developer 5.92s on-16 - -> the-sketch-ladder-the-number-the-g 5.07s on-17 - -> finding-one-a-picture-is-199-token 11.39s on-18 - -> finding-four-the-argument-with-our 5.88s on-19 - -> the-world-is-a-file 6.60s on-20 - -> two-kinds-of-there-all-along 5.97s on-21 - -> top 5.88s on-22 - -> the-day-the-box-got-faster 8.17s on-23 - -> the-same-question-down-the-whole-l 9.39s on-24 - -> the-payroll-and-the-meeting 8.46s on-25 - -> a-measurement-from-our-own-card 5.98s on-26 - -> the-second-librarian-searches-by-m 6.51s on-27 - -> what-to-take-with-you 16.82s on-28 - -> does-anyone-fetch-it 13.70s on-29 - -> REFUSED: overlong — (61727 tokens) on-30 - -> the-first-promise-your-visit-stays 5.33s on-31 - -> top 5.61s on-32 - -> what-to-take-with-you 6.61s on-33 - -> REFUSED: overlong — (29677 tokens) on-34 - -> REFUSED: overlong — (16138 tokens) on-35 - -> vendors 11.40s on-36 - -> what-to-take-with-you 12.51s on-37 - -> what-to-take-with-you 8.04s on-38 - -> what-to-take-with-you 11.52s on-39 - -> top 9.48s on-40 - -> REFUSED: overlong — (18796 tokens) on-41 - -> what-the-adapter-actually-did 10.36s on-42 ['no template tail — no closing sentence of the shape "the article does not specify what was left out, moved or not measured" where the article carries no such statement', "an operator, never the definite one — the hub's voice law (an operator 2026-09-03, re-stated 2026-09-04)", "the prediction's own subject kept — the page predicts that a few COMPANIES will sell intelligence as a utility; the commodity sentence is about SOLD INTELLIGENCE and may not be re-subjected onto the prediction"] -> the-indifferent-prediction 7.64s {"n": 42, "n_reached": 35, "n_labelled": 1, "refused": 7, "refused_by_reason": {"overlong": 7}, "correct": 0, "errors": 0, "brier": 0.5158998546788364, "reliability": [{"lo": 0.0, "hi": 0.1, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.1, "hi": 0.2, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.2, "hi": 0.3, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.3, "hi": 0.4, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo ANNOUNCE box benchbox · cards GPU-46890836-d8f, GPU-7aa0be10-974 · arm openjev-fp8-readout · task c · start 2026-09-21T18:48:14Z · expect ~5 min c-darwin-01-dessert-01 darwin -> darwin 0.29s c-darwin-01-dessert-01 darwin -> darwin 0.39s c-darwin-01-dessert-01 darwin -> darwin 0.30s c-darwin-01-dessert-02 darwin -> darwin 0.41s c-darwin-01-dessert-03 darwin -> darwin 0.42s c-darwin-01-dessert-03 darwin -> darwin 0.42s c-darwin-01-main-01 darwin -> darwin 0.33s c-darwin-01-main-02 darwin -> ibn_sina 0.30s c-darwin-01-main-02 darwin -> darwin 0.37s c-darwin-01-main-02 darwin -> darwin 0.42s c-darwin-01-main-02 darwin -> darwin 0.41s c-darwin-01-main-03 darwin -> darwin 0.41s c-darwin-01-starter-01 darwin -> sagan 0.24s c-darwin-01-starter-01 darwin -> darwin 0.27s c-darwin-01-starter-01 darwin -> darwin 0.30s c-darwin-01-starter-02 darwin -> darwin 0.28s c-darwin-01-starter-03 darwin -> darwin 0.41s c-darwin-01-starter-03 darwin -> darwin 0.40s c-einstein-01-dessert-01 einstein -> einstein 0.37s c-einstein-01-dessert-01 einstein -> einstein 0.40s c-einstein-01-dessert-02 einstein -> einstein 0.36s c-einstein-01-dessert-02 einstein -> hypatia 0.29s c-einstein-01-dessert-03 einstein -> einstein 0.42s c-einstein-01-main-01 einstein -> hypatia 0.30s c-einstein-01-main-01 einstein -> einstein 0.35s c-einstein-01-main-02 einstein -> einstein 0.40s c-einstein-01-main-03 einstein -> einstein 0.35s c-einstein-01-main-03 einstein -> einstein 0.35s c-einstein-01-main-03 einstein -> einstein 0.36s c-einstein-01-main-03 einstein -> einstein 0.41s c-einstein-01-starter-01 einstein -> einstein 0.31s c-einstein-01-starter-02 einstein -> einstein 0.27s c-einstein-01-starter-02 einstein -> einstein 0.30s c-einstein-01-starter-03 einstein -> einstein 0.40s c-einstein-01-starter-03 einstein -> einstein 0.35s c-einstein-01-starter-03 einstein -> einstein 0.28s c-hypatia-01-dessert-01 hypatia -> hypatia 0.41s c-hypatia-01-dessert-02 hypatia -> hypatia 0.33s c-hypatia-01-dessert-02 hypatia -> hypatia 0.28s c-hypatia-01-dessert-03 hypatia -> hypatia 0.30s c-hypatia-01-main-01 hypatia -> hypatia 0.37s c-hypatia-01-main-01 hypatia -> hypatia 0.30s c-hypatia-01-main-01 hypatia -> hypatia 0.40s c-hypatia-01-main-01 hypatia -> hypatia 0.34s c-hypatia-01-main-01 hypatia -> hypatia 0.28s c-hypatia-01-main-02 hypatia -> hypatia 0.36s c-hypatia-01-main-03 hypatia -> hypatia 0.37s c-hypatia-01-main-03 hypatia -> hypatia 0.34s c-hypatia-01-main-03 hypatia -> hypatia 0.42s c-hypatia-01-starter-01 hypatia -> hypatia 0.28s c-hypatia-01-starter-01 hypatia -> socrates 0.33s c-hypatia-01-starter-02 hypatia -> hypatia 0.31s c-hypatia-01-starter-02 hypatia -> hypatia 0.27s c-hypatia-01-starter-02 hypatia -> hypatia 0.32s c-ibn_sina-01-dessert-01 ibn_sina -> ibn_sina 0.32s c-ibn_sina-01-dessert-02 ibn_sina -> ibn_sina 0.33s c-ibn_sina-01-dessert-02 ibn_sina -> ibn_sina 0.35s c-ibn_sina-01-dessert-02 ibn_sina -> ibn_sina 0.35s c-ibn_sina-01-dessert-03 ibn_sina -> ibn_sina 0.36s c-ibn_sina-01-dessert-03 ibn_sina -> ibn_sina 0.41s c-ibn_sina-01-main-02 ibn_sina -> ibn_sina 0.33s c-ibn_sina-01-main-02 ibn_sina -> ibn_sina 0.36s c-ibn_sina-01-main-02 ibn_sina -> ibn_sina 0.37s c-ibn_sina-01-main-03 ibn_sina -> ibn_sina 0.40s c-ibn_sina-01-main-03 ibn_sina -> ibn_sina 0.38s c-ibn_sina-01-main-03 ibn_sina -> ibn_sina 0.41s c-ibn_sina-01-starter-01 ibn_sina -> ibn_sina 0.30s c-ibn_sina-01-starter-02 ibn_sina -> ibn_sina 0.32s c-ibn_sina-01-starter-03 ibn_sina -> ibn_sina 0.38s c-ibn_sina-01-starter-03 ibn_sina -> ibn_sina 0.36s c-ibn_sina-01-starter-03 ibn_sina -> ibn_sina 0.38s c-ibn_sina-01-starter-03 ibn_sina -> ibn_sina 0.35s c-sagan-01-dessert-01 sagan -> sagan 0.41s c-sagan-01-dessert-01 sagan -> sagan 0.32s c-sagan-01-dessert-01 sagan -> sagan 0.29s c-sagan-01-dessert-03 sagan -> sagan 0.34s c-sagan-01-dessert-03 sagan -> sagan 0.42s c-sagan-01-dessert-03 sagan -> sagan 0.34s c-sagan-01-dessert-03 sagan -> sagan 0.37s c-sagan-01-dessert-03 sagan -> sagan 0.41s c-sagan-01-main-01 sagan -> sagan 0.34s c-sagan-01-main-02 sagan -> sagan 0.37s c-sagan-01-main-02 sagan -> sagan 0.32s c-sagan-01-main-03 sagan -> sagan 0.41s c-sagan-01-main-03 sagan -> sagan 0.41s c-sagan-01-starter-01 sagan -> sagan 0.29s c-sagan-01-starter-02 sagan -> sagan 0.41s c-sagan-01-starter-02 sagan -> sagan 0.30s c-sagan-01-starter-03 sagan -> darwin 0.30s c-sagan-01-starter-03 sagan -> sagan 0.36s c-socrates-01-dessert-01 socrates -> socrates 0.42s c-socrates-01-dessert-01 socrates -> socrates 0.29s c-socrates-01-dessert-01 socrates -> socrates 0.37s c-socrates-01-dessert-02 socrates -> socrates 0.35s c-socrates-01-dessert-02 socrates -> socrates 0.39s c-socrates-01-dessert-02 socrates -> socrates 0.31s c-socrates-01-main-01 socrates -> socrates 0.33s c-socrates-01-main-02 socrates -> socrates 0.41s c-socrates-01-main-02 socrates -> socrates 0.30s c-socrates-01-main-02 socrates -> socrates 0.37s c-socrates-01-main-02 socrates -> socrates 0.30s c-socrates-01-main-03 socrates -> hypatia 0.35s c-socrates-01-main-03 socrates -> socrates 0.39s c-socrates-01-starter-01 socrates -> socrates 0.27s c-socrates-01-starter-02 socrates -> socrates 0.32s c-socrates-01-starter-03 socrates -> socrates 0.34s c-socrates-01-starter-03 socrates -> socrates 0.36s c-socrates-01-starter-03 socrates -> socrates 0.33s {"n": 108, "n_reached": 108, "n_labelled": 108, "refused": 0, "refused_by_reason": {}, "correct": 101, "errors": 0, "brier": 0.12186417194305185, "reliability": [{"lo": 0.0, "hi": 0.1, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.1, "hi": 0.2, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.2, "hi": 0.3, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.3, "hi": 0.4, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.4 census: reusing rows/census.openjev-fp8.json census: longest page 80515 tokens, num_ctx 16384, fits=False, overlong pages 7 ANNOUNCE box benchbox · cards GPU-46890836-d8f, GPU-7aa0be10-974 · arm openjev-fp8-generate · task a · start 2026-09-21T18:49:02Z · expect ~20 min on-01 yes -> REFUSED: overlong — (80515 tokens) on-02 yes -> REFUSED: overlong — (26442 tokens) on-03 yes -> yes 3.22s on-04 yes -> yes 5.29s on-05 yes -> yes 14.33s on-06 yes -> yes 16.20s on-07 yes -> yes 6.06s on-08 yes -> yes 3.99s on-09 yes -> yes 6.96s on-10 yes -> yes 12.08s on-11 yes -> yes 9.50s on-12 yes -> yes 2.03s on-13 yes -> yes 7.75s on-14 yes -> REFUSED: overlong — (31120 tokens) on-15 yes -> yes 5.83s on-16 yes -> yes 4.95s on-17 yes -> yes 11.18s on-18 yes -> yes 5.69s on-19 yes -> yes 6.51s on-20 yes -> yes 5.85s on-21 yes -> yes 5.79s on-22 yes -> yes 7.99s on-23 yes -> yes 9.15s on-24 yes -> yes 8.16s on-25 yes -> yes 5.81s on-26 yes -> yes 6.25s on-27 yes -> yes 16.48s on-28 yes -> yes 13.52s on-29 yes -> REFUSED: overlong — (61727 tokens) on-30 yes -> yes 5.23s on-31 yes -> yes 5.48s on-32 yes -> yes 6.50s on-33 yes -> REFUSED: overlong — (29677 tokens) on-34 yes -> REFUSED: overlong — (16138 tokens) on-35 yes -> yes 11.27s on-36 yes -> yes 12.33s on-37 yes -> yes 7.80s on-38 yes -> yes 11.21s on-39 yes -> yes 9.15s on-40 yes -> REFUSED: overlong — (18796 tokens) on-41 yes -> yes 10.09s on-42 yes -> yes 7.39s off-01 no -> no 7.74s off-02 no -> no 0.26s off-03 no -> no 0.31s off-04 no -> REFUSED: overlong — (80515 tokens) off-05 no -> no 12.02s off-06 no -> no 5.81s off-07 no -> no 4.95s off-08 no -> no 0.39s off-09 no -> no 5.69s off-10 no -> no 3.23s off-11 no -> no 0.75s off-12 no -> no 5.82s off-13 no -> no 5.77s off-14 no -> REFUSED: overlong — (61727 tokens) off-15 no -> no 16.47s off-16 no -> no 5.22s off-17 no -> no 2.03s off-18 no -> REFUSED: overlong — (31120 tokens) off-19 no -> no 8.15s off-20 no -> no 6.97s off-21 no -> no 7.80s {"n": 63, "n_reached": 53, "n_labelled": 53, "refused": 10, "refused_by_reason": {"overlong": 10}, "correct": 53, "errors": 0, "brier": 8.349285404259997e-05, "reliability": [{"lo": 0.0, "hi": 0.1, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.1, "hi": 0.2, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.2, "hi": 0.3, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.3, "hi": 0.4, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null ANNOUNCE box benchbox · cards GPU-46890836-d8f, GPU-7aa0be10-974 · arm openjev-fp8-generate · task b · start 2026-09-21T18:55:39Z · expect ~15 min on-01 - -> REFUSED: overlong — (80515 tokens) on-02 - -> REFUSED: overlong — (26442 tokens) on-03 - -> for-the-technically-curious 0.72s on-04 - -> where-these-answers-come-from 5.73s on-05 - -> top 14.56s on-06 - -> how-to-listen 16.27s on-07 - -> the-bill 6.34s on-08 - -> five-minutes-of-training-honestly- 4.14s on-09 - -> sixteen-of-nineteen 1.10s on-10 - -> two-hours-at-12-tok-s-the-test-tha 12.34s on-11 - -> what-the-battery-was-for-the-instr 9.82s on-12 - -> top 0.30s on-13 - -> the-answer-in-one-paragraph 8.05s on-14 - -> REFUSED: overlong — (31120 tokens) on-15 - -> the-developer 5.94s on-16 - -> the-sketch-ladder-the-number-the-g 5.10s on-17 - -> finding-one-a-picture-is-199-token 11.40s on-18 - -> finding-four-the-argument-with-our 5.91s on-19 - -> the-world-is-a-file 6.62s on-20 - -> two-kinds-of-there-all-along 5.99s on-21 - -> top 5.90s on-22 - -> the-day-the-box-got-faster 8.19s on-23 - -> the-same-question-down-the-whole-l 9.41s on-24 - -> the-payroll-and-the-meeting 8.48s on-25 - -> a-measurement-from-our-own-card 6.00s on-26 - -> the-second-librarian-searches-by-m 6.53s on-27 - -> what-to-take-with-you 16.83s on-28 - -> does-anyone-fetch-it 13.71s on-29 - -> REFUSED: overlong — (61727 tokens) on-30 - -> the-first-promise-your-visit-stays 5.35s on-31 - -> top 5.63s on-32 - -> what-to-take-with-you 6.63s on-33 - -> REFUSED: overlong — (29677 tokens) on-34 - -> REFUSED: overlong — (16138 tokens) on-35 - -> vendors 11.42s on-36 - -> what-to-take-with-you 12.53s on-37 - -> what-to-take-with-you 8.06s on-38 - -> what-to-take-with-you 11.54s on-39 - -> top 9.50s on-40 - -> REFUSED: overlong — (18796 tokens) on-41 - -> what-the-adapter-actually-did 10.38s on-42 ['no template tail — no closing sentence of the shape "the article does not specify what was left out, moved or not measured" where the article carries no such statement', "an operator, never the definite one — the hub's voice law (an operator 2026-09-03, re-stated 2026-09-04)", "the prediction's own subject kept — the page predicts that a few COMPANIES will sell intelligence as a utility; the commodity sentence is about SOLD INTELLIGENCE and may not be re-subjected onto the prediction"] -> the-indifferent-prediction 7.66s {"n": 42, "n_reached": 35, "n_labelled": 1, "refused": 7, "refused_by_reason": {"overlong": 7}, "correct": 0, "errors": 0, "brier": 1.0, "reliability": [{"lo": 0.0, "hi": 0.1, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.1, "hi": 0.2, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.2, "hi": 0.3, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.3, "hi": 0.4, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.4, "hi": 0 ANNOUNCE box benchbox · cards GPU-46890836-d8f, GPU-7aa0be10-974 · arm openjev-fp8-generate · task c · start 2026-09-21T19:00:34Z · expect ~5 min c-darwin-01-dessert-01 darwin -> darwin 0.31s c-darwin-01-dessert-01 darwin -> darwin 0.43s c-darwin-01-dessert-01 darwin -> darwin 0.34s c-darwin-01-dessert-02 darwin -> darwin 0.45s c-darwin-01-dessert-03 darwin -> darwin 0.46s c-darwin-01-dessert-03 darwin -> darwin 0.46s c-darwin-01-main-01 darwin -> darwin 0.37s c-darwin-01-main-02 darwin -> ibn_sina 0.34s c-darwin-01-main-02 darwin -> darwin 0.41s c-darwin-01-main-02 darwin -> darwin 0.46s c-darwin-01-main-02 darwin -> darwin 0.45s c-darwin-01-main-03 darwin -> darwin 0.45s c-darwin-01-starter-01 darwin -> sagan 0.28s c-darwin-01-starter-01 darwin -> darwin 0.31s c-darwin-01-starter-01 darwin -> darwin 0.34s c-darwin-01-starter-02 darwin -> darwin 0.32s c-darwin-01-starter-03 darwin -> darwin 0.45s c-darwin-01-starter-03 darwin -> darwin 0.44s c-einstein-01-dessert-01 einstein -> einstein 0.41s c-einstein-01-dessert-01 einstein -> einstein 0.43s c-einstein-01-dessert-02 einstein -> einstein 0.40s c-einstein-01-dessert-02 einstein -> hypatia 0.33s c-einstein-01-dessert-03 einstein -> einstein 0.46s c-einstein-01-main-01 einstein -> hypatia 0.34s c-einstein-01-main-01 einstein -> einstein 0.39s c-einstein-01-main-02 einstein -> einstein 0.44s c-einstein-01-main-03 einstein -> einstein 0.39s c-einstein-01-main-03 einstein -> einstein 0.39s c-einstein-01-main-03 einstein -> einstein 0.40s c-einstein-01-main-03 einstein -> einstein 0.45s c-einstein-01-starter-01 einstein -> einstein 0.35s c-einstein-01-starter-02 einstein -> einstein 0.31s c-einstein-01-starter-02 einstein -> einstein 0.34s c-einstein-01-starter-03 einstein -> einstein 0.44s c-einstein-01-starter-03 einstein -> einstein 0.39s c-einstein-01-starter-03 einstein -> einstein 0.32s c-hypatia-01-dessert-01 hypatia -> hypatia 0.45s c-hypatia-01-dessert-02 hypatia -> hypatia 0.37s c-hypatia-01-dessert-02 hypatia -> hypatia 0.32s c-hypatia-01-dessert-03 hypatia -> hypatia 0.34s c-hypatia-01-main-01 hypatia -> hypatia 0.42s c-hypatia-01-main-01 hypatia -> hypatia 0.34s c-hypatia-01-main-01 hypatia -> hypatia 0.44s c-hypatia-01-main-01 hypatia -> hypatia 0.38s c-hypatia-01-main-01 hypatia -> hypatia 0.32s c-hypatia-01-main-02 hypatia -> hypatia 0.40s c-hypatia-01-main-03 hypatia -> hypatia 0.41s c-hypatia-01-main-03 hypatia -> hypatia 0.38s c-hypatia-01-main-03 hypatia -> hypatia 0.46s c-hypatia-01-starter-01 hypatia -> hypatia 0.32s c-hypatia-01-starter-01 hypatia -> socrates 0.38s c-hypatia-01-starter-02 hypatia -> hypatia 0.35s c-hypatia-01-starter-02 hypatia -> hypatia 0.32s c-hypatia-01-starter-02 hypatia -> hypatia 0.36s c-ibn_sina-01-dessert-01 ibn_sina -> ibn_sina 0.36s c-ibn_sina-01-dessert-02 ibn_sina -> ibn_sina 0.37s c-ibn_sina-01-dessert-02 ibn_sina -> ibn_sina 0.39s c-ibn_sina-01-dessert-02 ibn_sina -> ibn_sina 0.39s c-ibn_sina-01-dessert-03 ibn_sina -> ibn_sina 0.40s c-ibn_sina-01-dessert-03 ibn_sina -> ibn_sina 0.45s c-ibn_sina-01-main-02 ibn_sina -> ibn_sina 0.37s c-ibn_sina-01-main-02 ibn_sina -> ibn_sina 0.40s c-ibn_sina-01-main-02 ibn_sina -> ibn_sina 0.42s c-ibn_sina-01-main-03 ibn_sina -> ibn_sina 0.44s c-ibn_sina-01-main-03 ibn_sina -> ibn_sina 0.42s c-ibn_sina-01-main-03 ibn_sina -> ibn_sina 0.45s c-ibn_sina-01-starter-01 ibn_sina -> ibn_sina 0.34s c-ibn_sina-01-starter-02 ibn_sina -> ibn_sina 0.36s c-ibn_sina-01-starter-03 ibn_sina -> ibn_sina 0.42s c-ibn_sina-01-starter-03 ibn_sina -> ibn_sina 0.40s c-ibn_sina-01-starter-03 ibn_sina -> ibn_sina 0.42s c-ibn_sina-01-starter-03 ibn_sina -> ibn_sina 0.39s c-sagan-01-dessert-01 sagan -> sagan 0.45s c-sagan-01-dessert-01 sagan -> sagan 0.36s c-sagan-01-dessert-01 sagan -> sagan 0.33s c-sagan-01-dessert-03 sagan -> sagan 0.38s c-sagan-01-dessert-03 sagan -> sagan 0.46s c-sagan-01-dessert-03 sagan -> sagan 0.38s c-sagan-01-dessert-03 sagan -> sagan 0.41s c-sagan-01-dessert-03 sagan -> sagan 0.45s c-sagan-01-main-01 sagan -> sagan 0.38s c-sagan-01-main-02 sagan -> sagan 0.42s c-sagan-01-main-02 sagan -> sagan 0.37s c-sagan-01-main-03 sagan -> sagan 0.45s c-sagan-01-main-03 sagan -> sagan 0.45s c-sagan-01-starter-01 sagan -> sagan 0.33s c-sagan-01-starter-02 sagan -> sagan 0.46s c-sagan-01-starter-02 sagan -> sagan 0.34s c-sagan-01-starter-03 sagan -> darwin 0.35s c-sagan-01-starter-03 sagan -> sagan 0.40s c-socrates-01-dessert-01 socrates -> socrates 0.46s c-socrates-01-dessert-01 socrates -> socrates 0.33s c-socrates-01-dessert-01 socrates -> socrates 0.41s c-socrates-01-dessert-02 socrates -> socrates 0.39s c-socrates-01-dessert-02 socrates -> socrates 0.43s c-socrates-01-dessert-02 socrates -> socrates 0.35s c-socrates-01-main-01 socrates -> socrates 0.37s c-socrates-01-main-02 socrates -> socrates 0.45s c-socrates-01-main-02 socrates -> socrates 0.34s c-socrates-01-main-02 socrates -> socrates 0.42s c-socrates-01-main-02 socrates -> socrates 0.34s c-socrates-01-main-03 socrates -> hypatia 0.39s c-socrates-01-main-03 socrates -> socrates 0.44s c-socrates-01-starter-01 socrates -> socrates 0.31s c-socrates-01-starter-02 socrates -> socrates 0.37s c-socrates-01-starter-03 socrates -> socrates 0.38s c-socrates-01-starter-03 socrates -> socrates 0.40s c-socrates-01-starter-03 socrates -> socrates 0.38s {"n": 108, "n_reached": 108, "n_labelled": 108, "refused": 0, "refused_by_reason": {}, "correct": 101, "errors": 0, "brier": 0.12962962962962962, "reliability": [{"lo": 0.0, "hi": 0.1, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.1, "hi": 0.2, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.2, "hi": 0.3, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.3, "hi": 0.4, "n": 0, "sum_p": 0.0, "correct": 0, "mean_p": null, "accuracy": null, "gap": null}, {"lo": 0.4 RC=0 RUN END 2026-09-21T19:01:27Z