{
 "schema_version": "1.0",
 "page_kind": "article",
 "pack_kind": "generate",
 "slug": "indra-ocr-on-rulesage-rulebook-pages",
 "title": "Indra OCR, measured on RuleSage's rulebook pages",
 "dek": "Under bars frozen before the run, Indra OCR was REJECTED for RuleSage's 635 image-only rulebook pages on small print alone, finding 0.30 of it against a bar of 0.60; body text, tables, degraded copies and spreads passed, and it beat Tesseract 5 (tessdata_best).",
 "published": "2026-10-28",
 "series": [
  "bench"
 ],
 "licence": "CC BY 4.0",
 "status": "pending-judge",
 "pack_note": "no judge is seated, so every chip is held",
 "notice": "items with published:false are held and are not this page's published words",
 "source": {
  "url": "https://research.strata2signal.com/indra-ocr-on-rulesage-rulebook-pages/",
  "md_url": "https://research.strata2signal.com/indra-ocr-on-rulesage-rulebook-pages/index.md",
  "md_sha": "780ae14490f5080295590d35dbfe02941d95c9426cc6610da4563872839d2488",
  "html_sha": "a23aee39d2c5c0d8f0e35905ecca45c880d6db9e318a04785d914c1115731b3a",
  "anchors_sha": "1f1b7b04289c0131d4c9cea98ba273696a7c6d223df2e1fbd6a18f38960034a3"
 },
 "short": {
  "paragraph": "OCR turns a picture of a page into text. RuleSage reads its rulebooks through the text layer a PDF carries, and 635 distinct pages in its archive, 2.9% of its 21,614 pages, have none, two whole games among them. We put Indra OCR 1.1.2 against those rulebooks on a laptop CPU, with the bars, the page sets and nineteen predictions frozen first. On 169 body pages, one page per rulebook, it read the text it found with 0.38% of characters wrong and found 0.978 of the visible text, against bars of 2% and 0.90; tables, a 150 dpi JPEG stand-in for a scan and oversize spreads passed too, and on the paired 190-page sample it beat Tesseract 5's tessdatabest model by 7.5 points of flexible character error. On 21 small-print pages it found 0.30 of the visible text against a bar of 0.60; Tesseract found 0.29 with tessdatabest and 0.33 with tessdatafast. That one row makes the verdict REJECTED; the plan names the outcome, \"rejected only on small print\", and says it licenses nothing new. A diagnostic that doubled the detector's working size lifted small-print coverage to 0.70 on the same pages, but almost all of that gain came from one page; on the other 20, coverage moved from 0.49 to 0.52, so the cause is not isolated. The diagnostic was planned in the frozen plan and is reported, never gated. The 635 pages stay unread by RuleSage, and no OCR text enters it.",
  "from_dek": false,
  "counts": {
   "words": 4357,
   "minutes": 20,
   "tables": 6,
   "kit": false
  },
  "bullets": []
 },
 "sections": [
  {
   "id": "the-words-this-page-leans-on",
   "heading": "The words this page leans on",
   "level": 2,
   "span": [
    2873,
    4331
   ],
   "chunks": [
    [
     2873,
     4331
    ]
   ],
   "chars": 1458,
   "digest": "the article defines key terms including text layer, OCR, reference, coverage, read CER, flex CER, and the 95% interval.",
   "digest_skipped": null
  },
  {
   "id": "the-question-and-the-census",
   "heading": "The question, and the census",
   "level": 2,
   "span": [
    4331,
    5483
   ],
   "chunks": [
    [
     4331,
     5483
    ]
   ],
   "chars": 1152,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "the-verdict-and-every-gate",
   "heading": "The verdict, and every gate",
   "level": 2,
   "span": [
    5483,
    8962
   ],
   "chunks": [
    [
     5483,
     8962
    ]
   ],
   "chars": 3479,
   "digest": "the verdict is REJECTED, with one failed row, G2-fine-cover, a small-print row. body pages met the bars, but small print did not. the table shows various gates, metrics, and statuses for different page sets.",
   "digest_skipped": null
  },
  {
   "id": "how-the-bars-were-set",
   "heading": "How the bars were set, and what was read",
   "level": 2,
   "span": [
    8962,
    12621
   ],
   "chunks": [
    [
     8962,
     12621
    ]
   ],
   "chars": 3659,
   "digest": "the plan was frozen on 2026-10-27. page sets include S1, S2, S3, and S4. the reference uses PDFBox 3.0.3 and Ghostscript. the engines ran on a CPU, specifically an Intel Core Ultra 9 290HX Plus, with no graphics card touched.",
   "digest_skipped": null
  },
  {
   "id": "the-page-classes",
   "heading": "The page classes",
   "level": 2,
   "span": [
    12621,
    16878
   ],
   "chunks": [
    [
     12621,
     16878
    ]
   ],
   "chars": 4257,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "the-real-image-only-pages",
   "heading": "The real image-only pages: text found, not text checked",
   "level": 2,
   "span": [
    16878,
    18471
   ],
   "chunks": [
    [
     16878,
     18471
    ]
   ],
   "chars": 1593,
   "digest": "for the 635 image-only pages, there is no reference, so only yield is reported. the engines found varying amounts of text across covers, scan files, and interior pages.",
   "digest_skipped": null
  },
  {
   "id": "the-diagnostic",
   "heading": "The diagnostic: the detector's working size",
   "level": 2,
   "span": [
    18471,
    20613
   ],
   "chunks": [
    [
     18471,
     20613
    ]
   ],
   "chars": 2142,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "the-predictions",
   "heading": "The predictions, written before the run",
   "level": 2,
   "span": [
    20613,
    22697
   ],
   "chunks": [
    [
     20613,
     22697
    ]
   ],
   "chars": 2084,
   "digest": "nineteen predictions were scored, with fourteen hits and five misses. the verdict was a HIT, while fine coverage was a MISS, below the band.",
   "digest_skipped": null
  },
  {
   "id": "what-this-means-for-rulesage",
   "heading": "What this means for RuleSage: nothing changes",
   "level": 2,
   "span": [
    22697,
    23200
   ],
   "chunks": [
    [
     22697,
     23200
    ]
   ],
   "chars": 503,
   "digest": "the gate said REJECTED, so nothing changes for RuleSage. the 635 image-only pages stay unread and no OCR text enters its corpus.",
   "digest_skipped": null
  },
  {
   "id": "two-things-we-could-do-next",
   "heading": "Two things we could do next",
   "level": 2,
   "span": [
    23200,
    24110
   ],
   "chunks": [
    [
     23200,
     24110
    ]
   ],
   "chars": 910,
   "digest": "two optional follow-ups are mentioned: a human rubric on 20 of the real pages and a registered run at the larger detector size.",
   "digest_skipped": null
  },
  {
   "id": "what-this-cannot-say",
   "heading": "What this cannot say",
   "level": 2,
   "span": [
    24110,
    24973
   ],
   "chunks": [
    [
     24110,
     24973
    ]
   ],
   "chars": 863,
   "digest": "the article notes it cannot say anything about accuracy on real image-only pages, table structure, handwriting, or speed.",
   "digest_skipped": null
  },
  {
   "id": "what-to-take-with-you",
   "heading": "What to take with you",
   "level": 2,
   "span": [
    24973,
    26042
   ],
   "chunks": [
    [
     24973,
     26042
    ]
   ],
   "chars": 1069,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "the-licence",
   "heading": "The licence, as published",
   "level": 2,
   "span": [
    26042,
    27138
   ],
   "chunks": [
    [
     26042,
     27138
    ]
   ],
   "chars": 1096,
   "digest": null,
   "digest_skipped": "credits"
  },
  {
   "id": "who-ran-this-and-thanks",
   "heading": "Who ran this, and thanks",
   "level": 2,
   "span": [
    27138,
    28358
   ],
   "chunks": [
    [
     27138,
     28358
    ]
   ],
   "chars": 1220,
   "digest": null,
   "digest_skipped": "credits"
  },
  {
   "id": "sources",
   "heading": "Sources",
   "level": 2,
   "span": [
    28358,
    29607
   ],
   "chunks": [
    [
     28358,
     29607
    ]
   ],
   "chars": 1249,
   "digest": "the article cites its own records, Indra OCR, Tesseract 5, and a paper by Clausner, Pletschacher and Antonacopoulos.",
   "digest_skipped": null
  }
 ],
 "chips": [
  {
   "id": "c-23bc07a4",
   "q": "what terminology defines the text layers and recognition models used?",
   "a": "The article defines text layers as the data behind a PDF, born-digital versus image-only. It describes OCR as a model reading a page's picture, specifically mentioning Indra OCR and Tesseract 5 with its tessdata_best and tessdata_fast models.",
   "cites": [
    "the-words-this-page-leans-on"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-e1137b1c",
   "q": "what was the final verdict of the benchmark testing?",
   "a": "The verdict was REJECTED, with one failed row, G2-fine-cover, a small-print row. While body pages met the required bars, the small print did not, meaning the outcome licenses nothing new.",
   "cites": [
    "the-verdict-and-every-gate"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-ba4ecbff",
   "q": "how were the page sets and reference text determined?",
   "a": "Page sets include S1 (born-digital), S2 (tables), S3 (image-only), and S4 (degraded). Reference text is derived from PDFBox 3.0.3, filtered to visible glyphs using Ghostscript to ensure only characters rendered in the page are counted.",
   "cites": [
    "how-the-bars-were-set"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-981363a5",
   "q": "what yield was observed for the real image-only pages?",
   "a": "On 351 covers, Indra found at least 20 characters on 245 pages. On 159 other interior and last pages, Indra found at least 20 characters on 67 pages, while Tesseract arms found it on 59 pages.",
   "cites": [
    "the-real-image-only-pages"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-339fdc70",
   "q": "what impact does this benchmark have on rulesage operations?",
   "a": "Nothing changes for RuleSage. Because the benchmark was rejected, the 635 image-only pages remain unread, no new OCR text enters the corpus, and Indra is not proposed for use.",
   "cites": [
    "what-this-means-for-rulesage"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  }
 ],
 "related": [
  {
   "slug": "nemotron-rerank-8b-on-our-own-workloads",
   "why": "shares ground with § What this means for RuleSage: nothing changes · § What changes at RuleSage: nothing"
  },
  {
   "slug": "the-quantization-ladder",
   "why": "shares ground with § The predictions, written before the run · § The predictions, judged"
  },
  {
   "slug": "the-instrument-travels",
   "why": "shares ground with § The verdict, and every gate · § The standings"
  }
 ],
 "thanks": "## Who ran this, and thanks {#who-ran-this-and-thanks}\n\nThanks to Datalab for releasing Indra OCR, its code under Apache-2.0 and its weights under the licence above, and to the Tesseract project for Tesseract 5.5.0 (Apache-2.0) and its tessdata_best models. The rulebooks are their publishers' copyrighted work and stay inside RuleSage's archive; this page prints counts and scores only. Software the bench stood on: **ONNX Runtime** 1.31.0 (MIT), which ran Indra on the CPU; **PDFBox** 3.0.3 (Apache-2.0), the reference's text; **Ghostscript** 10.06.0 (AGPL-3.0), the visibility renders; **pypdfium2** 5.14.0 (Apache-2.0 and BSD-3-Clause, over PDFium), the engines' input; **rapidfuzz** 3.14.6 (MIT), the alignments; **NumPy** 2.5.4 (BSD-3-Clause) and **OpenCV** 5.0.0 headless (Apache-2.0); **Python** 3.13 and **Java** 21. The matcher follows Clausner, Pletschacher and Antonacopoulos's flexible character accuracy as a greedy implementation of our own. A small human team asked the question, approved the plan before the run and will approve this page; a fleet of AI agents wrote the plan, the critique and the kit, ran the census, the smoke and the registered run, and drafted this page under that team's rulings.",
 "kit": null,
 "seat_class": {
  "writer": "gemma-class",
  "judge": null,
  "writer_runtime": "vllm"
 },
 "bench": {
  "bullets_written": 3,
  "bullets_kept": 0,
  "digests_written": 13,
  "digests_kept": 9,
  "chips_written": 8,
  "chips_kept": 5,
  "chips_grounded": 0,
  "chips_published": 0,
  "dropped_by": {
   "new_noun": 0,
   "figure": 2,
   "length": 0,
   "cite": 0,
   "judge": 0,
   "quote": 0,
   "directive": 0,
   "redaction": 0,
   "profanity": 0,
   "identifier": 0,
   "bare_figure": 1,
   "house_voice": 0,
   "editorial": 0,
   "hand-edit": 7
  },
  "new_noun_tokens_checked": [
   "0.01",
   "0.38%",
   "0.49",
   "0.52",
   "0.83%",
   "0.9623",
   "0.9780",
   "0.99",
   "024",
   "048",
   "1",
   "1.1.2",
   "150",
   "159",
   "2",
   "2.95%",
   "20",
   "2026-10-27",
   "21",
   "245",
   "290HX",
   "3.0.3",
   "351",
   "40",
   "402",
   "5",
   "5's",
   "59",
   "614",
   "635",
   "67",
   "72",
   "85",
   "9",
   "95%",
   "Antonacopoulos",
   "CER",
   "CPU",
   "Clausner",
   "Core",
   "G2-fine-cover",
   "Ghostscript",
   "HIT",
   "Indra",
   "Indra's",
   "Intel",
   "MISS",
   "OCR",
   "PDF",
   "PDFBox",
   "Pletschacher",
   "Plus",
   "REJECTED",
   "Read",
   "RuleSage",
   "RuleSage's",
   "S1",
   "S2",
   "S3",
   "S4",
   "Tesseract",
   "Tesseract's",
   "Ultra"
  ],
  "new_noun_tokens_withheld": 0,
  "figure_definition": "v3",
  "figure_boundary": "guarded",
  "drops": [
   {
    "kind": "digest",
    "reason": "figure",
    "item": "the table provides micro-averaged metrics for different page classes including body, fine, oversize, natural, and tables, comparing Indra 1.1.2 against Tesseract 5 arms.",
    "detail": "figures absent: ['5']"
   },
   {
    "kind": "bullet",
    "reason": "bare_figure",
    "item": "the median raster on those pages is placed at 150",
    "detail": "the cited section writes this figure as '150 pixels' and the bullet prints '150' alone"
   },
   {
    "kind": "chip",
    "reason": "figure",
    "item": "On S1 body pages, Indra 1.1.2 achieved a Read CER of 0.38% and coverage of 0.9780. This outperformed Tesseract 5's tessdata_best, which had a Read CER of 0.83% and coverage of 0.9623.",
    "detail": "figures absent from cited spans: ['5']"
   },
   {
    "kind": "bullet",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 49,
    "text_sha": "22b02c8e9743",
    "figure": "635",
    "withheld": "one bullet removed from this pack by hand",
    "removed_utc": "2026-10-27",
    "why": "ends on a bare 635 and drops what was counted: 635 distinct image-only pages in 402 files",
    "detail": "Carried forward by --carry-hand-edits on 2026-10-28: the writer produced this line again and the pack on disk records it removed by hand on 2026-10-27 (matched on its line key). The earlier record reads: removed from this pack by hand on 2026-10-27 (UTC), after the writer ran and after the gates had counted: ends on a bare 635 and drops what was counted: 635 distinct image-only pages in 402 files. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 220,
    "text_sha": "74c706fdbb03",
    "section": "the-question-and-the-census",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-10-27",
    "why": "the question loses its second half: the page asks whether Indra OCR reads rulebook pages accurately AND completely enough (coverage, the metric the verdict failed on), and whether it beats Tesseract 5 with its best English model (tessdata_best); the line names neither",
    "detail": "Carried forward by --carry-hand-edits on 2026-10-28: the writer produced this line again and the pack on disk records it removed by hand on 2026-10-27 (matched on its section). The earlier record reads: removed from this pack by hand on 2026-10-27 (UTC), after the writer ran and after the gates had counted: the question loses its second half: the page asks whether Indra OCR reads rulebook pages accurately AND completely enough (coverage, the metric the verdict failed on), and whether it beats Tesseract 5 with its best English model (tessdata_best); the line names neither. Not a gate drop, and re…"
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 171,
    "text_sha": "9544708a1dda",
    "section": "what-to-take-with-you",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-10-27",
    "why": "says Indra beat Tesseract 5 without the model it beat: the registered comparison is Tesseract 5 (tessdata_best) on the paired 190-page sample; the gap to tessdata_fast is reported and not gated",
    "detail": "Carried forward by --carry-hand-edits on 2026-10-28: the writer produced this line again and the pack on disk records it removed by hand on 2026-10-27 (matched on its section). The earlier record reads: removed from this pack by hand on 2026-10-27 (UTC), after the writer ran and after the gates had counted: says Indra beat Tesseract 5 without the model it beat: the registered comparison is Tesseract 5 (tessdata_best) on the paired 190-page sample; the gap to tessdata_fast is reported and not gated. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "bullet",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 70,
    "text_sha": "0fda7f4e2fab",
    "figure": "0.38",
    "withheld": "one bullet removed from this pack by hand",
    "removed_utc": "2026-10-28",
    "why": "welds the sentence's \"with either model\" onto Indra's own 0.38%, so the line reads as if 0.38% were Tesseract's read CER; the page gives Indra 0.38% against tessdata_best 0.83% and tessdata_fast 0.87%",
    "detail": "removed from this pack by hand on 2026-10-28 (UTC), after the writer ran and after the gates had counted: welds the sentence's \"with either model\" onto Indra's own 0.38%, so the line reads as if 0.38% were Tesseract's read CER; the page gives Indra 0.38% against tessdata_best 0.83% and tessdata_fast 0.87%. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 208,
    "text_sha": "8bb08b00f356",
    "section": "the-diagnostic",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-10-28",
    "why": "gives the one-page split (coverage 0.01 to 0.99) without the page's label that the split is read from the per-page result rows after the run and is not a registered figure",
    "detail": "removed from this pack by hand on 2026-10-28 (UTC), after the writer ran and after the gates had counted: gives the one-page split (coverage 0.01 to 0.99) without the page's label that the split is read from the per-page result rows after the run and is not a registered figure. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 271,
    "text_sha": "c4373643a3be",
    "chip_id": "c-60b4d609",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-28",
    "why": "pairs 2.95% with 635: on the page 2.95% is the share of 21,614 pages that 638 pages make (counting three duplicate archive files); 635 distinct pages are 2.9%",
    "detail": "removed from this pack by hand on 2026-10-28 (UTC), after the writer ran and after the gates had counted: pairs 2.95% with 635: on the page 2.95% is the share of 21,614 pages that 638 pages make (counting three duplicate archive files); 635 distinct pages are 2.9%. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 294,
    "text_sha": "4dd0209b569e",
    "chip_id": "c-5612eef6",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-28",
    "why": "names one of the diagnostic's two settings: the page raises the detector's working long side (1,024 to 2,048 px) and its adaptive cap (2,048 to 4,096) together, and labels the one-page and 20-page split as read from the per-page result rows after the run, not a registered figure; the answer drops both",
    "detail": "removed from this pack by hand on 2026-10-28 (UTC), after the writer ran and after the gates had counted: names one of the diagnostic's two settings: the page raises the detector's working long side (1,024 to 2,048 px) and its adaptive cap (2,048 to 4,096) together, and labels the one-page and 20-page split as read from the per-page result rows after the run, not a registered figure; the answer drops both. Not a gate drop, and recorded here so the ledger adds up."
   }
  ]
 },
 "generated_utc": "2026-10-28T00:04:08Z"
}