{
 "schema_version": "1.0",
 "page_kind": "article",
 "pack_kind": "generate",
 "slug": "a-harness-that-rewrites-itself",
 "title": "A harness that rewrites itself",
 "dek": "Google Research's Harness² lets an AI coding agent rewrite its own instructions, skills and scripts between attempts at the same task; we read its code and its paper, ran neither, and found it worth understanding, but not something to run on the machine you keep your keys on.",
 "published": "2026-10-20",
 "series": [
  "guide"
 ],
 "licence": "CC BY 4.0",
 "status": "pending-judge",
 "pack_note": "no judge is seated, so every chip is held",
 "notice": "items with published:false are held and are not this page's published words",
 "source": {
  "url": "https://research.strata2signal.com/a-harness-that-rewrites-itself/",
  "md_url": "https://research.strata2signal.com/a-harness-that-rewrites-itself/index.md",
  "md_sha": "1027cef418cc85f38d2b54f686368992ee3279c92f5fed7adf3840ec1835beda",
  "html_sha": "469cfcf8d2e863a8dc11b465dd1f34fc5774de69affa21ae92108cc2583f5700",
  "anchors_sha": "aac4feb1e343d59b3bb03859212621d26d8a8fc500524f125c8f5a09748d7bf9"
 },
 "short": {
  "paragraph": "Harness² is a Python program that runs a coding agent (OpenCode or OpenAI's Codex, open-source agents that let a language model work in a folder through a terminal) on a task, has a second model session rewrite the agent's instructions, skills and scripts twice, re-runs the task on each rewrite, merges them from what the runs showed, and runs the result. At the default that is four agent runs and four improver sessions per task (three edit, one writes a lesson), roughly six to eight times the base agent's tokens by the paper's own chart, read by eye; the model's weights never change. On JobBench it can be pointed at a local model through OpenCode's own settings with no edits to Harness²; LAB's grader we could not settle, and WorkBuddy-Bench needs real edits. The paper tests no model that fits a 24 GB card: its only open-weights model, GLM-5, has 744 billion parameters, and the rest are closed models of undisclosed size. On the two domains the paper plots, one step is within about a point of running the plain agent four times and keeping the best answer, read by eye; that control is noisy, so neither is shown better. The privacy traps: unless you keep a global instruction file for OpenCode, the pinned OpenCode loads ~/.claude/CLAUDE.md (Claude Code's personal instructions) into every session, it fetches a model catalogue at every launch, and the agents run with your full shell and network. Our verdict: worth understanding, not something to run on the machine you keep your keys on.",
  "from_dek": false,
  "counts": {
   "words": 3831,
   "minutes": 17,
   "tables": 5,
   "kit": false
  },
  "bullets": []
 },
 "sections": [
  {
   "id": "the-words-this-page-leans-on",
   "heading": "The words this page leans on",
   "level": 2,
   "span": [
    2963,
    3912
   ],
   "chunks": [
    [
     2963,
     3912
    ]
   ],
   "chars": 949,
   "digest": "this section defines key terms including executor, improver, rollout, token, context window, best-of-n, and k. an executor is an agent with a frozen model, while an improver is a second agent session that rewrites harness files. k is described as the budget knob for rollouts and editing stages.",
   "digest_skipped": null
  },
  {
   "id": "what-a-harness-is",
   "heading": "What a harness is, and what Harness² changes",
   "level": 2,
   "span": [
    3912,
    5186
   ],
   "chunks": [
    [
     3912,
     5186
    ]
   ],
   "chars": 1274,
   "digest": "a harness is the system around a model that enables it to act as an agent. Harness² exposes eight editable parts, including four that are always in the agent's context window and three met along the way. model weights are kept fixed.",
   "digest_skipped": null
  },
  {
   "id": "how-one-step-works",
   "heading": "How one step works",
   "level": 2,
   "span": [
    5186,
    7697
   ],
   "chunks": [
    [
     5186,
     7697
    ]
   ],
   "chars": 2511,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "what-the-paper-reports",
   "heading": "What the paper reports",
   "level": 2,
   "span": [
    7697,
    10265
   ],
   "chunks": [
    [
     7697,
     10265
    ]
   ],
   "chars": 2568,
   "digest": "the paper reports results across three benchmarks using Gemini 3.5 Flash and GLM-5. it compares Harness² against best-of-k base controls. for example, on WorkBuddy-Bench, Gemini 3.5 Flash under OpenCode goes from 72.4 to 77.5 (+5.1). no model that fits a 24 GB card is tested.",
   "digest_skipped": null
  },
  {
   "id": "what-a-step-costs",
   "heading": "What a step costs",
   "level": 2,
   "span": [
    10265,
    12471
   ],
   "chunks": [
    [
     10265,
     12471
    ]
   ],
   "chars": 2206,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "running-it-on-your-own-models",
   "heading": "Running it on your own models",
   "level": 2,
   "span": [
    12471,
    16935
   ],
   "chunks": [
    [
     12471,
     16935
    ]
   ],
   "chars": 4464,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "what-it-will-touch",
   "heading": "What it will touch on your machine",
   "level": 2,
   "span": [
    16935,
    23023
   ],
   "chunks": [
    [
     16935,
     23023
    ]
   ],
   "chars": 6088,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "licences",
   "heading": "Licences",
   "level": 2,
   "span": [
    23023,
    24151
   ],
   "chunks": [
    [
     23023,
     24151
    ]
   ],
   "chars": 1128,
   "digest": null,
   "digest_skipped": "credits"
  },
  {
   "id": "should-you-run-it",
   "heading": "Should you run it?",
   "level": 2,
   "span": [
    24151,
    25784
   ],
   "chunks": [
    [
     24151,
     25784
    ]
   ],
   "chars": 1633,
   "digest": null,
   "digest_skipped": null
  },
  {
   "id": "how-to-check-our-work",
   "heading": "How to check our work",
   "level": 2,
   "span": [
    25784,
    27105
   ],
   "chunks": [
    [
     25784,
     27105
    ]
   ],
   "chars": 1321,
   "digest": null,
   "digest_skipped": "credits"
  },
  {
   "id": "who-ran-this",
   "heading": "Who ran this, and thanks",
   "level": 2,
   "span": [
    27105,
    27907
   ],
   "chunks": [
    [
     27105,
     27907
    ]
   ],
   "chars": 802,
   "digest": null,
   "digest_skipped": "credits"
  },
  {
   "id": "sources",
   "heading": "Sources",
   "level": 2,
   "span": [
    27907,
    28782
   ],
   "chunks": [
    [
     27907,
     28782
    ]
   ],
   "chars": 875,
   "digest": "this section lists the sources used, including the arXiv paper, various github repositories for Harness², OpenCode, and Codex, and model and dataset cards on Hugging Face.",
   "digest_skipped": null
  }
 ],
 "chips": [
  {
   "id": "c-39f8ba23",
   "q": "what terminology defines the agent and its editor?",
   "a": "The executor is the agent that performs tasks using a frozen model, while the improver acts as an editor by reading the executor's attempts and rewriting harness files.",
   "cites": [
    "the-words-this-page-leans-on"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  },
  {
   "id": "c-a3ac2322",
   "q": "how does a harness influence an agent's capability?",
   "a": "An execution harness is the system around a model that enables it to act as an agent, providing tools and context that influence its performance.",
   "cites": [
    "what-a-harness-is"
   ],
   "published": false,
   "publish_state": "held",
   "quote": "",
   "quote_cite": ""
  }
 ],
 "related": [
  {
   "slug": "three-at-the-table",
   "why": "shares ground with § What the paper reports · § Measured on our own machines"
  },
  {
   "slug": "the-compressed-photograph",
   "why": "shares ground with § What the paper reports · § The same question, down the whole ladder"
  },
  {
   "slug": "a-rig-your-friend-already-owns",
   "why": "shares ground with § What the paper reports · § What we ran, and the rule behind every number"
  }
 ],
 "thanks": "## Who ran this, and thanks {#who-ran-this}\n\nA small (human) team asked the question and approved this page for publication. One AI agent read the repository and the paper and produced a reading with file-and-line receipts; a second wrote this page from it, re-opening most cited files and every cited page; a third checked every claim against its source, and the page was corrected from that check. A few receipts rest on the first agent's reading alone: OpenCode's order for merging settings, the LAB README's licence wording, the WorkBuddy dataset card and our own list of products. The prose was drafted by Fable 5.1 and revised by Claude Opus 5.5. Thanks to the authors for releasing the code with its receipts in it, and for writing down that one of their own agents fetched its grading rubric.",
 "kit": null,
 "seat_class": {
  "writer": "gemma-class",
  "judge": null,
  "writer_runtime": "vllm"
 },
 "bench": {
  "bullets_written": 3,
  "bullets_kept": 0,
  "digests_written": 9,
  "digests_kept": 4,
  "chips_written": 8,
  "chips_kept": 2,
  "chips_grounded": 0,
  "chips_published": 0,
  "dropped_by": {
   "new_noun": 0,
   "figure": 1,
   "length": 0,
   "cite": 0,
   "judge": 0,
   "quote": 0,
   "directive": 0,
   "redaction": 1,
   "profanity": 0,
   "identifier": 0,
   "bare_figure": 0,
   "house_voice": 0,
   "editorial": 0,
   "hand-edit": 12
  },
  "new_noun_tokens_checked": [
   "0.3",
   "1",
   "1.3",
   "11",
   "140",
   "2.3",
   "24",
   "27",
   "3.0",
   "3.5",
   "3.6",
   "3090",
   "31.5",
   "36.0",
   "4.5",
   "5",
   "5.1",
   "72.4",
   "77.5",
   "95",
   "CLAUDE.md",
   "Codex",
   "Face",
   "Flash",
   "GB",
   "GLM-5",
   "GeForce",
   "Gemini",
   "Harness",
   "Hugging",
   "ID",
   "JobBench",
   "NVIDIA",
   "OpenAI-compatible",
   "OpenAI-style",
   "OpenCode",
   "RTX",
   "WorkBuddy-Bench",
   "best-of-4"
  ],
  "new_noun_tokens_withheld": 0,
  "figure_definition": "v3",
  "figure_boundary": "guarded",
  "drops": [
   {
    "kind": "bullet",
    "reason": "figure",
    "item": "the budget knob k is set at 1",
    "detail": "terminal token carries no figure"
   },
   {
    "kind": "chip",
    "reason": "redaction",
    "item": null,
    "item_chars": 258,
    "withheld": "a string carrying a fenced literal",
    "detail": ""
   },
   {
    "kind": "bullet",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 73,
    "text_sha": "d8f478823a5f",
    "figure": "27",
    "withheld": "one bullet removed from this pack by hand",
    "removed_utc": "2026-10-19",
    "why": "Bare figure, and the wrong subject: 27 million carries no unit (tokens), and it is not what a JobBench task uses but what one step of Harness² at k = 1 spends on one, for Gemini 3.5 Flash under OpenCode, against about 4.5 million for the base agent. It also drops the method: the page reads both off the paper's Figure 6 by eye, rough to about a quarter.",
    "detail": "removed from this pack by hand on 2026-10-19 (UTC), after the writer ran and after the gates had counted: Bare figure, and the wrong subject: 27 million carries no unit (tokens), and it is not what a JobBench task uses but what one step of Harness² at k = 1 spends on one, for Gemini 3.5 Flash under OpenCode, against about 4.5 million for the base agent. It also drops the method: the page reads both off the paper's Figure 6 by eye, rough to about a quarter. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "bullet",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 67,
    "text_sha": "b0dc6ad64214",
    "figure": "3.6",
    "withheld": "one bullet removed from this pack by hand",
    "removed_utc": "2026-10-19",
    "why": "Bare figure, and garbled: 1.3 to 3.6 carries no unit (points) and drops whose deviations they are. The page gives Table 1's run-to-run standard deviations, the typical spread between repeated runs, at 1.3 to 3.6 points; 'a run-to-run score' and 'between 1.3 to 3.6' are not the page's.",
    "detail": "removed from this pack by hand on 2026-10-19 (UTC), after the writer ran and after the gates had counted: Bare figure, and garbled: 1.3 to 3.6 carries no unit (points) and drops whose deviations they are. The page gives Table 1's run-to-run standard deviations, the typical spread between repeated runs, at 1.3 to 3.6 points; 'a run-to-run score' and 'between 1.3 to 3.6' are not the page's. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 261,
    "text_sha": "138f18e90ec8",
    "section": "how-one-step-works",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-10-19",
    "why": "Lists two of the step's four agent runs: one step at k = 1 is a base run, a candidate run on each of the two edits, and a final run, four rollouts in all (the code's R = 4k), so the digest's list reads as two runs where the page counts four.",
    "detail": "removed from this pack by hand on 2026-10-19 (UTC), after the writer ran and after the gates had counted: Lists two of the step's four agent runs: one step at k = 1 is a base run, a candidate run on each of the two edits, and a final run, four rollouts in all (the code's R = 4k), so the digest's list reads as two runs where the page counts four. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 227,
    "text_sha": "00da15865ef9",
    "section": "what-a-step-costs",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-10-19",
    "why": "Drops the benchmark and the method: the 4.5 million (the base agent) and 27 million (one step at k = 1) tokens are JobBench's, for Gemini 3.5 Flash under OpenCode (WorkBuddy-Bench reads about 0.3 and 2.3 million), read off the paper's Figure 6 by eye, rough to about a quarter. The hours-to-days line loses 'an illustration, not a measurement', and it names the card by its number alone where the page says one NVIDIA GeForce RTX 3090.",
    "detail": "removed from this pack by hand on 2026-10-19 (UTC), after the writer ran and after the gates had counted: Drops the benchmark and the method: the 4.5 million (the base agent) and 27 million (one step at k = 1) tokens are JobBench's, for Gemini 3.5 Flash under OpenCode (WorkBuddy-Bench reads about 0.3 and 2.3 million), read off the paper's Figure 6 by eye, rough to about a quarter. The hours-to-days line loses 'an illustration, not a measurement', and it names the card by its number alone where the page says one NVIDIA GeForce RTX 3090. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 296,
    "text_sha": "35fd0be2690f",
    "section": "running-it-on-your-own-models",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-10-19",
    "why": "Drops the page's 'We have not run this': the provider-block shape and its caveats are what the code and docs describe, each read in the source and not tried against a live server, and the digest gives them as what running on your own models involves.",
    "detail": "removed from this pack by hand on 2026-10-19 (UTC), after the writer ran and after the gates had counted: Drops the page's 'We have not run this': the provider-block shape and its caveats are what the code and docs describe, each read in the source and not tried against a live server, and the digest gives them as what running on your own models involves. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 273,
    "text_sha": "b095244a66d3",
    "section": "what-it-will-touch",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-10-19",
    "why": "Gives Harness² what the page gives the OpenCode it launches, and drops the condition: the page says that at run time Harness² itself makes no network call but a connection check to a Codex address (its setup clones repositories, installs packages and downloads datasets), while the pinned OpenCode loads Claude Code's personal instruction file (CLAUDE.md) into every session unless a global OpenCode AGENTS.md exists, fetches the model catalogue and forks the installs. Each row was read in the source, not observed on the wire.",
    "detail": "removed from this pack by hand on 2026-10-19 (UTC), after the writer ran and after the gates had counted: Gives Harness² what the page gives the OpenCode it launches, and drops the condition: the page says that at run time Harness² itself makes no network call but a connection check to a Codex address (its setup clones repositories, installs packages and downloads datasets), while the pinned OpenCode loads Claude Code's personal instruction file (CLAUDE.md) into every session unless a global OpenCode AGENTS.md exists, fetches the model catalogue and forks the installs. Each row was read in the source, not observed on the wire. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "digest",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 224,
    "text_sha": "ea1c6fd286dd",
    "section": "should-you-run-it",
    "withheld": "one section digest removed from this pack by hand",
    "removed_utc": "2026-10-19",
    "why": "Reads the page's 'we' as the paper's authors: it is this site that did not run Harness², because none of its products is an agent loop. The test also loses half its pass rule: one step must beat best-of-4 by at least 3.0 points on average and also beat the stock agent with 95 per cent confidence.",
    "detail": "removed from this pack by hand on 2026-10-19 (UTC), after the writer ran and after the gates had counted: Reads the page's 'we' as the paper's authors: it is this site that did not run Harness², because none of its products is an agent loop. The test also loses half its pass rule: one step must beat best-of-4 by at least 3.0 points on average and also beat the stock agent with 95 per cent confidence. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 192,
    "text_sha": "210f77146f44",
    "chip_id": "c-66d7a63c",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-19",
    "why": "Names no executor: these are Gemini 3.5 Flash's gains under OpenCode, at one step (k = 1). Under Codex the page's table gives Gemini 3.5 Flash +3.0 on WorkBuddy-Bench and +1.4 on JobBench, so an answer to a question about the model gives only its larger column.",
    "detail": "removed from this pack by hand on 2026-10-19 (UTC), after the writer ran and after the gates had counted: Names no executor: these are Gemini 3.5 Flash's gains under OpenCode, at one step (k = 1). Under Codex the page's table gives Gemini 3.5 Flash +3.0 on WorkBuddy-Bench and +1.4 on JobBench, so an answer to a question about the model gives only its larger column. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 287,
    "text_sha": "342aaf08a7ff",
    "chip_id": "c-8a8a4064",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-19",
    "why": "Drops 'by eye' from all six token figures: the page reads them off the paper's Figure 6, a log axis, by eye and rough to about a quarter, for Gemini 3.5 Flash under OpenCode; the paper prints no table for them.",
    "detail": "removed from this pack by hand on 2026-10-19 (UTC), after the writer ran and after the gates had counted: Drops 'by eye' from all six token figures: the page reads them off the paper's Figure 6, a log axis, by eye and rough to about a quarter, for Gemini 3.5 Flash under OpenCode; the paper prints no table for them. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 261,
    "text_sha": "ead6f358697d",
    "chip_id": "c-ce98aa5b",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-19",
    "why": "States as requirements what the page says it has not run: the provider-block shape is what the code and docs describe ('We have not run this'), read in the source and not tried against a live server.",
    "detail": "removed from this pack by hand on 2026-10-19 (UTC), after the writer ran and after the gates had counted: States as requirements what the page says it has not run: the provider-block shape is what the code and docs describe ('We have not run this'), read in the source and not tried against a live server. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 273,
    "text_sha": "d3c0cdb3887d",
    "chip_id": "c-4fbd065c",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-19",
    "why": "'Without prior validation' is not the page's: the page says validation checks only that Python compiles and named files exist, and that no test reads a script before it runs. 'Directly on the machine' also drops the exception the page names: WorkBuddy-Bench rollouts get a container each.",
    "detail": "removed from this pack by hand on 2026-10-19 (UTC), after the writer ran and after the gates had counted: 'Without prior validation' is not the page's: the page says validation checks only that Python compiles and named files exist, and that no test reads a script before it runs. 'Directly on the machine' also drops the exception the page names: WorkBuddy-Bench rollouts get a container each. Not a gate drop, and recorded here so the ledger adds up."
   },
   {
    "kind": "chip",
    "reason": "hand-edit",
    "item": null,
    "item_chars": 227,
    "text_sha": "3018d0caac07",
    "chip_id": "c-80e519e1",
    "withheld": "one chip removed from this pack by hand",
    "removed_utc": "2026-10-19",
    "why": "Garbles the pass rule: the page's test passes only if one step beats best-of-4 by at least 3.0 points on average and beats the stock agent with 95 per cent confidence, not 'with the stock agent performing at a level with 95 per cent confidence'. The rule says when the test would pass, as a design, not what makes the method worth testing.",
    "detail": "removed from this pack by hand on 2026-10-19 (UTC), after the writer ran and after the gates had counted: Garbles the pass rule: the page's test passes only if one step beats best-of-4 by at least 3.0 points on average and beats the stock agent with 95 per cent confidence, not 'with the stock agent performing at a level with 95 per cent confidence'. The rule says when the test would pass, as a design, not what makes the method worth testing. Not a gate drop, and recorded here so the ledger adds up."
   }
  ]
 },
 "generated_utc": "2026-10-19T00:47:59Z"
}