#!/usr/bin/env python3 """engines.py -- how each engine is launched (argv, environment, config) and how its log is read for G0. llama.cpp at 3cf03257 (`llama-server`, flags read from common/arg.cpp at that commit): -m (shard 2 loads beside it) --host 127.0.0.1 --port P -ngl 99 --n-cpu-moe N -c 131072 -fa on -ctk q8_0 -ctv q8_0 -lm mmap --lazy-mode on -fit off -np 1 --cache-ram 0 --no-webui --jinja -lv 4 -t 8 -tb 8 RECON-v1.md line 446 names -ngl 99, --n-cpu-moe (fitted, then frozen), -lm mmap --lazy-mode on, -fa on and 8-bit KV. -t 8 -tb 8 (PREREG A3; AUDIT-HARNESS I-1, ruled (b) in ledger D-20261002-048): llama.cpp's default thread count comes from common_cpu_get_num_math(), whose P-core loop (common/common.cpp:182-195) skips every other core because it assumes SMT siblings sit side by side; the Core Ultra 9 290HX Plus has 8 P-cores and no SMT, so the default was 4 (L-IQ3 speed a1's server.log: "n_threads = 4 (n_threads_batch = 4) / 24"). Every llama.cpp launch from A3 on names the 8 P-cores; L-IQ3's speed a1 stands as recorded at 4. Each start's own thread line is parsed into every row. The rest, each with its reason: -c 131072 (the 128K slice must fit; Strata's setup writes the same context for a 24 GB card, setup.py:1643); -fit off (its default is ON and would re-fit "unset" parameters: the frozen N must be the N that runs); -np 1 (one sequence, as Strata serves); --cache-ram 0 (the host prompt cache defaults to 8 GiB of RAM charged to the unit, and every prompt opens with a fresh nonce, so it could never hit); --no-webui; --jinja (the default, stated). The API key is passed in LLAMA_API_KEY (its env name in arg.cpp), never on the command line. The fa-off variants (PREREG A4, ledger D-20261002-056): an arm row's "llama_flags" may name `flash_attn` (on | off), `cache_type_v` and `ctx`; every other flag is the line above. `-fa off` cannot keep the 8-bit V cache: 3cf03257's llama_init refuses a quantized V cache without flash attention (src/llama-context.cpp:3732-3740, "quantized V cache requires flash_attn to be enabled") and the server never reaches /health, so the fa-off rows name `-ctv f16` (llama.cpp's default cache type) and keep `-ctk q8_0` (the K cache has no such rule with flash attention off). `llama_line_problem` refuses that pair before any launch. Each start's own `flash_attn = ...` line and KV-cache types are read from its server.log (`parse_attention`), so a label never stands in for the engine's own word. The llama.cpp controls (PREREG A7, ledger D-20261004-095): "llama_flags" may also name `batch` and `ubatch` (`-b N -ub M`, placed straight after `-c`; None, the default, passes no flag and leaves llama.cpp's own 2048 / 512, common/common.h:451-452) and `threads` (`-t N -tb N` together; 8 by A3). An arm that names none of them keeps its argv byte for byte and its recorded three-key line (`llama_line`). 3cf03257 clamps the micro-batch to the logical batch (src/llama-context.cpp:247) and the logical batch to the context (:245), so `llama_line_problem` refuses a micro-batch above the logical batch in force and a logical batch above the context: the engine never runs a batch other than the one a table names. Each start's own `n_batch = ...` / `n_ubatch = ...` lines (src/llama-context.cpp: 309-310) are read from its server.log (`parse_batch`); `control_condition` reads an attempt's condition from its argv. Strata at 8fc40dde, WITHOUT setup.py: `serve/server.py --engine strata --config --port P`, run from the source tree with the lead's Python, the config written here with exactly the engine arguments setup.py:1801-1824 writes for this box (no vision, no MCP, no control vector), host 127.0.0.1, and STRATA_API_KEY in the environment (server.py:1661's default; never in the config or the command line): --pack --native --ple-gguf --expert-profile /data/expert-profile.bin --expert-cache auto --prefill auto --spec 4 --spec-min-p 0.5 --mtp --max-context 131072 --kv int8 [--kv-resident 32768 when MemTotal GiB >= setup's ram_gb + 1.80 + 1: Q2_0 yes, IQ3_S no on 61.2 GiB] S-IQ3-noMTP drops `--mtp ` (RECON line 448: "--spec 4 without --mtp"). G0 (PREREG section 2): "/health answers; the log shows the native AVX2 line and no control-vector line". For Strata the log lines are generate.cpp's own (read at 8fc40dde): the AVX2 line :1361, the control-vector lines :846 and :1485, the INFO line's cvec= field :3491, the CPU refusal (src/kernels/cpu/expert.cpp:375), the PCIe probe :1351, the expert-pool worker count :2430 (36fa455e: :1752 and :3180, the same words). The AVX2 line reads "...run on AVX-2 (multi-token ...)" by default and "...run on ggml-cpu vec_dot (STRATA_NO_IQ256 set)" when that variable is set (:1358-1363); only the first is the native line (AUDIT-HARNESS I-2), and the launch drops every STRATA_* variable but the API key (`server_environment`), so a stray STRATA_NO_IQ256 never reaches the engine. For llama.cpp G0 is /health plus the kit's device gate (one CUDA device, no Vulkan: engine_llamaserver.device_list) and the offload line (gates.all_layers), the estate's Vulkan trap (memory: ollama-enumerates-the-second-board). Strata v0.1.39 (6f32ec07, the exploratory S-NEW arm; STRATA-DELTA-2 sections 6.1-6.3, fixes H-1 to H-3). Three log lines changed or appeared after 36fa455e, read at 6f32ec07: - the PCIe probe line names its bursts, "PCIe probe: %.1f GB/s host->device (best of %s) -> pcie_frac %.2f (default %.2f)" (generate.cpp:2046-2047, since v0.1.37 #485). RE_PCIE takes the burst list as optional and non-capturing, so its four groups (GB/s, pcie_frac, default, the failed probe's default) read both forms (H-1); - the hit-rate line may carry the #588 suffix "; N more read by the GPU over PCIe or from another GPU (X% of all M routed)" (generate.cpp:7401-7413), printed only when that count is above 0. The hit rate itself keeps its meaning (the VRAM share of the experts the CPU pool looked up), so RE_HIT is unchanged; RE_HIT_OFFLOAD reads the suffix straight after the hit line it belongs to, and /metrics' new per-request "pcie_share" is kept (H-2); - a start whose GPU stage holds every expert in VRAM logs "(100% VRAM resident: zero-doorbell graph)" on its "strata verify: window up to ..." line (src/core/verify.cpp:527-529) and then never calls the host pool, the only place hits and lookups are counted, so the hit-rate line is not printed at all. A request slice that holds a serve line but no hit-rate line is recorded as hit_rate_line "absent", never a silent None, and G0 keeps the start's zero-doorbell line (H-3). One 24 GB card cannot hold IQ3_S's 46.84 GiB of experts, so on this box the line is expected to stay absent; it is recorded either way. The engine's environment, per arm (H-5; STRATA-DELTA-2 C-12 and section 9 item 1). `server_environment` drops every STRATA_* variable from the SERVER's environment (AUDIT-HARNESS I-2), so nothing typed in a shell reaches the engine; an arm that must hand the engine a variable names it in its row's "engine_env", and `strata_config` writes it into the config's "env", which v0.1.39's server copies into the engine's environment (serve/server.py:1560-1575, `child_env`: `env[str(k)] = str(v)`). Only the names and values in STRATA_ENGINE_ENV_ALLOWED pass (`engine_env_problem`): today one, STRATA_NO_LARGEPAGES=1, which keeps the expert arena on the 4 KB pages S-HEAD ran on instead of v0.1.39's transparent huge pages (src/core/pinned.cu:202-247, #650; this laptop's THP reads madvise/madvise, issue #771's slow-arena case). An arm with no "engine_env" gets the config it had before, byte for byte (no "env" key). The engine's own word is read back: its "strata generate: expert arena: " line (generate.cpp:3169 at 6f32ec07; the note from pinned.cu) is kept in G0's facts with the page backing it names (`arena_pages`), and `engine_env_readback` says when a STRATA_NO_LARGEPAGES arm's line does not read "large pages skipped (STRATA_NO_LARGEPAGES)" (the variable never reached the engine). """ import json import os import re import subprocess import common as C LLAMA_CTX = 131072 STRATA_CTX = 131072 RE_AVX2 = re.compile(r"this CPU has no AVX-512: the expert kernels run on AVX-2 $") RE_CVEC = re.compile(r"control vector mode = [^\n]*|strata generate: control vector: [^\n]*") RE_CPU_REFUSED = re.compile(r"this CPU cannot run the expert kernel[^\n]*") #: 36fa455e: "PCIe probe: 28.7 GB/s host->device -> pcie_frac 0.55 (default 0.55)"; v0.1.37+ adds " (best of )" after host->device (H-1). Groups: 1 GB/s, 2 pcie_frac, 3 default, 4 failed default. RE_PCIE = re.compile(r"PCIe probe: ([0-9.]+) GB/s host->device(?: \(best of [^)]*$)? -> pcie_frac ([0-9.]+) " r"$default ([0-9.]+)$|PCIe probe failed -> pcie_frac default ([0-9.]+)") RE_INFO = re.compile(r"^INFO (.*)$", re.M) RE_HIT = re.compile(r"decode expert cache hit rate: ([0-9.]+)% $(\d+) hits / (\d+) lookups$") #: v0.1.39 #588: the suffix that may follow a RE_HIT match on the same line (H-2); matched at that match's end, so it #: is always the suffix of the hit line read, never another request's. Groups: 1 offloaded, 2 percent, 3 routed. RE_HIT_OFFLOAD = re.compile(r"; (\d+) more read by the GPU over PCIe or from another GPU $([0-9.]+)% of all " r"(\d+) routed$") #: v0.1.39 #646: the start's verifier line when every expert of the stage is in VRAM (H-3). RE_ZERO_DOORBELL = re.compile(r"strata verify: window up to \d+ tokens, [0-9.]+ MiB of device buffers $100% VRAM " r"resident: zero-doorbell graph$") #: The expert arena's start-up line (H-5): generate.cpp:3169 at 6f32ec07 prints "strata generate: expert arena: " #: once, after the arena is loaded; the note says how its pages are backed (src/core/pinned.cu). The same words later #: start the line that hands VRAM-held experts back to the OS (generate.cpp:4299, " GiB of VRAM-held experts handed #: back"), which is not the start-up line. RE_ARENA = re.compile(r"strata generate: expert arena: (?![0-9.]+ GiB of VRAM-held)([^\n]*)") #: (what the note says, the backing it names), first match wins; read at 6f32ec07's src/core/pinned.cu (Linux branch #: :202-247) and 36fa455e's log ("MAP_HUGETLB unavailable (no hugetlb pool configured?); using 4 KB pages"). ARENA_PAGES = (("large pages skipped (STRATA_NO_LARGEPAGES)", "4 KB pages (STRATA_NO_LARGEPAGES)"), ("transparent huge pages requested (MADV_HUGEPAGE)", "transparent huge pages (MADV_HUGEPAGE)"), ("hugetlb 2 MB pages", "hugetlb 2 MB pages"), ("STRATA_ARENA_MMAP", "mapped read-only (STRATA_ARENA_MMAP)"), ("using 4 KB pages", "4 KB pages")) #: H-5: the only variables an arm row's "engine_env" may hand Strata's engine, each with the values it may take. STRATA_ENGINE_ENV_ALLOWED = {"STRATA_NO_LARGEPAGES": ("1",)} RE_SERVE = re.compile(r"strata serve: prompt (\d+) tokens = (\d+) reused \+ (\d+) read in ([0-9.]+) ms $([0-9.]+) " r"tok/s$, (\d+) generated in ([0-9.]+) ms $([0-9.]+) tok/s$, drafts accepted (\d+) of (\d+)") RE_UUID = re.compile(r"GPU-[0-9a-fA-F]{8}-[0-9a-fA-F-]{27}") #: llama.cpp 3cf03257 common_params_print_info / system_info, e.g. "system_info: n_threads = 4 (n_threads_batch = 4) #: / 24 | CUDA : ..." (L-IQ3 speed a1 server.log line 7) and the start banner "build 1 (3cf0325) with GNU ..." (line 2). RE_LLAMA_THREADS = re.compile(r"n_threads = (\d+) $n_threads_batch = (\d+)$(?: / (\d+))?") RE_LLAMA_BUILD = re.compile(r"build (\d+) $([0-9a-f]{7,40})$") #: Strata generate.cpp:2430 at 8fc40dde (:3180 at 36fa455e): "strata generate: %d expert-pool workers%s%s". RE_STRATA_WORKERS = re.compile(r"strata generate: (\d+) expert-pool workers( \+ the host thread)?( $UNUSED: --no-pool$)?") #: The thread count every llama.cpp launch names (PREREG A3): the 290HX Plus's 8 P-cores. LLAMA_THREADS = 8 #: The cache types `-ctk` / `-ctv` accept at 3cf03257 (common/arg.cpp:304-314, kv_cache_types), and the quantized ones #: among them: a quantized V cache needs flash attention on (src/llama-context.cpp:3732-3740). LLAMA_CACHE_TYPES = ("f32", "f16", "bf16", "q8_0", "q4_0", "q4_1", "iq4_nl", "q5_0", "q5_1") LLAMA_QUANTIZED_CACHE_TYPES = ("q8_0", "q4_0", "q4_1", "iq4_nl", "q5_0", "q5_1") #: The registered line's settings (RECON-v1.md line 446: -fa on and 8-bit KV; PREREG A3: -t 8 -tb 8); an arm row's #: "llama_flags" overrides only the keys it names (PREREG A4's fa-off variants; PREREG A7's micro-batch control and #: thread reading). `batch` / `ubatch` None: no -b / -ub flag is passed, so llama.cpp's own defaults stand. LLAMA_LINE = {"flash_attn": "on", "cache_type_v": "q8_0", "ctx": LLAMA_CTX, "batch": None, "ubatch": None, "threads": LLAMA_THREADS} #: The keys every recorded line carries (`llama_line`); PREREG A7's keys join the record only when an arm names them, #: so the line recorded for every earlier arm keeps exactly these three keys. LLAMA_LINE_RECORDED = ("flash_attn", "cache_type_v", "ctx") #: 3cf03257's logical batch and micro-batch when no -b / -ub is passed (common/common.h:451-452). LLAMA_DEFAULT_BATCH = 2048 LLAMA_DEFAULT_UBATCH = 512 def _positive_int(v): return isinstance(v, int) and not isinstance(v, bool) and v >= 1 def llama_line_problem(flash_attn="on", cache_type_v="q8_0", ctx=LLAMA_CTX, batch=None, ubatch=None, threads=LLAMA_THREADS): """None when llama.cpp 3cf03257 runs this line as named, else the sentence that refuses it before any launch.""" if flash_attn not in ("on", "off"): return f"-fa {flash_attn!r}: the harness names on or off, never auto (auto lets the engine choose the arm)" if cache_type_v not in LLAMA_CACHE_TYPES: return f"-ctv {cache_type_v!r} is not one of 3cf03257's cache types {', '.join(LLAMA_CACHE_TYPES)}" if flash_attn == "off" and cache_type_v in LLAMA_QUANTIZED_CACHE_TYPES: return (f"-fa off with -ctv {cache_type_v}: 3cf03257 refuses a quantized V cache without flash attention " f"(src/llama-context.cpp:3732-3740, \"quantized V cache requires flash_attn to be enabled\"); the " f"server would exit before /health") if not isinstance(ctx, int) or isinstance(ctx, bool) or ctx < 4096: return f"-c {ctx!r}: a context of at least 4096 tokens, as an int" # PREREG A7: the batch keys and the thread count for flag, v in (("-b", batch), ("-ub", ubatch)): if v is not None and not _positive_int(v): return f"{flag} {v!r}: a positive int, or None to pass no flag (llama.cpp's own default)" if not _positive_int(threads): return f"-t / -tb {threads!r}: a positive int (PREREG A3 names 8)" in_force = batch if batch is not None else LLAMA_DEFAULT_BATCH if ubatch is not None and ubatch > in_force: src = "-b" if batch is not None else "no -b, so 3cf03257's default, common/common.h:451" return (f"-ub {ubatch} above the logical batch in force, {in_force} ({src}): the engine would clamp the " f"micro-batch to it (src/llama-context.cpp:247) and run another micro-batch than the line names") if batch is not None and batch > ctx: return (f"-b {batch} above -c {ctx}: 3cf03257 clamps the logical batch to the context " f"(src/llama-context.cpp:245), so the engine would run another batch than the line names") return None def llama_line(overrides=None): """(line, problem): LLAMA_LINE with an arm row's "llama_flags" applied, and the sentence that refuses it (a key the harness does not override, or a line 3cf03257 would refuse or silently change), else None. The line holds the three recorded keys plus any PREREG A7 key the row names, so an arm that names none records the same line as before A7.""" overrides = dict(overrides or {}) extra = sorted(set(overrides) - set(LLAMA_LINE)) full = dict(LLAMA_LINE, **{k: v for k, v in overrides.items() if k in LLAMA_LINE}) line = {k: v for k, v in full.items() if k in LLAMA_LINE_RECORDED or k in overrides} if extra: return line, f"llama_flags names {', '.join(extra)}: the harness overrides only {', '.join(LLAMA_LINE)}" return line, llama_line_problem(**line) def llama_argv(llama_server, shard1, port, n_cpu_moe, flash_attn="on", cache_type_v="q8_0", ctx=LLAMA_CTX, batch=None, ubatch=None, threads=LLAMA_THREADS): """The llama-server argv. The keywords default to the registered line (LLAMA_LINE); an arm row's "llama_flags" passes others (PREREG A4: `-fa off -ctv f16`; PREREG A7: `-b 4096 -ub 4096`, or 16 threads). The defaults add no flag, so every arm that names none of A7's keys gets the argv it had before A7, byte for byte. A line `llama_line_problem` refuses raises ValueError.""" why = llama_line_problem(flash_attn, cache_type_v, ctx, batch, ubatch, threads) if why: raise ValueError(why) batch_flags = (["-b", str(batch)] if batch is not None else []) + (["-ub", str(ubatch)] if ubatch is not None else []) return [llama_server, "-m", shard1, "--host", "127.0.0.1", "--port", str(port), "-ngl", "99", "--n-cpu-moe", str(int(n_cpu_moe)), "-c", str(ctx)] + batch_flags + ["-fa", flash_attn, "-ctk", "q8_0", "-ctv", cache_type_v, "-lm", "mmap", "--lazy-mode", "on", "-fit", "off", "-np", "1", "--cache-ram", "0", "--no-webui", "--jinja", # -lv 4: at 3cf03257 the default threshold (3) hides the library's INFO lines, so neither # "using device CUDA0 (" (src/llama.cpp:306) nor "offloaded N/M layers to GPU" (src/llama-model.cpp:1846) # reached the log and G0 read devices [] / offload unread on the first L-IQ3 fit (2026-10-02 20:44Z). "-lv", "4", # PREREG A3: the 8 P-cores, for generation and for batch (prompt) processing alike (PREREG A7's # L-IQ3-t16 names 16 for both). "-t", str(threads), "-tb", str(threads)] def _flag_value(argv, *names): """The value after the last of `names` in an argv, or None.""" argv = list(argv or []) out = None for i, a in enumerate(argv[:-1]): if a in names: out = argv[i + 1] return out def argv_batch(argv): """{"b", "ub"} as an argv names them (PREREG A7), each an int, or None where it names none (the engine's default).""" out = {} for key, names in (("b", ("-b", "--batch-size")), ("ub", ("-ub", "--ubatch-size"))): v = _flag_value(argv, *names) out[key] = int(v) if v is not None and v.isdigit() else None return out def _a_or_an(spoken): """The article for a number as it is read aloud: "an" for 8..., 11... and 18... (eight, eleven, eighteen).""" lead = spoken.split(",")[0] return "an" if lead[:1] == "8" or lead in ("11", "18") else "a" def control_condition(argv): """PREREG A7: a llama.cpp attempt's condition as every table prints it beside L-IQ3-t8, read from that attempt's own argv (never from its arm row): `llama.cpp at a 4,096-token micro-batch` for `-ub 4096`, `llama.cpp at 16 threads` for `-t 16 -tb 16` (a split pair reads `-t 16 / -tb 12 threads`), both joined by "; " when both are named. None when the argv names no -ub and A3's thread count (8) for both -t and -tb.""" parts = [] ub = argv_batch(argv)["ub"] if ub is not None: n = f"{ub:,}" parts.append(f"llama.cpp at {_a_or_an(n)} {n}-token micro-batch") th = argv_threads(argv or []) if (th["t"], th["tb"]) != (LLAMA_THREADS, LLAMA_THREADS) and (th["t"], th["tb"]) != (None, None): parts.append(f"llama.cpp at {th['t']} threads" if th["t"] == th["tb"] else f"llama.cpp at -t {th['t']} / -tb {th['tb']} threads") return "; ".join(parts) or None #: llama.cpp 3cf03257's own batch lines (src/llama-context.cpp:309-310): "llama_context: n_batch = N" and #: "llama_context: n_ubatch = N", each after the clamps at :245 and :247, so they state what the engine runs. RE_LLAMA_NBATCH = re.compile(r"llama_context: n_batch\s+=\s+(\d+)") RE_LLAMA_NUBATCH = re.compile(r"llama_context: n_ubatch\s+=\s+(\d+)") def parse_batch(text): """{n_batch, n_ubatch, line}: what a start's own server.log says it runs (PREREG A7), each None when the log carries no such line (a start that died before its context was made).""" text = text or "" b, ub = RE_LLAMA_NBATCH.search(text), RE_LLAMA_NUBATCH.search(text) return {"n_batch": int(b.group(1)) if b else None, "n_ubatch": int(ub.group(1)) if ub else None, "line": (f"n_batch = {b.group(1) if b else '?'}, n_ubatch = {ub.group(1) if ub else '?'}" if (b or ub) else None)} def argv_threads(argv): """{"t", "tb"} as the argv names them (None where it names none: the engine's own default).""" out = {"t": None, "tb": None} for i, a in enumerate(argv[:-1]): if a in ("-t", "--threads"): out["t"] = int(argv[i + 1]) elif a in ("-tb", "--threads-batch"): out["tb"] = int(argv[i + 1]) return out def parse_threads(engine, text, argv=None): """The thread count the engine's own log states, with the line it was read from (PREREG A3: on every row). llama.cpp: `n_threads = N (n_threads_batch = M) / L`; Strata: `N expert-pool workers[ + the host thread]`.""" text = text or "" if engine == "llama.cpp": m = RE_LLAMA_THREADS.search(text) req = argv_threads(argv or []) if not m: return {"engine": engine, "n_threads": None, "n_threads_batch": None, "line": None, "requested": req, "why": "no `n_threads = N (n_threads_batch = M)` line in the server log"} line = text[max(0, text.rfind("\n", 0, m.start()) + 1):m.end()] rec = {"engine": engine, "n_threads": int(m.group(1)), "n_threads_batch": int(m.group(2)), "logical_cpus": int(m.group(3)) if m.group(3) else None, "line": line.strip()[-200:], "requested": req, "source": "server.log (common_params_print_info system_info)"} rec["matches_request"] = (req["t"] is None or req["t"] == rec["n_threads"]) and \ (req["tb"] is None or req["tb"] == rec["n_threads_batch"]) return rec m = RE_STRATA_WORKERS.search(text) if not m: return {"engine": engine, "workers": None, "host_thread": None, "line": None, "why": "no `N expert-pool workers` line in the engine log (Strata takes no thread flag; its pool " "picks one worker per physical core, src/kernels/cpu/pool.cpp:58-93)"} return {"engine": engine, "workers": int(m.group(1)), "host_thread": bool(m.group(2)), "pool_unused": bool(m.group(3)), "line": m.group(0), "source": "strata-engine.log (generate.cpp)"} #: llama.cpp 3cf03257's own attention lines: src/llama-context.cpp:312 "llama_context: flash_attn = " (llama_flash_attn_type_name, src/llama.cpp:38-45) and the KV cache's "llama_kv_cache: size = ... K (): #: ..., V (): ..." (the first one: the attention layers' cache; the indexer's K-only cache follows it). RE_LLAMA_FA = re.compile(r"llama_context: flash_attn\s+=\s+(\w+)") RE_LLAMA_KV = re.compile(r"llama_kv_cache: size = [^\n]*?K $(\w+)$: [^\n]*?V $(\w+)$") def parse_attention(text): """{flash_attn, fa_line, k_type, v_type, kv_line}: what the start's own server.log says about its attention, each None when the log carries no such line (a start that died before the context was made).""" text = text or "" fa, kv = RE_LLAMA_FA.search(text), RE_LLAMA_KV.search(text) def line(m): return text[text.rfind("\n", 0, m.start()) + 1:m.end()].strip()[-200:] if m else None return {"flash_attn": fa.group(1) if fa else None, "fa_line": line(fa), "k_type": kv.group(1) if kv else None, "v_type": kv.group(2) if kv else None, "kv_line": line(kv)} def parse_llama_build(text): """{"build", "commit_short", "line"} from llama-server's start banner, or None when the log carries none.""" m = RE_LLAMA_BUILD.search(text or "") if not m: return None return {"build": int(m.group(1)), "commit_short": m.group(2), "line": m.group(0)} def commit_matches(found, expected): """A full or abbreviated (>= 7 hex) commit against the expected one, prefix-wise.""" if not found or not expected or len(found) < 7 or len(expected) < 7: return False return found.startswith(expected) or expected.startswith(found) def cmake_home(exe, levels=3): """(CMakeCache.txt path, CMAKE_HOME_DIRECTORY) of the build tree an executable sits in, walking up from its folder at most `levels` folders; (None, None) when no CMakeCache.txt is found.""" d = os.path.dirname(os.path.realpath(exe)) for _ in range(levels): cache = os.path.join(d, "CMakeCache.txt") if os.path.exists(cache): try: for line in open(cache, encoding="utf-8", errors="replace"): if line.startswith("CMAKE_HOME_DIRECTORY:"): return cache, line.split("=", 1)[1].strip() except OSError: pass return cache, None d = os.path.dirname(d) return None, None def git_head(src): try: r = subprocess.run(["git", "-C", src, "rev-parse", "HEAD"], capture_output=True, text=True, timeout=30) return r.stdout.strip() if r.returncode == 0 else None except (OSError, subprocess.TimeoutExpired): return None def build_identity(exe, expected_commit, src=None): """AUDIT-HARNESS I-8: the engine's identity READ, never asserted. {ok, why, exe, cmake_cache, cmake_home, src, git_head, expected_commit}. The executable's build tree must name `src` (or, when `src` is None, a source tree) as its CMAKE_HOME_DIRECTORY, and that tree's HEAD must be the expected commit.""" cache, home = cmake_home(exe) rec = {"exe": exe, "exe_realpath": os.path.realpath(exe), "cmake_cache": cache, "cmake_home": home, "src": src, "expected_commit": expected_commit} why = [] if cache is None: why.append(f"no CMakeCache.txt beside or above {exe}: not a tree the lead built from source") elif home is None: why.append(f"{cache} names no CMAKE_HOME_DIRECTORY") tree = src or home if src is not None and home is not None and os.path.realpath(home) != os.path.realpath(src): why.append(f"the executable was built from {home}, not from the source tree given ({src})") head = git_head(tree) if tree else None rec["git_head"] = head if head is None: why.append(f"`git -C {tree} rev-parse HEAD` read nothing") elif not commit_matches(head, expected_commit): why.append(f"the source tree's HEAD is {head}, not the arm's commit {expected_commit}") rec["ok"] = not why rec["why"] = "; ".join(why) or "the build tree names the source tree, whose HEAD is the arm's commit" return rec def server_environment(base, server_env, key_env, key): """(env, dropped names): the server's environment. AUDIT-HARNESS I-2: every STRATA_* variable but the API key's is dropped (STRATA_NO_IQ256 swaps the native kernels for vec_dot); proxies are dropped as before.""" env = dict(base) dropped = [] for k in list(env): if k in ("http_proxy", "https_proxy", "HTTP_PROXY", "HTTPS_PROXY", "ALL_PROXY", "all_proxy", "NO_PROXY") or \ (k.startswith("STRATA_") and k != key_env): dropped.append(k) env.pop(k) env.update(server_env or {}) env[key_env] = key return env, sorted(dropped) def strata_engine_args(*, quant, pack, mtp_rt, strata_src, mtp=True, mem_total_gib=None, model_root=C.MODEL_ROOT): total = mem_total_gib if mem_total_gib is not None else C.read_meminfo().get("MemTotal", 0) / C.GIB args = ["--pack", pack, "--native", C.shard_path(quant, 1, model_root), "--ple-gguf", C.shard_path(quant, 2, model_root), "--expert-profile", os.path.join(strata_src, "data", "expert-profile.bin"), "--expert-cache", "auto", "--prefill", "auto", "--spec", "4", "--spec-min-p", "0.5"] if mtp: args += ["--mtp", mtp_rt] args += ["--max-context", str(STRATA_CTX), "--kv", "int8"] if C.kv_resident_on(quant, total): args += ["--kv-resident", "32768"] return args def engine_env_problem(env): """None when an arm row's "engine_env" (H-5) may reach Strata's engine, else the sentence that refuses it before any launch: it must be a mapping of names in STRATA_ENGINE_ENV_ALLOWED to one of the string values listed there. None or an empty mapping hands the engine nothing.""" if env is None: return None if not isinstance(env, dict): return f"engine_env {env!r}: a mapping of variable names to string values" allowed = ", ".join(f"{k}={'|'.join(v)}" for k, v in sorted(STRATA_ENGINE_ENV_ALLOWED.items())) for k, v in sorted(env.items(), key=lambda kv: str(kv[0])): if k not in STRATA_ENGINE_ENV_ALLOWED: return (f"engine_env names {k!r}: the harness hands Strata's engine only {allowed} (H-5; every other " f"STRATA_* variable stays out, AUDIT-HARNESS I-2)") if not isinstance(v, str) or v not in STRATA_ENGINE_ENV_ALLOWED[k]: return f"engine_env {k}={v!r}: the harness allows only {allowed} (a string)" return None def strata_config(*, exe, args, strata_src, pack, model_name, log, lib_dirs, port, env=None): """The config server.py reads (setup.py:1825-1829's keys), minus every optional feature: no "vision", no "mcp_servers", no "api_key" (it rides the environment), no "gpu" (one card), host 127.0.0.1. `env` is the arm row's "engine_env" (H-5): checked by `engine_env_problem` (ValueError when refused) and written as the config's "env", which the server hands the engine; with none, the config carries no "env" key, as before H-5.""" why = engine_env_problem(env) if why: raise ValueError(why) cfg = {"exe": exe, "args": list(args), "cwd": strata_src, "tokenizer": os.path.join(pack, "tokenizer"), "model_name": model_name, "log": log, "lib_dirs": list(lib_dirs), "port": int(port), "host": "127.0.0.1"} if env: cfg["env"] = {k: env[k] for k in sorted(env)} return cfg def arena_pages(arena_line): """The page backing an expert-arena note names (ARENA_PAGES), "other" for a note none of them matches, or None when there is no arena line.""" if arena_line is None: return None return next((what for needle, what in ARENA_PAGES if needle in arena_line), "other") def engine_env_readback(env, g0): """H-5: None when the engine's own arena line agrees with the env the config handed it (or the env names nothing the line can show), else the sentence that says it does not: a STRATA_NO_LARGEPAGES arm whose start logged another page backing, or no arena line at all, ran without the variable reaching the engine.""" if not env or "STRATA_NO_LARGEPAGES" not in env: return None line = (g0 or {}).get("arena_line") if line is None: return "ENGINE ≠ ENV: the engine log carries no `expert arena:` line, so STRATA_NO_LARGEPAGES is unconfirmed" if arena_pages(line) == "4 KB pages (STRATA_NO_LARGEPAGES)": return None return (f"ENGINE ≠ ENV: the config set STRATA_NO_LARGEPAGES=1 but the arena line reads " f"{arena_pages(line)} ({line[:120]!r})") def strata_server_argv(python, strata_src, cfg_path, port): return [python, os.path.join(strata_src, "serve", "server.py"), "--engine", "strata", "--config", cfg_path, "--port", str(port)] def read_text(path): try: with open(path, "rb") as fh: return fh.read().decode("utf-8", "replace") except OSError: return "" def parse_info(text): m = RE_INFO.findall(text or "") if not m: return None out = {} for kv in m[-1].split(): k, _, v = kv.partition("=") out[k] = v return out def strata_g0(engine_log, server_log): """{verdict pass|refused|cpu_refused, lines, info, pcie, zero_doorbell_line, arena_line, arena_pages}: G0's log half for Strata. The zero-doorbell line (v0.1.39, H-3) and the expert arena's line with the page backing it names (H-5) are recorded, never a verdict: the first explains a hit rate the engine did not print, the second shows whether an arm's engine_env reached the engine.""" t = engine_log + "\n" + server_log avx2 = RE_AVX2.search(t) cvec = RE_CVEC.search(t) info = parse_info(t) pcie = RE_PCIE.search(t) cpu = RE_CPU_REFUSED.search(t) zero = RE_ZERO_DOORBELL.search(t) arena = RE_ARENA.search(t) rec = {"avx2_line": avx2.group(0) if avx2 else None, "control_vector_line": cvec.group(0) if cvec else None, "info": info, "pcie_line": pcie.group(0) if pcie else None, "pcie_gbps": float(pcie.group(1)) if pcie and pcie.group(1) else None, "pcie_frac": float(pcie.group(2) or pcie.group(4)) if pcie else None, "cpu_refusal_line": cpu.group(0) if cpu else None, "zero_doorbell_line": zero.group(0) if zero else None, "arena_line": arena.group(1).strip() if arena else None, "arena_pages": arena_pages(arena.group(1).strip()) if arena else None} why = [] if cpu: rec.update(verdict="cpu_refused", why=f"the engine refused the CPU: {cpu.group(0)}") return rec if not avx2: why.append("the log carries no native AVX2 line") if cvec: why.append(f"the log carries a control-vector line: {cvec.group(0)!r}") if info and info.get("cvec", "0") not in ("0", ""): why.append(f"the INFO line reads cvec={info.get('cvec')}") rec["verdict"] = "refused" if why else "pass" rec["why"] = "; ".join(why) or "native AVX2 line present; no control-vector line; INFO cvec=0" return rec def llama_g0(log_text, device_list, all_layers): dev = device_list(log_text) lay = all_layers(log_text, engine_key="llama-server") why = [] if dev.get("cuda") != 1 or dev.get("vulkan") != 0: why.append(f"devices {dev.get('names')}: one CUDA device and no Vulkan device is the rule") if lay.get("verdict") != "ok": why.append(lay.get("sentence") or "offload unread") return {"verdict": "refused" if why else "pass", "devices": dev, "offload": {k: lay.get(k) for k in ("verdict", "line", "offloaded", "of", "sentence")}, "why": "; ".join(why) or "one CUDA device, no Vulkan; " + (lay.get("sentence") or "")} def early_exit_verdict(engine, engine_log, server_log, device_list): """AUDIT-HARNESS B-2: the server EXITED before /health answered. Only the engine's own words make that a G0 verdict: Strata's CPU refusal line (the stop condition cpu_refused) or a control-vector line; llama.cpp's device lines naming a Vulkan device or a second CUDA device. Anything else (a path, a library, an argument the server rejected: the harness's slip) is `exited_before_ready`, written as STOPPED and re-opened with --attempt N+1, never REFUSED.""" if engine == "strata": g = strata_g0(engine_log, server_log) if g["verdict"] == "cpu_refused": return g if g.get("control_vector_line"): g.update(verdict="refused", why=f"the server exited before /health; its log carries a control-vector " f"line: {g['control_vector_line']!r}") return g g.update(verdict="exited_before_ready", why="the server exited before /health answered and its log carries " "no line G0 refuses on (no CPU refusal, no control-vector line)") return g dev = device_list(server_log) rec = {"devices": dev, "log_tail": (server_log or "")[-1500:]} if (dev.get("vulkan") or 0) > 0 or (dev.get("cuda") or 0) > 1: rec.update(verdict="refused", why=f"the server exited before /health; its device lines name {dev.get('names')}" f": one CUDA device and no Vulkan device is the rule") else: rec.update(verdict="exited_before_ready", why="the server exited before /health answered and its log carries " "no line G0 refuses on (no Vulkan device, no second CUDA device)") return rec def strata_request_facts(log_slice, metrics_req): """Per request: the hit rate and drafts from the engine log's own lines and /metrics' newest request. hit_rate_line: "present" (a hit-rate line was read), "absent" (the slice holds the engine's serve line but no hit-rate line: the request decoded with no pool lookup, e.g. on v0.1.39's zero-doorbell graph, H-3) or None (no serve line either: nothing to judge). offloaded / pcie_share_log / routed: v0.1.39's #588 suffix on that same hit line (H-2), None when the line carries none (36fa455e never prints it; v0.1.39 prints it only above 0).""" text = log_slice or "" hits = list(RE_HIT.finditer(text)) srv = RE_SERVE.findall(text) out = {"hit_rate_log": None, "hits": None, "lookups": None, "hit_rate_line": None, "offloaded": None, "pcie_share_log": None, "routed": None, "drafts_accepted": None, "drafts_offered": None, "engine_generated": None, "engine_decode_tok_s": None, "engine_prompt_tok_s": None, "reused": None} if hits: h = hits[-1] out.update(hit_rate_log=float(h.group(1)) / 100.0, hits=int(h.group(2)), lookups=int(h.group(3)), hit_rate_line="present") off = RE_HIT_OFFLOAD.match(text, h.end()) if off: out.update(offloaded=int(off.group(1)), pcie_share_log=float(off.group(2)) / 100.0, routed=int(off.group(3))) elif srv: out["hit_rate_line"] = "absent" if srv: s = srv[-1] out.update(reused=int(s[1]), engine_prompt_tok_s=float(s[4]), engine_generated=int(s[5]), engine_decode_tok_s=float(s[7]), drafts_accepted=int(s[8]), drafts_offered=int(s[9])) if metrics_req: out["metrics"] = {k: metrics_req.get(k) for k in ("hit_rate", "pcie_share", "decode_tok_s", "engine_generated", "prompt_ms", "decode_ms", "reused", "output_tokens", "finish", "projection")} g, a = out.get("engine_generated"), out.get("drafts_accepted") if g and a is not None and g - a > 0: out["tokens_per_round_derived"] = round(g / (g - a), 3) out["tokens_per_round_rule"] = ("derived: rounds = generated - drafts accepted (each verify round emits its " "accepted drafts plus one token, generate.cpp:4090-4119); the serve loop " "prints no per-request round count") else: out["tokens_per_round_derived"] = None return out def scrub_uuid_file(path): """A card UUID never stays in a kept log (LANE-PREAMBLE section 7): replaced in place, count returned.""" try: t = open(path, encoding="utf-8", errors="replace").read() except OSError: return 0 n = len(RE_UUID.findall(t)) if n: with open(path, "w", encoding="utf-8") as fh: fh.write(RE_UUID.sub("GPU-", t)) return n def write_config(path, cfg): with open(path, "w", encoding="utf-8") as fh: json.dump(cfg, fh, indent=1)