docs: prioritised backlog, and the replay proof it found broken

[IanMayo]

What this changes, and why

Adds docs/v2/backlog.md — the 70 unticked task lines across specs/1NN-*/tasks.md, triaged against the tree rather than against the record — discharges the five lines that triage identified as P1, and fixes the proof one of those five turned out to depend on.

The count was misleading in three directions at once, which is why a plain list of unticked lines was not the deliverable. 37 of the 70 are feature 123, complete on #107 — they only look outstanding from main. 20 are declined or deliberately not done and carry their argument in the line; they are listed separately as not backlog, so they stop being re-read as work every time somebody counts checkboxes. That left eleven, and five are done here: 65 unticked lines remain, of which 8 are work.

The replay proof was broken. scripts/replay-proof.ts claimed in its header to run "every byte-identity test in the suite — the generator's, the whole loop's, and the advisories-and-bundles one", and selected them with vitest run -t replay. The generator's test is AT-04 seed: two runs from one root seed are byte-identical… inside describe('the synthetic ocean (feature 102)') — neither string contains "replay". Seven ran, 623 were skipped, and it printed replay proof held over the one property the generator is most likely to lose. A name filter has no floor either: an unmatched selector skips everything and exits 0.

Fixing it took three rounds, because each fix reproduced the fault at one remove.

• Round one replaced the selector with an AT-04: byte-identity marker read off disk. That closed the rename hole and opened another: a byte-identity test written without a marker sits outside the proof exactly as the generator's did, and the marker set was a judgement no artefact recorded.
• Round two added AT-04: not byte-identity for considered exclusions and a sweep refusing any determinism-named test carrying neither. The sweep earned itself immediately, finding two byte-identity tests outside the proof under both the old selector and the first marking: runtime.test.ts's is deterministic: the same seed provisions the same manifest — precisely the half 101 T037's own deferral text named as "already tested" — and planner.test.ts's recommends deterministically: one seed, one plan, twice.
• Round three found the sweep itself blind three ways, each silently: a marker written as a trailing comment marked the next test and dropped its own; test( and multi-line it( were unseen; and prose quoting a test name was treated as a test. It also found that the proof's verification had no test at all — delete the missing-test check and pnpm check, pnpm test and pnpm gates all stay green while it prints held over a file vitest never collected.

Nine tests are marked, two excluded by name as single-run claims.

And it now runs. The proof had been wrong since the 101–109 close-out and nothing noticed, because nothing ran it. It is split by cost: check-replay-markers is gate 22, so the sweep fires on every change at no test time, with its own planted-violation fixtures; the half that runs the tests is a CI step of its own — measured at 91 seconds in CI.

Rejected: widening the name pattern to replay|byte-identical. It closes today's gap and not the class — expected set and observed set would both come from the names, so a rename moves them together and the proof shrinks in silence.

Also rejected: filing the remaining eight backlog items as GitHub issues. They are already distributed across the feature records that own them, and a second copy would be a second thing to keep true.

The instance, opened at the change

Nothing visible — no shell or panel code changes. The only executable changes are build-time scripts and a gate; the app/src edits are comment lines.

The blog entry

No entry. A working document plus a fix to a proof script is neither a new face in the shell nor a piece of backend simulation worth watching work — D17's test, and this fails it in both halves.

What was watched failing

The proof, five ways, each reverted after being seen to fail: no marker anywhere (the floor a name filter never had — vitest run -t zzz-no-such-test-name skips all 630 and exits 0); a marker not above an it(, named by file and line rather than guessed; a marked test skipped, caught because matching is by (file, title) so a pass elsewhere cannot mask it; a byte-identity test whose marker was forgotten, named by the sweep; and a planted per-run drift in the generator's draw path (+ plantedDrift++ * 1e-9) → six of nine failed, the generator's named among them.

The gate, on planted fixtures beside the other gates': an unmarked byte-identity test; a marker adrift from its test; a trailing marker, which must mark its own test and not the next; test( and a multi-line name, which must be swept; prose quoting a test name, which must not be; and two marked tests sharing a title in one file, which must be refused.

The proof's verification, now that it has tests at all (scripts/tests/replay-proof.test.ts): a skipped marked test, a failed one, an uncollected file, and a same-named pass in another file that must not stand in for it.

Clean tree: replay proof held: all 9 marked byte-identity tests ran and reproduced their runs byte-identically, exit 0, no stray file — the report goes to the system temp directory, not into app/.

pnpm check green: typecheck, lint, 630 app tests, 50 gate tests, build, 22 gates. CI green on 734cb5d, all 24 steps.

Backend changes: what was brought up, and what was seen

Not a backend change. Eleven test files gain a comment line each; no test body, assertion or production path is touched.

Left undone, and why

• scripts/ is typechecked and linted by nothing. No root tsconfig.json, @types/node only in app/node_modules, eslint . rooted at app/. A trial typecheck gives 11 errors, 9 inside gate fixture trees that any real config must exclude; the two real ones are scripts/build-snapshots.ts:93 (a 'departure' era where four are accepted — in the script that produces the committed artefacts) and a duplicate join import in scripts/gates/check-vocabulary.ts. That needs a root tsconfig, a lockfile change and a careful look at the snapshot builder: a separate argument from this one.
• The document is a fourth record layer with no gate, which every review round led with. Mitigated rather than solved: it says the lines are the authority, carries a command to regenerate its counts, and names what dies when 123: two-layer kernel, cost in simulation time, Forecast tab; 124 spec #107 merges. Generating it from the lines would be the real fix.
• Issues Feature 111: the Background tab (eight standards explainers) #54 and Telemetry deferrals from 107: per-region statistics, failure kinds, latency #61 are not closed. Closing an issue is a different decision from ticking a task line. Telemetry deferrals from 107: per-region statistics, failure kinds, latency #61 still owes its failure-kinds half, which specs/107-operator-view/spec.md records.
• The remaining eight backlog items are untouched, including the two the document ranks highest: feature 113's watched-turn captures (the one unfulfilled Constitution IX obligation) and the leakage-mask gate.

Review

Four rounds of /adversarial-review — two on the document, two on the code — each with two independent passes, plus verifier agents on round one's findings. Each round found the previous fix unsound at one remove, which is the useful part of the record: round two caught a miscitation round one's correction had introduced, which would have sent a developer to edit a correct line while leaving the real stale one untouched; round four caught the gate reproducing, in its own implementation, the fault it exists to catch.

Across all rounds nine findings did not survive verification and were dropped — two of them errors the passes themselves made: ClockStrip.tsx attributed to feature 105 (a shallow-clone artefact — 4dd407c is a graft boundary, so git log --diff-filter=A misattributes) and a line number that was already correct.

One thing no pass caught: scripts/replay-proof.ts carried two raw NUL bytes for two commits, so its diff rendered as Bin and was unreviewable. The passes read files directly, so a binary diff is invisible to them.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KCv1sDnpZZXoXJ2w8osB3s

[IanMayo]

Adversarial review

Two rounds, each with two independent pr-adversary passes (correctness; simplicity and the record) given the diff and base only, plus five pr-finding-verifier agents on round one's findings. Round one reviewed c010a77, round two reviewed f5a3484. Every confirmed finding was fixed before this pull request opened, so there are no inline comments — the value left here is the assumptions map and the record of what was refuted.

Assumptions and design decisions, ranked

1. That a backlog belongs in a committed file at all. Both rounds, both passes, led with this. The file is a fourth record layer on top of the task lines, their in-line reasons, and six issues, with no gate to keep it true — and it had already drifted from the tree at the moment it was written. Mitigated, not solved: it now opens by saying the lines are the authority, carries a command to regenerate its own counts, and names what dies when #107 merges. Generating it from the lines would be the real fix. A reader who trusts it over the lines is still the failure mode.
2. That a figure quoted from a task line is a fact about the tree. The draft applied the tree-is-authority test to checkboxes and not to the prose around them. 240×168 was the instance: it appears in no source file, against layout.ts's 208×116 and a derived 450×464.
3. That every cited path and line is right, because every row ends in an imperative a developer will act on. Round two found three wrong, one of which would have corrupted a correct line.
4. That two experiments hold without an artefact. The planted 5 °C kernel shift and the 125 kernel invocations were reproduced by a verifier, not by anything on disk. The structural claim behind them is checkable from run.json and build-snapshots.ts; the numbers are not.
5. That "declined" and "deferred with a trigger" are one category. They are not; two rows had their triggers restored.

Confirmed and fixed before opening

• docs/v2/plan.md:61 named as the second stale §5.11. It is a gh-pages table row, and that file's only §5.11 (line 178) is correct. Real site: specs/113-operator-flowchart/plan.md:61. Following the instruction would have corrupted a right line and left the real one.
• The proposed replay-proof remedy reproduced the fault: vitest exits 0 on an empty selection (checked — 630 skipped, exit 0), so a tag degrades the proof to zero tests and it still prints held.
• 240×168 matches nothing on disk (app/src/panels/operator/layout.ts:73-86).
• "Everything from P3 down in parallel" false for P5's forecast eras, discharged in scheduler.ts, where #107 adds 251 lines.
• grep -c over several files never prints the total the self-check recipe was offered to produce.
• The grep 'feature 113' evidence undercounts: FR-36's marker wraps across srd.md:417-418.
• Citation drift: run.json authors block is 2305–2310; advisories.test.ts it() is at :305.
• Over-claims: the §5.20 quotation was a paraphrase; the third leakage fact sits inside if (noisyFirst && noisySecond); feature 124 stays unstarted after the merge.

Refuted, and dropped

Six findings did not survive verification — two of them errors the passes themselves made:

• That ClockStrip.tsx arrived with feature 105. The clone is shallow and 4dd407c is a graft boundary, so git log --diff-filter=A misattributes every file it touches; specs/101-foundations-shell/plan.md:19 places it in 101.
• That the ComposerPane fetch is at line 73. It is at :74, as cited.
• That the document claimed issue #61 discharged — its wording was already hedged, and the spec records the failure-kinds half as open.
• That FR-35's missing marker is a defect rather than the residue the row anticipates.
• That the NetCDF decision already exists on disk. Neither disposition does, and #62 scopes transfer rather than format.
• That 104 T307 and 103 T208 are stale declines. T307's named consumer landed and sends no Accept header; 107 delivered no rate control.

Checked and sound

All arithmetic (70 = 37 + 20 + 2 + 11, and the per-feature allocation); 11 unticked V1 lines; 83/0 on the #107 branch against 11 ticked on main; 35 lines in 124; every srd.md citation (FR-52@583, FR-60@641, V2-C21@173, §5.11=112, §5.12=113, §5.13=110, FR-61@653, FR-62@664, FR-35@407 unmarked, FR-39@444-451, §5.20@1158); the six open issues and their scopes; start-condition.test.ts:228. The replay-proof hole was reproduced independently by both passes (7 passed, 623 skipped). No constitutional trap applies — no code, no generated output, no snapshots, no seam crossing, no wall-clock, no new gate. pnpm gates is 21 clean, which is a non-result here: no gate reads docs/v2/.

The three things worth opening the diff for

1. P1 — the replay proof reports "held" over a test it never runs, and would report it over zero tests.
2. P0 — check-snapshot-drift gives the forecast kernel no cover, so #107's empty snapshot diff is correct rather than a missed regeneration.
3. P6 — the density figure in the task line disagrees with layout.ts by roughly 4× in area.

Opus 5, two passes per round, findings verified against head before posting.

---

Generated by Claude Code

[github-actions]

This branch, running: https://deepbluecltd.github.io/drogna/instances/claude-outstanding-tasks-specs-qvtgnn/

Intro · Background · Data · Operator · Map · Messages · Sampling · Courses · Feasibility

Built from 734cb5d0fe8049b6b26b98e8226b8ba08b6f6e39, and replaced in place by every further push to this branch. Synthetic throughout, as the header says.

[IanMayo]

Adversarial review — the replay-proof change (c8c3b65)

Two independent passes over the code change, given the diff and base only. This round was the most productive of the three: it found that the fix closed one hole and opened another, and that two of the ticks were falsified by their own commit. Everything confirmed is fixed in 0e19e8a.

Assumptions and design decisions, ranked

1. That the marker set is the complete set of byte-identity tests. Nothing established it. Both passes led here, and pass B proved it false by finding two unmarked ones — runtime.test.ts's is deterministic: the same seed provisions the same manifest (the half 101 T037's own deferral text named as already tested) and planner.test.ts's recommends deterministically: one seed, one plan, twice. Swapping a mechanical selector for a curatorial one and not auditing the curation is the same fault one level up. Fixed: a second marker for considered exclusions, and a sweep that refuses any determinism-shaped test carrying neither.
2. That "ran and passed" could be decided by title alone. The verification pooled titles across all files and then widened the match with endsWith. Pass A reproduced both variants in a rig: a skipped marked test reported as held, masked by a passing test elsewhere. Fixed: matched by (file, title) from the report's own path, no suffix relaxation.
3. That --reporter=json was a free change of reporter. It replaced the default rather than joining it, so a divergence printed a name and no diff — and the report was deleted in a finally before the verdict was computed. Fixed: both reporters, cleanup after the verdict.
4. That line numbers are a durable way to cite the tree. Falsified by the same commit: the FR-35 amendment inserted three lines into srd.md above almost every citation the ticks and the backlog make. Fixed: anchors throughout.
5. That the proof runs on somebody's schedule. It is in no workflow, no pnpm check, and no gate; scripts/ is neither typechecked nor linted by CI. Not fixed here — flagged below.

Confirmed and fixed in 0e19e8a

• The marker set was incomplete by two, and nothing detected an unmarked byte-identity test.
• File-blind title matching could report a skipped marked test as held.
• A divergence printed no diff, and the evidence was deleted unread.
• app/replay-proof-report.json was written into the repo, ungitignored, and the cleanup did not run on the catch path (process.exit skips the enclosing finally). Now written under the system temp directory.
• 113 T006 claimed grepping for "feature 113" finds three of four markers. It finds two — the FR-35 marker this branch added wraps across a line break exactly as FR-36's does. The tick was falsified by its own edit.
• Eleven line citations stale at head by the size of this branch's own srd.md insertion.
• specs/109-map/tasks.md T807 and specs/109-map/spec.md both still said the proof "runs every replay test" — the claim this work exists to retract, left standing one file over.
• Dead machinery: fullName in the passes set and the endsWith scan, whose only reachable effect was to loosen the match.

Watched failing, five ways

Each reverted after being seen to fail: no marker anywhere (the floor a name filter never had — vitest run -t <unmatched> skips all 630 and exits 0); a marker not above an it(, named by file and line rather than guessed; a marked test skipped; a byte-identity test whose marker was forgotten, which the sweep names; and a planted per-run drift in the generator's draw path, which took six down with the generator's named among them. Clean tree: nine marked tests ran and held, no stray file.

Not fixed, and why

pnpm replay-proof runs on nobody's schedule. It is absent from .github/workflows/ci.yml, from pnpm check and from scripts/gates.registry, and scripts/ is outside app/tsconfig.json's include and outside eslint .'s cwd — so this script's ~180 lines are typechecked by tsx's type-stripping alone. Unlike every gate in scripts/gates/, it has no entry in scripts/tests/. That is exactly how it stayed silently wrong from the 101–109 close-out until now, and it is beyond what this pull request was for: wiring it into CI is a change to the check surface that deserves its own argument. Worth an issue.

Checked and sound

pnpm check green — typecheck, lint, 630 tests, build, 21 gates. The diagnosis behind the change verified independently by both passes: vitest list reports 630 tests and -t replay matches exactly 7, the generator's among the skipped; an unmatched selector exits 0. The backlog's arithmetic re-derived line by line (65 = 37 + 20 + 8, with the declined table at exactly 20 rows). No constitutional trap applies — the app/src changes are comment lines only, no import moved, nothing generated or snapshotted touched. Both passes confirmed the plan.md §5.11→§5.12 fix lands on the correct file, the one an earlier round nearly got wrong.

Opus 5, two passes, findings verified against head before fixing.

---

Generated by Claude Code

[IanMayo]

Adversarial review — the gate and CI wiring (d398362)

Two independent passes over the code change. This round found that the gate repeated, in its own implementation, the class of fault it exists to catch — three times — and that the proof's verification was held by nothing at all. Everything confirmed is fixed in 734cb5d.

Assumptions and design decisions, ranked

1. That the marker set is complete, and that a marker sits on the line above a test. Both passes led here. The second half was the fault: reading only the line below meant a marker written as a trailing comment recorded the next test as the proof's and let its own drop out of the sweep, unreported. Both halves failed at once, silently, and it is the original fault's exact shape — the record names one test, the run proves another.
2. That candidacy is a name-vocabulary judgement over it( with the name on the same line. Three legal spellings were invisible: test(, a multi-line call, and it.each. The asymmetry is what made it dangerous — a marker above an unrecognised form is reported, a missing marker on one is not, and the missing marker is the case the sweep is for.
3. That the verification logic is exercised by something. It was not. Delete the missing-test check and pnpm check, pnpm test and pnpm gates all stay green while the proof prints held over a file vitest failed to collect. The gate half was permanently held by planted fixtures; this half was held by nothing.
4. That prose is not code. scan() did not skip comment lines, though scripts/gates/lib.ts exports isCommentLine for exactly this. A docblock quoting a test name — a habit this repository has, including in the new gate's own header — was reported as a violation, and both escapes were wrong: reword the prose, or mark it and add a test that does not exist to the proof's expected set.
5. That vitest's title identifies a test within a file. It is the leaf only, with no ancestor describe, so two marked tests sharing a title in one file would make a pass for either indistinguishable from a skip of the other. Not reachable today; now refused.

Confirmed and fixed in 734cb5d

• Trailing marker marks its own test; the sweep no longer loses it.
• test( and multi-line names are swept; the lookahead reads three lines.
• Comment lines are skipped, so prose about the rule is not a use of it.
• Duplicate marked titles within a file are refused with the reason.
• The verification moves to scripts/replay-verify.ts with scripts/tests/replay-proof.test.ts holding it: a skipped marked test, a failed one, an uncollected file, and a same-named pass in another file that must not stand in for it. Four new fixtures hold the gate half.
• CLAUDE.md said pnpm check is "what CI runs" — false the moment this branch added a CI step. Now "most of what CI runs", with the proof named beside it and the reason it is not in pnpm check.
• The backlog claimed the new gate's test sits "beside the other 21". It is 18: check-schema-masters and check-snapshot-drift have no planted-violation test. Worth stating in a document whose closing section is about not trusting an unwatched check.
• #107 adds 244 lines to scheduler.ts, not 251 — that was the changed-line total.
• The CI comment now names the duplication it accepts (the App tests step already runs these files) rather than only its duration, and says what would make selecting by name safe now that the report check supplies a floor.

Deliberately not fixed

scripts/ is typechecked and linted by nothing. No root tsconfig.json exists and @types/node sits only in app/node_modules; eslint . is rooted at app/. A trial typecheck gives 11 errors, 9 of them inside gate fixture trees that any real config must exclude. The two real ones are scripts/build-snapshots.ts:93 (a 'departure' era passed where four are accepted — in the script that produces the committed artefacts) and a duplicate join import in scripts/gates/check-vocabulary.ts. Fixing that means a root tsconfig, a lockfile change, and touching the snapshot builder: a separate argument from this one.

Checked and sound

pnpm check green in pieces after two OOM kills of the combined run — typecheck, lint, 630 app tests, 50 gate tests, 22 gates, build — and the proof holds over 9 marked tests. Both passes independently reproduced the diagnosis behind the change (vitest list → 630, -t replay → 7, unmatched selector → exit 0) and confirmed the 9/2 split is honest, both exclusions being genuine single-run claims. The gate and the proof cannot disagree about the marked set, because the proof imports scan rather than re-deriving it — both passes named that as the change's best decision. Constitution clear throughout: the app/src changes are comment lines only, nothing generated or snapshotted, no seam crossing, scripts/gates.registry appended rather than rewritten.

One process note, since it bears on how much these reviews are worth: neither pass caught that scripts/replay-proof.ts carried two raw NUL bytes for two commits, making its diff render as Bin and therefore unreviewable. The passes read files directly, so a binary diff is invisible to them. That is a gap in how the review is run, not in the passes.

Opus 5, two passes, findings verified against head before fixing.

---

Generated by Claude Code