feat(pm): archive the open board before the closed history, and move the snapshot cron off the patrol's hour - #17442
Queued
os-litant wants to merge 1 commit into
Queued
Conversation
…the snapshot cron off the patrol's hour The first scheduled run of the archiver walked `state=all` oldest-first: 800 requests bought 137 closed issues and 268 closed pull requests from February and zero open cards. The records an account suspension destroys are the OPEN board, so that half was being archived last. A first walk now runs two phases in one order. The open phase asks the `/issues` listing for `state=open` (it carries pull requests too) and walks it to a short page; only when it is complete does the closed-history walk start from its own cursor. The two cursors are kept apart, so a run that runs out of budget inside the open set resumes inside it and the history cursor waits untouched. `walk_phase` and `walk.open_set` in the manifest separate "open set complete, history resuming" from "still inside the open set". An archive written before this order existed re-walks the open board first and keeps its history cursor rather than discarding it. The count check's pending predicate moves with it: its arithmetic needs only the open set, so the run that finishes the open phase has a reading even while the history is still resuming, and a later history run does not — that enumeration is from an earlier run. Every pending verdict now carries the reason it is pending. Also fixed, in the same file and required by the negative control: the nested `board.read_at` stamp escaped `materialManifest`, so a steady-state run whose whole job is to write nothing would move it and commit a manifest-only diff on every scheduled run. The cron moves from `37 1,7,13,19` — the half-state patrol's cron to the minute — to `7 2,8,14,20`. Both are scheduled board readers spending one per-repository `GITHUB_TOKEN` hour of 1,000 requests. The header now names that pool, the other standing spenders, and why the cap is 800. Claude-Session: https://claude.ai/code/session_01YKEjmbYNvYWJvWGSWx26zK Co-authored-by: Claude <noreply@anthropic.com>
This was referenced Sep 10, 2026
os-litant
marked this pull request as ready for review
September 10, 2026 15:10
os-litant
added this pull request to the merge queue
Sep 10, 2026
Any commits made after this event will not be merged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #17434
The archiver's first scheduled run measured the defect this fixes: walking
state=alloldest-first, 800 requests bought 137 closed issues and 268 closed pull requests from February and zero open cards. The records an account suspension destroys are the open board, so that half was being archived last — days away, at ~405 numbers per run.1. Open first — the walk order
A first walk now runs two phases, in one order, never interleaved:
openstate=open&sort=updated&direction=asc— the/issueslisting carries pull requests too, so one walk is the whole open boardhistorystate=allfrom the stored history cursor — the closed record, which has no other reader anywhereincrementalsincewalk, unchanged by any of the aboveThe two cursors are kept apart on purpose: a run that runs out of budget inside the open set resumes inside it, and the history cursor waits untouched. The page-walk rule is unchanged — a short page ends a walk, and the walk re-anchors on the last row's value rather than incrementing an offset.
Before / after, measured on this board with
--limit=20into a scratch directory (the same flags, two builds):The manifest says which phase, so a reader can tell the two situations apart (from the same drill):
walk_phase: historywithopen_set.complete: trueis "open set complete, history resuming";walk_phase: openis "still inside the open set". A resumed run continues the phase it stopped in — drilled live: the second run plannedresuming the open set at 2026-08-18T04:39:33Z — the closed history does not start until it is complete, read 60 more numbers, and stayed in the open phase.Migration, which the next scheduled run will take: the archive on
board-archivewas written before this order existed, so it carries nowalkblock. Its cursor is a history cursor by construction (the only walk that version had wasstate=all), so it is kept and the open board is walked first from the beginning. An archive whose old walk had completed is already past both phases and stays incremental.2. The request arithmetic for the open set
Measured, not estimated — cumulative over the two drill runs above: 79 unique open issues archived in 74 requests = 0.94 requests per number (one listing page per re-anchor, plus one comments read for a number that has comments).
The board's own arithmetic, read by the drill at 14:20Z:
open_issues_count596 minus 14 open pull requests = 582 open issues.So the open set fits one 800-request run, with room to start the history walk in the same run. It is not assumed to: the resume-inside-the-open-phase path is drilled above and self-tested, because a busier board or a slower page would land it in a second run.
3. The cron, and the budget it is capped against
Both are scheduled board readers authenticating as this repository's
GITHUB_TOKEN, and that budget is 1,000 requests per hour per repository — one pool, not a per-workflow allowance — so a snapshot run spending its full 800 leaves the patrol the remainder of the same window. The collision reading: the 13:37Z patrol run (34484272265) and the 13:41Z snapshot run (34484192186) were both green on the day this was filed, so nothing had been starved yet. The offset is prevention, taken while the archive was still small enough that no run had spent its whole cap against the patrol's window — and the header now says exactly that.Deviation from the card's suggested
7 4,10,16,22, on a measurement. Of the five hour sets that share no hour with the patrol's1,7,13,19,2,8,14,20is the least contended on this repository's cron inventory (22 scheduled workflows read out of.github/workflows/):2,8,14,20(chosen)4,10,16,22(suggested)0,6,12,18release.ymlruns0 */6— every one of those four hours3,9,15,21/5,11,17,23Four runs a day and
workflow_dispatchare unchanged. The workflow header now names the shared pool, every standing spender on it, and why the per-run cap is 800: it is the largest cap that still leaves a fifth of the hour to whatever else lands in the same window. No retry loop, no cap raise, no second token.4. The count check's pending predicate — Zone 2C, half falsified
The card's assumption was that the check turns from
pendingto a verdict once the open set is complete, because its arithmetic needs only the open set. That holds for the run that enumerates the open set, and not for the runs after it. The history walk takes days; every card closed or opened during it is still archived in its old state, so a verdict in a history-phase run would report the clock as a board defect (a permanently red alarm is silence — this tool's own header says so aboutgone.json).So the predicate moved from "this run stopped" to "this run did not enumerate the open board", which is strictly more precise, and every
pendingnow carries the reason:pendingwiththe open set is complete but was enumerated in an earlier run, and the closed history is still walking — cards closed or opened since then are archived in their old state;The check's own contract is kept:
pendingis a third answer and never a pass (okis null).5. Bounded in-place fix, in the same file: the nested run stamp
The card's negative control — "an identical re-run still writes nothing" — did not hold in the steady state on
main, and writing item 4's self-test case for it is what surfaced it.materialManifeststrips the three top-level fields that move every run, butboard.read_atis nested and escaped it, so a run whose whole job is to write nothing would move that stamp and commit a manifest-only diff on every scheduled run — burying the real diffs exactly the way the file's header promises this tool never does. It is now stripped with the others.Reverse verification (fix committed first, mutation proved on disk, restore proved against the HEAD blob):
Direction: the negative control turns red under the ablation and green on restore. Both legs ran from the committed state, against the same live board, minutes apart.
Verification
pnpm check:pm-board-snapshot— 89 cases across 8 batteries (70 before). New: the open-first order, the two phase cursors, the resume-inside-the-open-phase case, the migration of a pre-phase archive, thestate=open/state=allspellings, the pending predicate's five answers, and the nested run stamp. The 70 existing cases are all still present; the 7 that namedselectSincenow nameselectWalkPlan(a second cursor decision function would have been a fork) with their assertions and case names unchanged.node scripts/pm/dispatch-gates.mjs --commandson the real diff and reconciled with--ran; exit codes captured before any pipe.npx eslint --no-inline-config scripts/pm/board-snapshot.mjsexit 0, no output.scripts/pm/check-governed-merges.mjs --teston the final file list: NOT governed.Acceptance notes
ARCHIVE_SCHEMAstays 1: the file's own rule is to bump it when a record's shape changes, and no record shape moved — the manifest gained fields, and the manifest's only reader is this tool.pendingverdict gained areasonkey, and every non-pending verdict carriesreason: nullso the key set is identical in both branches and the manifest diff stays stable.--dry-run --limit=5 --max-requests=25into the runner's temp dir, no archive checkout, no commit. It now reads 5 open numbers instead of 5 closed ones, at the same cap.noted, not filed: the drill archive's count check exits 2 with a SHORTFALL, which is correct for a 79-record scratch directory but means a--limitdrill can never end green; a drill flag and a scheduled run share one exit register. Successor: whoever next adds a flag to this tool.noted, not filed: a card closed on the board while the open phase is walking stays archived as open until the history walk reaches it, so the first open-phase verdict can carry a surplus of the walk-window churn. Inherent to comparing a windowed enumeration with a point-in-time count; the board's arithmetic is now read at the tightest available instant (immediately after the open phase) to keep the window as small as it can be. Successor: whoever acts on the first non-pendingverdict.Generated by Claude Code