feat(grading): stop the grading queue at the plan's quota limit

[kamui]

Why

The grading queue has a dollar cap but cannot see the subscription plan's usage windows. When a five-hour or seven-day limit is hit, every session in flight fails like any other failure: the queue stops with up to one failed attempt per worker, each kept and charged. The regrade of the last-push cohort is about 190 Opus sessions on a plan, so a limit is expected and has to be an orderly, resumable stop.

regrade.py now reads the plan's meters the way the review queue does and stops launching before the limit.

Scope

• bench/tools/subscription_meters.py (new): meters, used, read_meters, sign_in_expiring, quota_stop, moved out of review_queue.py. Standard library only.
• bench/tools/review_queue.py: uses the module. Its behaviour is unchanged, except that a wrong-shaped meter response is now a blocked launch and no longer an uncaught AttributeError.
• bench/tools/regrade.py: reads the meters before the preflight, before preparing a batch and again before writing its reservation, reusing a reading for up to 300 s. At or over quotaStopPercent (optional authorization field, default 95), with a Claude sign-in about to expire, or with unreadable meters, it launches nothing more, lets running sessions settle and map, and ends quota-stopped with exit 4.
• bench/tools/regrade_queues.sh: logs a quota stop in driver.log and passes exit 4 on.
• docs/research/cohort-rebuild-2026-10-05/regrade/plan.py: pins the new module for new authorizations.
• docs/grading-readiness.md: the quota section.

Tradeoffs

• The meter code is its own module so that regrade.py does not import the review queue. An authorization pins the runner's files, and importing review_queue.py would have made every review-queue edit block grading until a new authorization.
• A reading is reused for 300 s, not the review queue's 3600 s. Several Opus workers can move a five-hour window from under 95% to the limit within an hour.
• An unreadable meter stops the queue resumably; it is not ignored.
• A failure or an exhausted cap that follows a quota stop decides the run's state, so a session that hits the real limit while the queue settles is not reported as a clean stop.
• A cap that cannot fund the next batch now stops as budget-stopped before any reading or preflight. Before, the preflight ran first.

Blast Radius

Both queues' launch paths. The review queue's readings, messages and exit codes were compared with main against a fake provider and for an unreadable sign-in file, and are byte-identical. A saved grading authorization pins the old regrade.py, so it needs a new version before it runs with this controller, as for any controller change. No queue is running on main's tools; the review-code run in flight uses its own freeze commit.

Verification

• python3 -m unittest discover -s bench/tools -p "test_regrade.py": 63 pass.
• python3 -m unittest discover -s bench/tools -p "test_review_queue.py": 62 pass.
• bun run test:bench: 918 tests, one failure, test_codex_grading.test_catalog_preserves_native_model_instructions, which also fails on main on this host and passes in CI.
• One live Claude meter read through the grading path returned both usage windows (before the code moved into its own module; the move is covered by the fake-provider comparison). The Codex meter path is not run live.
• Reviewed by Codex GPT-6.1 Sol (high): four findings, all fixed with a test each.

Part of #30

[kamui]

Approved (advisory) — no findings.

Ambiguities

• A failure while the quota-stopped queue settles — (a) Every later failure overrides other stops; (b) Failure replaces quota-stopped; the first non-quota stop keeps precedence. Applied: The second reading follows the PR tradeoffs and existing stop ordering. Failed attempts remain recorded and rechecked on resume.

Reviewed 3027b0d. All 8 changed files and the affected launch, settlement and restart paths reviewed. No actionable defect found. Publishing as COMMENT under kamui with gating withheld.

Review details

Intent: Stop grading launches at the subscription quota while retaining running sessions and resumable prepared work.

Issue fit: Meets this PR's controller scope and supports #30. Grading, authorization, audit and dataset publication remain separate work in that issue.

Reviewed: 3027b0d against merge-base 068e8a6.

Checks:

• At the reviewed head, 63 regrade tests and 7 focused review-queue meter tests passed.
• CI grading logs report 918 tests, 18 skipped and no failures at the reviewed head.
• Live provider meter calls were not repeated; coverage uses fake-provider tests and code traces.
• Independent checks confirmed draining within the cap and prepared-workspace reuse. If cap exhaustion wins first, budget-stopped retains precedence; failed receipts remain recorded and gate new launches on resume.