Skip to content

Commit e9bba50

Browse files
claude[bot]claude
andauthored
ci(lint): report how many gates never ran when Lint & Repo Gates aborts (#17678)
* ci(lint): report how many gates never ran when the job aborts `Lint & Repo Gates` runs ~161 gates as separate Actions steps and stops at the first non-zero exit. That abort is deliberate and adjudicated (#13690); what was missing is the number it leaves behind. One red is a LOWER BOUND on the problems in the tree, and a gate that never executed produces the same "no output" a passing gate does. Measured on run 34468505490 before writing this: the jobs API and the web UI do mark the un-run steps `skipped` (112 of them), but never count them, and use that same word for a step its own `if:` turned off. The check-run `output` was null on every field and the job log said nothing at all. Adds `scripts/report-unmeasured-gate-tail.mjs` plus one `if: failure()` step that prints the count to the job log, the job summary and one `notice` annotation. Fail-fast is untouched, no gate moved, no `continue-on-error`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU * ci(lint): count the gate tail by position, not by a mid-run conclusion Caught by the live double-failure probe (run 34583803573), not by reasoning: the reporter runs while the job is still in progress, and the runner stamps `skipped` on the steps behind the failure progressively. The completed job ended with 161 skipped declared steps; at the instant the reporter read the API only 12 carried that word, so it printed `never_ran=12` — a lower bound on a lower bound, wearing the look of a precise number. Behind the failure the tail is now counted by POSITION, and `conclusion` is used only to subtract the steps that demonstrably did run. A mid-run fixture pins it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU * ci(lint): stop the tail reporter from listing itself, and gate the wiring Run 34584689309 ended its own report with `- Report how many gates never ran`, while that step was the one printing it: the jobs API had not yet marked the step `in_progress`, so the in-flight rule did not catch it. The workflow now hands the step its own name through OS_TAIL_REPORT_STEP and the reporter subtracts it. The two lines agreeing is not left to a comment — `wiringVerdict()` reads lint.yml as text and the self-test fails on a stale value, a missing one, or a rename. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU * ci(lint): keep a green run byte-identical — the self-test is failure-only too Acceptance clause 2 of the card is that an all-green head's output does not change at all. An unconditional self-test step would have printed one line on every green run, and one line is still new noise. Both new steps now carry `if: failure()`, so a green run of this job produces exactly the bytes it did before: no log output, no summary, no annotation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 1a25f4a commit e9bba50

2 files changed

Lines changed: 813 additions & 4 deletions

File tree

.github/workflows/lint.yml

Lines changed: 72 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -177,6 +177,13 @@ jobs:
177177
runs-on: ubuntu-latest
178178
permissions:
179179
contents: read
180+
# ⭐ Read-only, and it buys exactly one thing: the LAST step of this job
181+
# asks the jobs API how many of the steps below it never ran, because the
182+
# size of that tail is the one fact no step can work out locally (the
183+
# runner alone knows which step index it stopped at). See
184+
# `scripts/report-unmeasured-gate-tail.mjs`. Nothing else in this job
185+
# reads it, and no step writes anything anywhere.
186+
actions: read
180187

181188
steps:
182189
- name: Checkout repository
@@ -4468,14 +4475,75 @@ jobs:
44684475
# author copying the outer number into the inner block got a 3.6-second
44694476
# cache and no error anywhere. No baseline, by ruling: the gate is green
44704477
# only at zero offenders, and a red is a rename under an ADR-0087
4471-
# conversion (or a describe to fix), never a ledger row. Placed LAST in
4472-
# this job on purpose: it is a new tree-wide ratchet, and the job aborts
4473-
# at the first non-zero exit — nothing behind it would be masked by its
4474-
# red. Reads src/ through tsx (no build); self-tests first, wired into
4478+
# conversion (or a describe to fix), never a ledger row. Placed LAST
4479+
# AMONG THE GATES on purpose: it is a new tree-wide ratchet, and the job
4480+
# aborts at the first non-zero exit — nothing behind it would be masked
4481+
# by its red. (The two steps after it are not gates and cannot be masked:
4482+
# one is an offline self-test, the other runs only ON failure and exits 0.
4483+
# The qualifier is this PR's, forced by appending them; the ruling it
4484+
# describes is untouched, and no gate's relative order moved.) Reads src/ through tsx (no build); self-tests first, wired into
44754485
# the package script as `check:self-test-wired` requires.
44764486
- name: Duration-shaped spec keys carry their unit in the key name
44774487
run: pnpm --filter @objectstack/spec check:duration-unit-keys
44784488

4489+
# ── NOT GATES. The two steps below judge nothing about the tree. ──────
4490+
#
4491+
# ⭐ One red in this job is a LOWER BOUND on the number of problems in the
4492+
# tree, not a count — every gate behind the failing one never executed,
4493+
# and a gate that never executed produced no output, which is the same
4494+
# shape a passing gate has. The GATE ORDER note above `jobs:` already
4495+
# names this quantity ("the unmeasured tail") and is tuned to shrink it;
4496+
# until now nothing printed it. These steps print it.
4497+
#
4498+
# ⛔ The fix is NOT to make the skipped gates run: that would undo the
4499+
# adjudicated fail-fast order. ⛔ Nor is it `continue-on-error`, which
4500+
# trades a red for silence. The job's conclusion is untouched.
4501+
#
4502+
# Measured before this was written, on run 34468505490 of this very
4503+
# workflow (job 102842713265, failed at step #55): the jobs API and the web
4504+
# UI DO mark the un-run steps `skipped` — 112 of them there — but they
4505+
# never count them, and they use that same word for a step its own `if:`
4506+
# turned off, so the one surface that says it cannot tell the two apart.
4507+
# The check-run `output` was null on title, summary and text alike, and the
4508+
# job log — what an API or agent reader gets — said nothing at all. So what
4509+
# is added here is a machine-readable COUNT, on all three surfaces.
4510+
4511+
# Offline, no token, no network: drives the reporter's judge() over
4512+
# recorded jobs-API shapes, and checks that the step below still hands
4513+
# itself its own name. Wired as a step because an unrun self-test is a
4514+
# phantom check. Sub-second.
4515+
#
4516+
# ⭐ `if: failure()` for the same reason the step below carries it, and it
4517+
# is the whole negative control: a green run of this job must be BYTE
4518+
# IDENTICAL to what it was before any of this existed. A self-test that
4519+
# ran unconditionally would print one line on every green run — one line
4520+
# is still new noise, and this change is not allowed to cost any. It is
4521+
# exercised on exactly the runs its subject is used on, which in this repo
4522+
# is not a rare event. ⛔ Do not "fix" this by dropping the condition.
4523+
# It fails LOUDLY: it is a separate step, so a broken reporter shows up as
4524+
# its own red step rather than as a missing paragraph.
4525+
- name: Unmeasured-gate-tail reporter self-test
4526+
if: failure()
4527+
run: node scripts/report-unmeasured-gate-tail.mjs --self-test
4528+
4529+
# Runs BECAUSE something failed. On a green head `failure()` is false, so
4530+
# it is skipped and contributes zero bytes to the log, the summary and the
4531+
# annotations — measured, not assumed. It always exits 0: the job is
4532+
# already failed when it runs, and a non-zero exit here would only add a
4533+
# second annotation pointing at the reporter instead of at the gate.
4534+
- name: Report how many gates never ran
4535+
if: failure()
4536+
env:
4537+
GITHUB_TOKEN: ${{ github.token }}
4538+
# ⚠️ Must equal this step's own `name:` above. The reporter subtracts
4539+
# it so that it does not list ITSELF among the steps that never ran —
4540+
# measured in run 34584689309, where it did exactly that, because the
4541+
# jobs API had not yet marked it `in_progress`. Drift between the two
4542+
# lines costs one extra, visibly silly line in the report; it cannot
4543+
# go wrong quietly.
4544+
OS_TAIL_REPORT_STEP: Report how many gates never ran
4545+
run: node scripts/report-unmeasured-gate-tail.mjs
4546+
44794547
# Lane 1 of 4 behind the required `TypeScript Type Check` context. The
44804548
# aggregator job at the bottom of this file explains the split, holds the
44814549
# contract, and is the thing the merge queue actually requires — read it

0 commit comments

Comments
 (0)