Skip to content

Commit 9a1ca2b

Browse files
committed
ci(lint): tooling self-tests run on a PR only when their own inputs change
`Lint & Repo Gates` was the wall clock of a product PR's CI (27.4 min over 184 steps on PR 19314's head, above the longest test shard), and 18.6 of those minutes were the tooling's own self-tests, corpora and censuses. The single `PM dispatch-gates self-test` step was 11.8 min on a three-file `packages/spec` diff, because that family's read-set included the whole-tree censuses its battery runs: the content of every JS/TS and .sh file, the nested .gitignore files, and the tracked NAME set that made any ADDED path anywhere run it. Narrow that family to the tool's own inputs — the workflow tree and composite actions it discovers, every gate source it resolves under scripts/ and a workspace package's own scripts/, the package.json that names a check:* script, the agent configuration its live cases read, and root configuration — and put four more tooling steps behind the same selector with a read-set each: entry_guard 0.62 min scripts/** self_test_workflow_commands 0.48 min scripts/** + the workflow tree declared_population_live 0.40 min the derivation's read-set bare_root_worklist 0.32 min the derivation's read-set Product ratchets and censuses stay unconditional: the two engine ratchets, the tenancy and tenant-audit censuses and everything else that judges product code is untouched, as is ESLint. Steps under ~0.3 min stay unconditional too — the scripts/ shared-module self-tests (0.27) and the changeset-family gate self-tests (0.15) among them. The skip these five carry is a WEAKER claim than a ratchet's and the code and the workflow both say so: a ratchet skips only where it provably reads nothing, while a tooling self-test skips where the tool's own inputs are untouched and its battery may still read the changed path through a census. Gate weakening is a maintainer floor; the authorizing sentence is quoted where the read-sets are declared. What bounds it is unchanged: `push` to main and the hourly scheduled run take the unscoped branch and run the whole battery. The selector's self-test grows with it — the nine ids, the per-family read-sets, an ADDED path inside a read-set, the shape of PR 19314 under both `merge_group` and `pull_request`, and a `pin_step` for each new family against the real lint.yml. Claude-Session: https://claude.ai/code/session_012GcsUbuqFGBibkEDMRC1eE Co-Authored-By: Claude <noreply@anthropic.com>
1 parent 2cac363 commit 9a1ca2b

3 files changed

Lines changed: 333 additions & 111 deletions

File tree

‎.github/workflows/lint.yml‎

Lines changed: 105 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -31,9 +31,12 @@ on:
3131
# the push-on-`main` run of THIS workflow is the only post-merge full-battery
3232
# run of the families `scripts/ci/select-gate-families.sh` scopes away on
3333
# merge groups (the PM dispatch-gates self-test, both ratchets, the
34-
# verify-lock self-test, the comment-mask corpus). A scoped family that goes
35-
# red on `main` after a queue build skipped it had, until this trigger, no
36-
# run that would notice and no filer that would say so.
34+
# verify-lock self-test, the comment-mask corpus — and since #19498 the
35+
# entry-guard sweep, the declared-population gate, the bare-root worklist
36+
# self-test and the self-test workflow-command gate, whose whole-tree reads
37+
# that ruling took off the PR path). A scoped family that goes red on `main`
38+
# after a queue build skipped it had, until this trigger, no run that would
39+
# notice and no filer that would say so.
3740
#
3841
# The selector already treats every event that is neither `merge_group` nor
3942
# `pull_request` as "run every family", so this trigger alone restores the
@@ -245,18 +248,33 @@ jobs:
245248
# over the ten merge-group runs measured for #16496; the PM dispatch-gates
246249
# self-test alone 597 s), and none of its expensive steps read a
247250
# merge group's file surface: a docs-only group paid the full battery.
248-
# So on `merge_group` and `pull_request` the FIVE scoped families below
251+
# So on `merge_group` and `pull_request` the NINE scoped families below
249252
# -- each step carrying `if: steps.gate-families.outputs.<id> != 'skip'`
250253
# -- run only when the changed paths touch the files that family reads.
251254
# `push` on main and the scheduled full run keep the whole battery: the
252255
# script runs everything for any event it does not scope.
253256
#
257+
# #19498 added four of those nine (`entry_guard`,
258+
# `declared_population_live`, `bare_root_worklist`,
259+
# `self_test_workflow_commands`) and narrowed `pm_dispatch_gates`, on
260+
# ruling #208 on #19491 (R4) and the maintainer's sentence quoted in the
261+
# selector's header: the tooling's self-tests were 18.6 minutes of the
262+
# 27.4-minute job a three-file `packages/spec` PR paid. ⛔ Product
263+
# ratchets and censuses stay unconditional -- the tenancy and
264+
# tenant-audit censuses, the engine gates and everything else here judge
265+
# product code and are not in that ruling.
266+
#
254267
# ⭐ The invariant is FAIL-OPEN, and it holds at both layers. The script
255268
# runs every family when the base cannot be resolved, the diff fails or
256269
# is empty, a path is one it does not classify (a new top-level
257270
# directory, an unlisted root file), or any change is a deletion, rename
258271
# or type change; a family is skipped ONLY when every changed path is
259-
# positively classified into a class that family provably never reads.
272+
# positively classified into a class that family's DECLARED read-set
273+
# excludes. For the other four -- both ratchets, the corpus walk and the
274+
# verify-lock self-test -- that read-set is what the family provably
275+
# reads. For the five tooling self-tests it is the tool's OWN INPUTS,
276+
# which is a weaker claim; it is argued where it is declared, in the
277+
# selector's header.
260278
# The `!= 'skip'` spelling means an ABSENT output -- the selector never
261279
# ran, or wrote nothing -- also runs the step. Both halves are pinned by
262280
# `scripts/ci/select-gate-families.selftest.sh` (`check:select-gate-
@@ -575,8 +593,18 @@ jobs:
575593
# `scripts/invoked-as.mjs` may read `process.argv[1]`, and that module's
576594
# own self-test drives a real probe through a real symlink. Rationale and
577595
# the rejected behavioural-sweep alternative: the gate script's header.
578-
# Scans ~115 files, no spawns; ~0.2s.
596+
# Scans ~115 files, no spawns; ~0.2s of work, measured at 0.62 min as a
597+
# step on run 35506407130.
598+
#
599+
# Scoped (#19498): its population is `scripts/**` — its own
600+
# ROOT_DIR_WATCH_HINTS declaration, held against the root it really walks
601+
# by its own self-test — so a group that changes no file under scripts/
602+
# (and no root configuration) skips it. The selection step at the top of
603+
# this job decides that, `push` on main and the hourly run keep it
604+
# unconditional, and the ruling that authorizes moving a tooling
605+
# self-test off the PR path is quoted in the selector's header.
579606
- name: scripts/ entry guards go through one predicate
607+
if: steps.gate-families.outputs.entry_guard != 'skip'
580608
run: pnpm check:entry-guard
581609

582610
# Every `scripts/**` TypeScript parse goes through ONE module (#10133 /
@@ -911,19 +939,28 @@ jobs:
911939
# listed, with no reader of what the test actually reads. What runs
912940
# here now is different in kind and was the maintainer's call (#16496,
913941
# 「同意你的建议,你负责执行派发所有可行的优化」): the selection step at
914-
# the top of this job classifies every changed path against this
915-
# self-test's MEASURED read-set (every workflow, every gate source under
916-
# `scripts/**` and `packages/*/scripts/**`, every `package.json`,
917-
# `.claude/**`, `skills/**`, `AGENTS.md`, `CLAUDE.md`, `tsconfig.json`,
918-
# every `.gitignore` -- nested ones included -- the CONTENT of every
919-
# JS/TS file in the tree (the compound-anchor census of
920-
# `function ...SelfTest...(` declarations and the exposed-scratch-dir
921-
# sweep of every mkdtempSync/mkdirSync caller both assert over it), and
922-
# the tracked NAME set it sweeps -- so any added, deleted or renamed
923-
# file runs it too), and skips this step only when every path is one
924-
# the test provably never reads: a modified doc, changeset or non-source
925-
# workspace file. Every doubt runs it, the self-test of the selector
926-
# pins that, and `push` on main keeps it unconditional.
942+
# the top of this job classifies every changed path against a declared
943+
# read-set for this step, and skips it when no changed path is in that
944+
# set. Every doubt runs it, the self-test of the selector pins that, and
945+
# `push` on main keeps it unconditional.
946+
#
947+
# ⚠️ Since #19498 that read-set is NARROWER than what the battery reads,
948+
# and the gap is stated rather than papered over. The read-set is the
949+
# tool's own inputs: every workflow and composite action, every gate
950+
# source under `scripts/**` and a workspace package's own `scripts/`,
951+
# every `package.json`, `.claude/**`, `skills/**`, `AGENTS.md`,
952+
# `CLAUDE.md` and root configuration. The battery ALSO reads the CONTENT
953+
# of every JS/TS and `.sh` file in the tree (the compound-anchor census
954+
# of `function ...SelfTest...(` declarations and the exposed-scratch-dir
955+
# sweep of every mkdtempSync/mkdirSync caller, with the nested
956+
# `.gitignore` files it consults) and the tracked NAME set it sweeps —
957+
# which is why, until #19498, any added file anywhere ran it. On PR
958+
# #19314 this step alone was 11.8 minutes of a 27.4-minute job for a
959+
# three-file `packages/spec` diff, and ruling #208 on #19491 (R4) took
960+
# those whole-tree reads off the PR path on the maintainer's sentence,
961+
# quoted in the selector's header. ⛔ A defect in that wider set can now
962+
# first appear on `main`; the push-on-main and hourly full runs are what
963+
# bound it, and widening the skip further is again a maintainer call.
927964
#
928965
# The gate runs the SELF-TEST only. The live derivation
929966
# (`node scripts/pm/dispatch-gates.mjs <path>`) answers a question about a
@@ -997,8 +1034,18 @@ jobs:
9971034
# names — the stronger rule ("a gate that enumerates a directory must
9981035
# declare one") was implemented, measured at 86 findings over 114
9991036
# enumerating gate files, and refused as an allowlist with a verdict
1000-
# attached. Reads the derivation once over the tracked corpus; ~5s.
1037+
# attached. Reads the derivation once over the tracked corpus; ~5s of
1038+
# work, measured at 0.40 min as a step on run 35506407130.
1039+
#
1040+
# Scoped (#19498): it imports `discoverFamilies` and `trackedFiles` from
1041+
# the dispatch derivation, so its inputs are the workflow tree, every
1042+
# gate source that discovery resolves, and the tracked NAME set — and
1043+
# only a name that DISAPPEARS can turn a live declaration dead, which is
1044+
# a structural change the selector already runs everything for. A group
1045+
# confined to product source, tests, docs or changesets skips it here and
1046+
# pays for it on `push` to main and on the hourly run.
10011047
- name: A declared gate population reaches the tree
1048+
if: steps.gate-families.outputs.declared_population_live != 'skip'
10021049
run: pnpm check:declared-population-live
10031050

10041051
# ADR-0087 D4's per-release correctness gate (#17080). The REAL run needs
@@ -1029,21 +1076,32 @@ jobs:
10291076
# `dispatch-gates.mjs` reads the worklist, and no verdict in it reaches a
10301077
# dispatch prompt.
10311078
#
1032-
# Unconditional, for the same reason as the step above: a self-test that
1033-
# can be skipped is the gap moving rather than closing. Reads the workflow
1034-
# tree and every gate source once; ~0.5s.
1079+
# Reads the workflow tree and every gate source once; ~0.5s of work,
1080+
# measured at 0.32 min as a step on run 35506407130.
1081+
#
1082+
# ⚠️ This step WAS unconditional, on the reasoning that a self-test which
1083+
# can be skipped is the gap moving rather than closing. #19498 scopes it
1084+
# anyway, and the trade is explicit rather than reasoned away: it imports
1085+
# the same derivation as the two steps above, so its inputs are the
1086+
# workflow tree and the gate sources, and on a group touching neither the
1087+
# gap does move — to `push` on main and the hourly full run, which keep
1088+
# the whole battery. Gate weakening is a maintainer floor; the sentence
1089+
# that authorizes this one is quoted in the selector's header.
10351090
- name: PM bare-root worklist self-test
1091+
if: steps.gate-families.outputs.bare_root_worklist != 'skip'
10361092
run: node scripts/pm/bare-root-worklist.mjs --self-test
10371093

10381094
# Part-of/closing-keyword guard self-test (#8476). The guard itself is a
10391095
# PR-scoped blocking check in its own workflow — it needs a pull request
10401096
# body to judge, which this job does not have — so what runs HERE is its
10411097
# self-test, which is the half with a verdict independent of any PR.
1042-
# Unconditional for the same reason as the two steps above: a self-test
1043-
# that runs only when someone remembers is a check whose coverage is a
1044-
# function of who remembered, and the failure it hides is quiet — a break
1045-
# in the verdict layer lands green and surfaces later as a card silently
1046-
# closed by the sentence written to keep it open.
1098+
# Unconditional, and not on the two steps above's borrowed reason — they
1099+
# are scoped since #19498 and this one is not, because the cost that
1100+
# bought that ruling is not here: this step is a tenth of a second, and a
1101+
# self-test that runs only when someone remembers is a check whose
1102+
# coverage is a function of who remembered. The failure it hides is quiet
1103+
# — a break in the verdict layer lands green and surfaces later as a card
1104+
# silently closed by the sentence written to keep it open.
10471105
#
10481106
# The self-test also pins the WIRING (the guard workflow still invokes
10491107
# the script, still subscribes to `edited`, still passes the body through
@@ -1727,8 +1785,17 @@ jobs:
17271785
# Invoked as `node scripts/…` rather than through a `pnpm check:*` alias:
17281786
# see the GATE INVOCATION IDIOM note at the top of this file. Reads
17291787
# `scripts/` and `.github/workflows/` off disk and spawns the selected
1730-
# self-tests; no network.
1788+
# self-tests; no network. Measured at 0.48 min as a step on run
1789+
# 35506407130.
1790+
#
1791+
# Scoped (#19498): its declared population is
1792+
# `scripts/**/*.mjs`, `scripts/**/*.mts` and `scripts/**/*.sh`, and the
1793+
# rest of its read-set is the workflow tree and `.github/actions` it
1794+
# discovers the runnable self-tests from — so a group touching none of
1795+
# those skips it here and runs it on `push` to main and on the hourly
1796+
# full run.
17311797
- name: Self-test workflow-command gate
1798+
if: steps.gate-families.outputs.self_test_workflow_commands != 'skip'
17321799
run: |
17331800
node scripts/check-self-test-workflow-commands.mjs --self-test
17341801
node scripts/check-self-test-workflow-commands.mjs
@@ -1806,8 +1873,11 @@ jobs:
18061873
# silently degrading every gate failure from "here is the command" into
18071874
# "no substitute available".
18081875
#
1809-
# Unconditional and un-`if:`-ed, like every self-test above it — an
1810-
# exemption is precisely what a self-test must not have, or the gap moves.
1876+
# Unconditional and un-`if:`-ed — an exemption is precisely what a
1877+
# self-test must not have, or the gap moves. (Five self-tests in this job
1878+
# do carry one since #19498: each cost minutes on every product PR, and
1879+
# each was moved by the ruling quoted in the selector's header. This one
1880+
# costs seconds and is not among them.)
18111881
#
18121882
# NO NETWORK, measured rather than assumed (#9898), because a self-test
18131883
# that reached GitHub would put this required context at the mercy of API
@@ -1848,9 +1918,10 @@ jobs:
18481918
# future edit invalidates silently and precisely the rows no reviewer reads.
18491919
# Nothing but this step is an instrument for them.
18501920
#
1851-
# Unconditional and un-`if:`-ed, like every self-test around it — an
1852-
# exemption is precisely what a self-test must not have, or the gap simply
1853-
# moves. One `--self-test` per `run:` block, deliberately: the masking shape
1921+
# Unconditional and un-`if:`-ed — an exemption is precisely what a
1922+
# self-test must not have, or the gap simply moves; the five self-tests
1923+
# scoped in #19498 are the ruled exception and this sub-second one is not
1924+
# among them. One `--self-test` per `run:` block, deliberately: the masking shape
18541925
# `check-step-collectors.mjs` guards is a block driving TWO OR MORE distinct
18551926
# scripts. A discovery collector over `scripts/pm/*.sh --self-test` — which
18561927
# would also catch the next such script arriving unwired — is ruled out of

0 commit comments

Comments
 (0)