perf(ci): main push runs fall back to the whole tree from a stale diff base and a broad trigger list #6134

Problem

A push to main runs ci.yml over the whole tree in two cases. In the first, the range since the newest green push run touches the shared test machinery: ci.yml, .github/actions, the requirements files, the runner and selector scripts, scripts/lib, .shellcheckrc, the toolchain pins, or package*.json (.github/workflows/ci.yml:296-301 at f605a5963). In the second, no green push run among the last 50 is an ancestor of HEAD (ci.yml:286-295). #5886 made this behaviour deliberate.

Two things make whole-tree push runs more frequent than that design intends.

1. The resolved diff base is sometimes much older than the newest green ancestor. In three of today's whole-tree runs, a newer green push run was an ancestor of HEAD and its range touched none of the machinery paths. The step still picked a commit from days or weeks earlier, and that older range did touch the machinery. Why the loop skipped the newer runs is not known yet. The step does not log the candidate list, so the cause cannot be read from the run logs. Possible causes, all unconfirmed: the actions/workflows/ci.yml/runs?...&status=success listing returned an incomplete or stale page, or git merge-base --is-ancestor failed for the newer SHAs, with the error hidden by 2>/dev/null.
2. During CI rework almost every main push touches the machinery. That is by design, but the trigger list is broad. A change to one file in scripts/lib/ sends every suite to the whole-tree path, even when the selector could map that file to the suites that source it.

Evidence

All ci.yml push runs on main created since 2026-10-03T08:00Z: 22 runs, of which 21 succeeded and 1 was cancelled. "Whole tree" is taken from the scope job's notice annotation.

Wall time is updated_at - run_started_at, so it includes time spent queued behind the push concurrency group.

Whole-tree runs where the logged base was older than an available green ancestor whose range touched no machinery path:

The other four whole-tree runs (bb6da9540, d022d4bd3, 6bb7cd8c9, ceed24b17) would have gone whole-tree from the correct base anyway: each range changed ci.yml, scripts/affected-tests* or scripts/lib/*. Two of them also logged 790e56bde as their base.

Push runs share one concurrency group with cancel-in-progress: false (ci.yml:100-102), so a long whole-tree run also delays the next push. f192c0ff6 (run 37144369970) was created at 18:29:15Z, but its first job was not created until 18:37:49Z, the second the whole-tree run on ceed24b17 (11.4 min) finished.

Impact

• Main build time after merge: diff-based push runs have a p50 of 4.7 min and whole-tree runs 9.1 min. The three stale-base runs cost roughly 4-5 min each over a diff-based run (estimate: the whole-tree p50 minus the diff p50, not measured per run).
• A whole-tree push run holds the push concurrency group, so the next merge's push run waits behind it.

Proposed fix

1. Make the base resolution observable, then fix it. In "Resolve the diff base" (ci.yml:286-295), log how many runs the listing returned, the first few SHAs, and why each SHA was rejected (not an ancestor, or a git error). Drop the 2>/dev/null on merge-base so errors show. Once the cause is known: if the listing is incomplete, order by created_at and fall back to the parent of the previous push's before SHA. If it is a git error, fix the fetch.
2. Narrow the whole-tree trigger list (ci.yml:296-299):
   • scripts/lib/*: let scripts/affected-tests.sh map each changed lib file to the suites that source it (perf(ci): select only the suites a change runs or reads #6059 added same-language read edges to the selector) instead of forcing the whole tree. Keep the whole tree for scripts/lib/test-harness.sh only.
   • .github/requirements-ci*.txt: run that language's corpus, not every suite.
   • package.json / package-lock.json: run the Node suites only.
   • ci.yml / .github/actions: keep the whole tree, since a change to the gating itself is what the whole-tree run protects against.
3. Add a selector test case per narrowed path, so a later edit cannot quietly widen the list again.

Acceptance criteria

• The "Resolve the diff base" step logs the candidate count and the reason each SHA was rejected.
• Over 20 consecutive main push runs, no whole-tree run's logged base is older than the newest green push-run ancestor that had completed when its scope job started.
• A push whose range changes only one non-harness file under scripts/lib/, only a requirements-ci*.txt file, or only package-lock.json runs a scoped selection rather than the whole tree, with a test in scripts/affected-tests.test.sh for each case.
• Re-measure with the method above over a 24 h window after the change: the whole-tree share and the main push wall p50 are reported alongside today's 33% and 6.4 min.

Refs #3932 (CI performance target), #5886 (introduced the push-range base).