You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Scoped test selection (--changed=origin/main on PRs) has produced three DISTINCT full-red CI modes in one week, each from a legitimate PR shape its design didn't anticipate:
In a one-shot gate culture any red check auto-closes contributor PRs, so each of these was a contributor-facing outage, not an inconvenience. Scoped selection is now DISABLED via the repo variable SCOPED_TEST_SELECTION_ENABLED=false (its own designed kill switch, set 2026-07-23): every PR runs the full sharded suite until this issue lands. Sharding itself has caused none of this and stays.
Requirements
⚠️ Success signal must come from the test run itself, not from coverage-file side effects. The verify step's job is 'the suite really ran and passed', and lcov emptiness is not evidence of that in scoped mode.
Rework 'Verify coverage report exists' for the scoped branch: treat vitest exit 0 + a written junit/blob report as the pass signal; an empty/missing lcov in scoped mode with a passing run is legitimate (record it as a step output so codecov upload is skipped the same way no_tests_matched already is).
Add the missing regression shapes to whatever harness guards ci.yml (test-only diff, packages-only diff, docs-only diff) so the next edge case is caught before it ships.
Only after both: re-enable the variable, in the same PR that documents the re-enable decision.
Problem
Scoped test selection (
--changed=origin/mainon PRs) has produced three DISTINCT full-red CI modes in one week, each from a legitimate PR shape its design didn't anticipate:no_tests_matchedguard after PR docs(discovery-plane): finalize operator guide now that the plane is shipped and live #8165 died on it.git fetch --depth=1cutting origin/main's history) — silently selected zero files on every scoped PR; fixed 2026-07-21.src/**files (test-only/docs-drift diffs, e.g. PR fix(docs): repair #8182 fallout — audit manifest, mangled runner YAML snippet, kill-switch route assertion #8192): vitest passes,--coverage.all=falsewrites an EMPTY lcov, and 'Verify coverage report exists' fails the green run. Theno_tests_matchedguard can't see this case — tests DID match.In a one-shot gate culture any red check auto-closes contributor PRs, so each of these was a contributor-facing outage, not an inconvenience. Scoped selection is now DISABLED via the repo variable
SCOPED_TEST_SELECTION_ENABLED=false(its own designed kill switch, set 2026-07-23): every PR runs the full sharded suite until this issue lands. Sharding itself has caused none of this and stays.Requirements
no_tests_matchedalready is).Links & Resources
.github/workflows/ci.yml (validate-tests), #8173, #8165, PR #8192 (case 3's victim), scripts/compute-test-shards.ts
Boundaries
Keep duration-aware sharding untouched. No coverage-threshold changes.
maintainer-only — CI trust infrastructure.