From 9b07fbb2dc5dd11a7f23b325c409618cfb27d76d Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 01:56:25 -0400 Subject: [PATCH 01/18] perf(ci): select tests by language-routed edges, resolved names and declared scopes The selector now takes a reference across languages only where the line runs or loads the file, ignores comment-only mentions in suites too, resolves ambiguous and structural basenames to the file they mean, follows Python imports, and replaces the whole-plugin rule and the always-run list with test-scope headers on the suites that scan a directory. An unmapped file can select only its own language's corpus (--unmapped-corpus, exit 4), and --replay shows a selector change's effect on recent main commits. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/ci-runner-routing.md | 9 +- scripts/affected-tests-always.txt | 14 - scripts/affected-tests-no-suite.txt | 37 +- scripts/affected-tests.sh | 1556 +++++++++++++------------- scripts/affected-tests.test.sh | 479 +++++--- scripts/check-docs-only-gate.test.sh | 6 - scripts/lib/gate-entry.test.sh | 4 + scripts/lib/test-harness.test.sh | 3 + 8 files changed, 1150 insertions(+), 958 deletions(-) delete mode 100644 scripts/affected-tests-always.txt diff --git a/docs/ci-runner-routing.md b/docs/ci-runner-routing.md index b575752e34..f9c8d7d4f2 100644 --- a/docs/ci-runner-routing.md +++ b/docs/ci-runner-routing.md @@ -104,10 +104,11 @@ four (four on the whole tree or an UNMAPPED file), and, per leg, whether its slice needs the animation wheels, the inventory's parser packages or the DuckDB CLI. A leg installs only those; the shfmt and DuckDB downloads are cached. -The selector's rule R8 covers the gap a full main run used to cover: a change -anywhere under `plugins/

/` also selects every shell suite under that plugin, -because suites that scan their own plugin directory never name the file that -changed. +A suite that scans a directory never names the file that changed, so it +declares what it reads in a `# test-scope: ` header, and the selector's +rule R8 selects it for any changed file matching the glob. The rules, and the +`--replay` mode that shows a selector change's effect on recent main commits, +are in the header of `scripts/affected-tests.sh`. ## Contract-only `ci-status` diff --git a/scripts/affected-tests-always.txt b/scripts/affected-tests-always.txt deleted file mode 100644 index 2d02e2c9b2..0000000000 --- a/scripts/affected-tests-always.txt +++ /dev/null @@ -1,14 +0,0 @@ -# Suites scripts/affected-tests.sh adds to EVERY selection under --with-always, -# which CI passes. Each asserts against the LIVE repository rather than against -# fixtures, so a change it never names can still break it, and no selection -# rule can see that edge. A suite whose live-tree case only repeats a `Check -# ...` gate step that runs on every diff does not belong here: the gate is -# already the always-run half. -# -# Format: one ` ` per line, parsed by scripts/lib/read-list.sh -# in `leading` mode. An entry naming no existing suite fails the selector -# (exit 2): an always-run list must not outlive what it lists. - -scripts/affected-tests.test.sh its LIVE cases derive the copy set from every scripts/sync-*.sh, read ci.yml's test-linux fan-out and the real no-suite list; a new manifest or a ci.yml edit reaches none of them by name -scripts/lib/gate-entry.test.sh scans every script under scripts/ for a hand-rolled base-ref predicate and for stale allowlist entries; a new gate script is never named by it -scripts/lib/test-harness.test.sh asserts every scripts/ suite that sources the harness ends on test_harness::report; a new suite is never named by it diff --git a/scripts/affected-tests-no-suite.txt b/scripts/affected-tests-no-suite.txt index 05a56ff031..5268b29496 100644 --- a/scripts/affected-tests-no-suite.txt +++ b/scripts/affected-tests-no-suite.txt @@ -119,21 +119,24 @@ plugins/miro/server/* # bare LICENSE file has no extension, so it needs its own line. LICENSE -# Eval fixtures carrying a non-markdown, non-JSON extension. `.md` and `.json` -# fixtures are already covered by the two entries above; a fixture shaped like -# shell or like an instruction file is deliberately given a `.txt` extension so -# that scripts/check-shell-portability.sh does not lint a file that is input -# data rather than code, and so a fixture shaped like a memory file is not named -# in a way that would load it as live instructions. That rename is what leaves -# these with no extension-based entry. +# Eval data: input the evals read, never code a suite runs, whatever its +# extension (a fixture repository's .js and .py, a case's scaffold.sh). # -# The lane covering them is the skill-quality gate, which reads every file under -# an evals/fixtures/ tree twice: scripts/check-orphaned-fixtures.sh --check -# fails when a fixture is consumed by no grader, and -# plugins/skill-quality/scripts/check-evals-quality.sh fails when a case names a -# fixture that does not resolve. A fixture that a shell suite genuinely names is -# still selected by the reference rule before this line is consulted. -plugins/*/evals/fixtures/*.txt +# Skill evals (plugins/

/skills//evals/): the skill-quality gate reads +# every file under an evals/fixtures/ tree twice, since +# scripts/check-orphaned-fixtures.sh --check fails when a fixture is consumed by +# no grader and plugins/skill-quality/scripts/check-evals-quality.sh fails when +# a case names a fixture that does not resolve. +# +# Plugin eval suites (plugins/

/evals//: case.yaml, prompt.md, +# scaffold.sh, graders/, samples/): an ON-DEMAND lane, not a CI one. They run +# under `claude plugin eval`, a paid model run, and /evals:validate +# (plugins/evals/skills/validate/scripts/validate-cases.py) checks a suite +# offline; both are run by hand, the way the Linear schema check below is. +# +# A file under an evals/ tree that a suite genuinely names is still selected by +# the reference rule before this line is consulted. +plugins/*/evals/* # Kept-trace fixtures for the evals plugin's result checkers. test_run_validity.py # and test_calibrate_judge.py read them through their fixture directories, never @@ -163,9 +166,3 @@ plugins/harness-ops/lib/plugin_cache_versions.py plugins/harness-ops/skills/audit-native-overlap/scripts/discover.py plugins/harness-ops/skills/inventory/scripts/docs_crosscheck.py -# The cleanup skill's eval fixture: a small repository (production modules, -# a config, a judge findings file) that the cleanup evals copy and run the -# skill over. The lane covering it is the skill-quality gate: -# scripts/check-orphaned-fixtures.sh --check and check-evals-quality.sh, as for -# the *.txt fixtures above. Its unittest files are still selected as suites. -plugins/testing/skills/cleanup/evals/fixtures/* diff --git a/scripts/affected-tests.sh b/scripts/affected-tests.sh index 0d08f53951..0ed16a7d7d 100755 --- a/scripts/affected-tests.sh +++ b/scripts/affected-tests.sh @@ -1,9 +1,8 @@ #!/usr/bin/env bash -# Select the test suites that cover a set of changed files, so a developer can -# run what their change actually affects instead of the whole corpus. Four -# ecosystems carry suites here and each names them differently: shell -# **/*.test.sh, Node **/*.test.js and **/*.test.mjs, Python **/test_*.py, and -# Pester **/*.Tests.ps1. +# Select the test suites that cover a set of changed files, so a change runs the +# suites it affects instead of the whole corpus. Four ecosystems carry suites +# here and each names them differently: shell **/*.test.sh, Node **/*.test.js +# and **/*.test.mjs, Python **/test_*.py, and Pester **/*.Tests.ps1. # # The full corpus is tens of minutes of wall clock on a Windows box # (Git Bash pays ~140ms per process spawn, and these suites are spawn-bound), @@ -16,33 +15,23 @@ # scripts/affected-tests.sh --base use as the diff base (default: origin/main) # scripts/affected-tests.sh --explain report WHY each suite was selected (stderr) # scripts/affected-tests.sh --allow-unmapped downgrade an unmapped file to a warning +# scripts/affected-tests.sh --unmapped-corpus select an unmapped file's whole language corpus (exit 4) # scripts/affected-tests.sh --shard / keep only leg i of n of the selection -# scripts/affected-tests.sh --with-always add the live-tree suites in scripts/affected-tests-always.txt # scripts/affected-tests.sh --print-fanout P print the copy set DERIVED for shared source P +# scripts/affected-tests.sh --replay [--against ] +# select every first-parent commit of against +# its parent; with --against, diff each selection with +# the selector at (see REPLAY) # -# ALWAYS-RUN. A few suites assert against the LIVE repository (every sync -# manifest, every script under scripts/), so a change none of them names can -# still break them. --with-always adds the suites listed in -# scripts/affected-tests-always.txt to the selection; CI passes it, and a local -# run leaves it off because those suites are slow on a Windows host. An entry -# naming no suite is an error (exit 2), not a quiet skip. -# -# SHARDING. `--shard /` narrows the SELECTION, not the derivation: every -# rule below runs in full, the unmapped check fires in full, and only then is -# the sorted suite list partitioned by index modulo n. Legs are therefore a -# partition in the mathematical sense: the union of legs 0..n-1 is exactly the -# unsharded selection and no two legs share a suite. That is the property CI -# needs when it fans one selection across n runners and calls the change tested. -# Modulo, not contiguous blocks: the sorted list clusters suites by directory, -# so a block partition hands one leg a whole slow plugin while another gets -# nothing, and interleaving spreads the clusters. An EMPTY leg is not an error; -# it exits 0, because "this leg had nothing to run" and "nothing was affected" -# are the same statement about that runner. +# --with-always is accepted and changes nothing: the suites that assert against +# the live tree declare what they read with a test-scope header (R8) instead. # # Exit: 0 selected (or nothing to do); 1 an unmapped changed file, or a failing # suite under --run; 2 usage or a broken derivation; 3 --run ran every shell # suite it selected but ALSO selected suites in other ecosystems, whose runner -# it deliberately will not guess (see the --run note at the foot of this file). +# it deliberately will not guess (see the --run note at the foot of this file); +# 4 --unmapped-corpus widened the selection to the corpus of an unmapped file's +# language. Under --run the first that applies wins, in the order 1, 3, 4. # # HOST: --run IS A LINUX GATE. On a Windows Git Bash host a standing set of # suites fails for reasons that belong to the host and not to the tree: text-mode @@ -54,299 +43,159 @@ # the listing forms to see what a change affects and run individual suites by # hand. CI's Linux lanes are the gate that decides. # -# DIRECTION: over-selection is safe, under-selection is not. Every rule below is -# deliberately generous, because a suite that runs needlessly costs seconds, -# while a suite that should have run and did not is the regression this tool -# exists to stop. The one narrowing is COMMENTS below: a whole-line comment in a -# non-suite file no longer makes that file a dependent. -# -# COMMENTS. In a NON-suite file, a line that is only a comment (`#` in shell, -# Python and PowerShell; `//`, `/*` or a `*` block-comment continuation in Node) -# creates no R4 dependent. A `# shellcheck source=` directive and a JSDoc type -# import (`@import`, `import('...')`) are read by tools, not people, and still -# count, as does a trailing comment on a code line. So does a comment naming -# .py in a .py that imports by module name (`import `, -# `from import`): the import never spells the .py, so the comment is the -# only text edge to the module. Suites match on every -# line, so R3 is unchanged. Hub files cite the scripts they sit beside in prose: -# lib/hook-utils.sh and its 20 plugin copies name run-guards.sh only in -# comments, so a change to run-guards.sh made every copy a "dependent" and pulled -# in every suite naming hook-utils.sh — 278 to 328 of ~450 suites on 19% of -# sampled pull requests, most of the corpus for a change to one plugin. The cost -# of the narrowing is bounded only for shell suites: CI runs the shell corpus on -# main twice a day. Python and Node suites that no .test.sh wraps run only when -# selected, so a comment-only edge into one of them is not re-checked on main. +# SHARDING. `--shard /` narrows the SELECTION, not the derivation: every +# rule below runs in full, the unmapped check fires in full, and only then is +# the sorted suite list partitioned by index modulo n. The union of legs +# 0..n-1 is exactly the unsharded selection and no two legs share a suite. +# Modulo rather than contiguous blocks, because the sorted list clusters suites +# by directory. An EMPTY leg exits 0: "this leg had nothing to run" and +# "nothing was affected" are the same statement about that runner. # # FAIL LOUD, NOT OPEN. A changed file that maps to NO suite is an ERROR, not an # empty selection: "zero suites" reads as "nothing to run" when it actually -# means "nothing here knows what covers this". The only exceptions are the path +# means "nothing here knows what covers this". The exceptions are the path # classes recorded in scripts/affected-tests-no-suite.txt, each of which names -# the non-shell CI lane that does cover it. Anything else fails, and -# --allow-unmapped is the one-flag escape. +# the lane that does cover it, and deletions, which have no content left to +# cover. --allow-unmapped downgrades the error to a warning; --unmapped-corpus +# keeps the report and adds every suite of the file's language to the selection: +# the shell corpus for .sh and .bash, Python for .py, Node for .js .mjs .cjs, +# Pester for .ps1 .psm1, and the shell corpus for any other file, because shell +# suites are the ones that read data files out of the tree. # -# SELECTION RULES +# SELECTION RULES. A suite runs when the changed file is code the suite runs or +# loads, or data the suite (or code it runs) reads. Every rule finds that +# relation from text and paths, so each one is written to say no when the text +# does not show the relation. # R1 self a changed suite selects itself, in any ecosystem. # R2 co-located

/. selects the sibling suites covering it, -# under each ecosystem's OWN naming convention: .test.sh, -# .test.js, .test.mjs, .Tests.ps1, and -# /test_.py — the last also with `-` folded to `_`, -# which is how this repo names the Python suites covering its -# hyphenated scripts (check-manifest-duplicate-keys.py -> -# test_check_manifest_duplicate_keys.py) — and -# /tests/test_.py, the one convention here that -# puts the suite in a SUBDIRECTORY rather than beside the -# file (13 of the 59 non-suite .py files in this repo). R3 -# cannot stand in for that arm and never could: a Python -# suite reaches its subject with `import babysit_lease`, so -# the filename is never spelled and there is nothing for a -# text match to find. EVERY match is taken, -# never just the first: a .py can carry both a test_.py -# and a wrapping .test.sh, and stopping at one -# under-selects, which is the unsafe direction. -# R3 referenced any SUITE that NAMES the file — carries its basename as a -# whole path token, see MATCHING below — is a covering suite -# (it names the file, so it exercises it). This rule is a text -# match and therefore language-agnostic: a Pester suite -# dot-sourcing Foo.ps1 and a Node suite importing foo.js are -# found exactly the way a shell suite is. Widening the corpus -# is what taught it the other three ecosystems; the rule -# itself did not change. -# R4 dependents any other source file (.sh .bash .js .mjs .cjs .py .ps1 -# .psm1 — the same set the reverse-lookup pathspec searches, -# and the same set lang_family() names; all three move -# together or a path is classified into a family nothing ever -# greps) that -# NAMES the file the same way, on a line that is not only a -# comment (see COMMENTS above), is a dependent; R2/R3 are -# then applied to IT, transitively. This is what carries a lib -# change out to the hooks that source it. -# R5 shared-lib a file that is the `src` of a scripts/sync-*.sh selects -# every path in that script's published `copy` list, and then -# R2/R3/R4 on each copy. The copy set is DERIVED by invoking -# `--print-manifest` on every run, never hardcoded here and -# never scraped out of `src=` / `copies=(` source text: the -# manifests are what CI's *-sync lanes enforce, so a new -# carrying plugin is picked up the moment it exists. Deriving -# it is the whole point — a list copied into this file would -# silently rot, and the rot would show up as an under-selection. +# under each ecosystem's own naming: .test.sh, +# .test.js, .test.mjs, .Tests.ps1, +# /test_.py and /tests/test_.py, the +# two Python forms also with `-` folded to `_`. Every match is +# taken: a .py can carry a test_.py and a wrapping +# .test.sh at once. +# R3 same language a file in the changed file's language that NAMES it (see +# MATCHING) on a line that is not only a comment is a +# dependent; R1, R2 and R3 then apply to it, transitively, +# with no depth cap. A suite that names it is selected. This is +# what carries a library change out to what sources it. +# R4 other language a file in another language counts only where the naming +# line runs or loads the file: an interpreter or process API on +# the line (bash, sh, python3, node, pwsh, source, subprocess, +# spawn*, exec*, ...), a path to the file rather than its bare +# name, or a shell script as the named file (another language +# has no other use for one). A chain takes at most one such +# transition and then keeps walking its new language freely. +# A data file (any extension that is not code) reaches code of +# every language that names it, and that first step spends no +# transition: data has no language of its own to stay inside. +# R5 shared lib a file that is the `src` of a scripts/sync-*.sh selects +# every path in that script's published `copy` list, then R2, +# R3 and R4 on each copy. The copy set is DERIVED from +# `--print-manifest` on every run, never hardcoded or scraped, +# because the manifests are what CI's sync lanes enforce. # R6 sync script a changed scripts/sync-*.sh selects its own co-located test -# plus everything its published `src` selects, since its -# failure mode is the copies drifting from that source. +# plus everything its published `src` selects. # R7 path class a path under plugins/autonomy/reference/ selects the -# plugin-contract validator's suite. The validator bans -# vendor names across that whole directory, but its files -# are markdown that no suite names, so R1-R4 never reach -# them and they fell to the no-suite *.md class. Only a -# path rule can see them. The validator's fleet-token ban -# over the rest of plugins/autonomy/ is not mapped here; -# CI's manifest validation step runs the validator itself -# on every diff. -# R8 plugin any changed path under plugins/

/ also selects every -# shell suite (*.test.sh) under plugins/

/. Suites that -# scan their own plugin directory (a markdown lint, a -# manifest or prose check) cover files no rule above can -# reach: a SKILL.md edit is a no-suite class and a suite -# that globs its plugin never spells the file's name. Both -# suite breaks that only a full main run caught had that -# shape. Shell suites only, because they are -# what the whole-tree run executes; a plugin's Node, Python -# and Pester suites keep reaching a change through R1-R4. -# R8 ADDS suites and never MAPS a file: whether a changed -# file is UNMAPPED is still decided by R1-R7 alone, so the -# FAIL LOUD contract below is unchanged. -# -# R3/R4 skip STRUCTURAL basenames — README.md, SKILL.md, plugin.json and the -# like — because those name a repo-wide role rather than one artifact, so a -# basename match carries no coverage signal (SKILL.md alone appears in 20 -# unrelated suites). Such files fall through to R2, then to the no-suite list. -# -# The transitive walk has no depth cap WITHIN one ecosystem. It terminates on -# the visited set, and its worst case is selecting every suite — the safe -# direction. -# -# ACROSS ecosystems each path may take exactly ONE language transition, and this -# asymmetry is load-bearing rather than tidiness. Within a language, "B's text -# contains A's basename" is a real dependency: shell sources shell, Node imports -# Node. Across languages it usually is not — a .ps1 that happens to contain the -# characters "paths.js" is not loading it — so chaining those coincidences -# compounds them. Measured before this rule existed, with the corpus widened to -# four ecosystems and the walk left uncapped: EVERY .js file in the repo selected -# the same 156 of 439 suites, and one selected 376, because the walk crossed -# js -> ps1 -> sh and saturated on the hubs at the far end -# (Assert-CheckResult.ps1, hook-utils.sh). An answer identical for every file in -# a language carries no information about which file changed, which is the tool -# failing at its whole job even though it fails in the "safe" direction. -# -# The budget is one TRANSITION per path, not one hop. A crossed-to file keeps -# walking its OWN language freely; what it may not do is cross a second time. -# The distinction matters and the weaker "one hop then stop" rule was wrong: -# helper.py -> runner.sh -> command.sh -> command.test.sh is a real chain whose -# second edge is shell-to-shell, and stopping dead at runner.sh dropped -# command.test.sh — an under-selection, the direction this file calls unsafe. It -# is the repeated re-crossing that saturates, not depth within one language. -# -# This costs nothing that was ever load-bearing: before the corpus was widened -# the reverse lookup was `-- '*.sh'`, so cross-language edges did not exist at -# all and no shell-to-shell chain is shortened by any of this. -# -# MATCHING, for R3 and R4. One file NAMES another when the basename stands -# there as a WHOLE PATH TOKEN: bounded on both sides by a character that cannot -# occur inside a single path component, which is anything outside -# [A-Za-z0-9_.-]. `/` is deliberately OUTSIDE that class, so a path-qualified -# mention names the file — `source "$dir/hook-utils.sh"`, `"./gadget.js"`, -# `. (Join-Path $PSScriptRoot 'Get-Thing.ps1')` — and so does a bare mention in -# prose or, in a suite, a comment. A leading or trailing run of `.` is sentence punctuation -# rather than part of a name, so a comment ending "... is covered by -# .test.sh." names that suite too — which is how most of this repo's -# comments cite the suite covering them. -# -# A leading run of shell PARAMETER-EXPANSION OPERATOR characters is stripped for -# the same reason: `"${TARGET:-.sh}"` is a mention of .sh, but the -# `-` of the `:-` sits INSIDE the token class, so without the strip the token is -# `-.sh` and nothing matches. The stripped set is `-`, `+`, `=`, `?` — every -# operator character of the `${V:-x}` / `${V:+x}` / `${V:=x}` / `${V:?x}` family -# and its colon-less forms. Only `-` is load-bearing today: the other three are -# already outside [A-Za-z0-9_.-] and so already end a token on their own. They -# are stripped anyway so the two sets cannot drift apart if the token class is -# ever widened. This errs toward OVER-selection — a token that merely BEGINS -# with one of those characters now names the file — which is the direction -# DIRECTION above calls safe, and every pair it admits is one the pre-token -# substring rule admitted too. What it buys is the failure it removes: without -# it, a file whose only mention from some dependent is `${VAR:-.sh}` still -# looked MAPPED whenever it had a co-located suite, so the run exited 0 while -# that dependent's suite was quietly left out. A silent under-selection is -# exactly what this tool exists to refuse. +# plugin-contract validator's suite, which bans vendor names +# across that directory without naming any file in it. +# R8 declared scope a suite that enumerates a directory of the live tree +# (a grep -r, a find, a glob over a plugin or scripts/) never +# names the files it reads, so it declares them in its header: +# # test-scope: plugins/github/*.md +# (`//` for Node). A changed file matching a glob selects the +# suite and counts as mapped. The globs use the dialect of the +# no-suite list: matched against the repo-relative path, `*` +# crosses `/`. Only the leading comment block is read, so a +# fixture line further down can never declare one; a +# changed suite whose declaration sits below that block, or +# names a glob matching no file, fails the run (exit 2). # -# What no longer counts is a basename buried INSIDE a longer token: -# `handoff-paths.json` is not a mention of `paths.js`, -# `status.showUntrackedFiles` is not a mention of `status.sh`, and -# `common.sh.tmpl` is not a mention of `common.sh` — all three are real hits -# this corpus used to serve. Those were not merely noisy over-selection. They -# were FALSE COVERAGE, which is the fail-OPEN direction: a file nothing covers -# came back with a non-empty selection and exit 0 instead of failing unmapped, -# so the FAIL LOUD contract above silently did not apply to it. Measured on this -# corpus: a new hooks helper named with a basename that another path merely ENDS -# with, covered by nothing at all, selected 131 suites at exit 0 — most of the -# shell corpus, and an answer that would have been the same for any name in that -# collision class; under this rule the same file is UNMAPPED. (The count tracks -# the corpus and will drift; "most of it" is the part that matters.) Naming a -# file "distinctively" was never a defense — whether the collision happens is -# decided by every OTHER path already in the repo, not by how the new name reads -# on its own. This paragraph deliberately does not spell that helper's basename: -# a name written HERE is a name this file then references, and the tool would map -# the file to this hub instead of failing loud, which is the very report the -# example exists to describe. Every example below is spelled the same careful -# way, and for the same reason. +# MATCHING. One file NAMES another when the basename stands in a line as a WHOLE +# PATH TOKEN: bounded on both sides by a character outside [A-Za-z0-9_.-]. `/` +# is outside that class, so a path-qualified mention names the file. A trailing +# run of `.` is sentence punctuation and is dropped, and so is a leading run of +# `-`, `+`, `=`, `?` or `.`, so `${TARGET:-}` and `...` name . +# A basename buried inside a longer token (`handoff-paths.json` against +# `paths.js`) is not a mention: a substring match there is FALSE COVERAGE, a +# file nothing covers coming back mapped at exit 0. A basename the token rule +# cannot spell, one with a character outside the class, keeps the substring +# test rather than losing its coverage. # -# The class cuts BOTH ways, and this is the strict edge of the rule: a -# token-class character abutting the basename ends the match exactly as a letter -# does, because `-`, `_` and `.` are INSIDE the class while `/` is not. With the -# leading strips above, that is now a TRAILING-side statement only: -# `-shaped` in a sentence stops naming , while a leading `-`, the one -# `${2:-}` puts in front of a default, does not. The asymmetry is forced -# rather than chosen — a leading strip cannot reach the trailing case, where the -# whole token is `-shaped` and there is no prefix to remove. Two files in -# this corpus are mentioned in the trailing form today and keep their suites only -# because they are ALSO named in bounded form elsewhere. A file mentioned only in -# such a form loses R3/R4 outright — and that lands LOUD rather than silent: with -# no other rule reaching it, the file is reported UNMAPPED at exit 1, which is -# the fail-CLOSED direction this tool is built to shout about, not the silent -# under-selection it is built to refuse. +# AMBIGUOUS NAMES. A basename two or more files carry, and the structural names +# (README.md, SKILL.md, AGENTS.md, CLAUDE.md, index.md, CHANGELOG.md, LICENSE, +# plugin.json, marketplace.json, settings.json, hooks.json, package.json, +# package-lock.json), name a specific file only when the mention RESOLVES to +# it, because a bare `SKILL.md` or `config.json` says nothing about which one. +# Any mention from the file's own directory resolves. Elsewhere a bare name +# never does, and a path does when it ends in the shortest suffix of the file's +# path that no other file of that name ends in, or in the file's path relative +# to a directory that holds both files: `$SCRIPT_DIR/lib/x.sh` from a script +# beside lib/, `$PLUGIN_DIR/skills/interview/SKILL.md` or +# `$PLUGIN_ROOT/hooks/hooks.json` from inside the plugin. Shared-library +# basenames are the exception and keep the plain rule: R5's copies share a +# basename on purpose, change together with their source, and a suite naming +# its own plugin's copy is naming the shared source. # -# SWEPT over every tracked file when this landed: 534 files selected fewer -# suites (15.8% fewer selected-suite slots across the tree), 4 files that had -# been UNMAPPED became mapped, and NOTHING became unmapped. 25 files did drop to -# an EMPTY selection — every one a markdown context file whose basename had been -# landing inside a longer one (a `.md` matching inside a -# `-.md`), and every one already covered by a class in -# scripts/affected-tests-no-suite.txt, so they report as no-suite at exit 0 -# rather than as a finding. "Nothing became unmapped" is therefore the true -# statement, and "nothing lost its whole selection" is NOT. +# COMMENTS. A line that is only a comment (`#` in shell, Python and +# PowerShell; `//`, `/*` or a `*` continuation in Node) names nothing, in suites +# and in code alike: prose that cites a file is not a dependency on it. A +# `# shellcheck source=` directive and a JSDoc type import (`@import`, +# `import('...')`) are read by tools and still count, as does a trailing +# comment on a code line. # -# That sweep was measured before the expansion-operator strip landed, and the -# strip only ADDS pairs back, every one of them a pair the old substring rule -# also served. So it moves the counts toward the pre-rule baseline and can -# reverse none of the sweep's directions: nothing that was mapped becomes -# unmapped, and no file gains a suite the old behavior did not already give it. -# Measured on the corpus as it stands, the strip re-admits two (file, basename) -# pairs in total, one of which R3/R4 discards anyway as a structural basename. +# PYTHON IMPORTS. An import never spells the .py, so R3 also reads Python +# import lines: `import foo`, `from foo import x`, `from . import foo` and the +# dotted forms name foo.py (or the package foo/__init__.py) when the importer +# sits in the directory foo is imported from or below it, when the dotted path +# spells the path of foo, or when foo is the only module of that name in the +# importer's plugin, the reach of a sys.path insert. # -# SKILL OWNERSHIP. Skills reuse reference and script names freely, so inside a -# skill directory, plugins//skills//, that carries its own , -# a bare means that one, never another skill's file of the same name. -# When every frontier file carrying a basename sits inside a skill directory, a -# hit keeps the pair only when one of these holds: -# - the hit lies inside one of those same skill directories and its line does -# not name the file only through another skill; -# - the matched line spells the file path-qualified: a path token ending in -# / with / as one of its components, which the repo-relative -# path also satisfies. A token that spells plugins//skills// -# must match the owning plugin as well as the skill; -# - the line mentions the file bare, or under a path that does not run through -# skills//, and the hit's own skill directory, if it has one, -# carries no file of that basename. -# A path token through skills// names another skill's file and never keeps -# the pair, wherever the hit lives. A bare mention inside a skill that carries -# its own file of that name is that skill citing its own file, and is dropped -# too. A skill WITHOUT such a file keeps the pair, because its plain mention can -# only mean somebody else's file: a test reaching a sibling skill's script as -# `$/`, or building the path from parts, spells no skill name. +# REPLAY. `--replay ` selects every first-parent commit of +# (`git rev-list --first-parent `) against its parent, the squash-merged +# pull request's net diff, in a scratch clone checked out at that commit, with +# THIS script's rules, no-suite list and test-scope declarations, so it answers +# "what would this selector have run for those pull requests". It prints one +# `commit ` line per commit, an indented +# `unmapped ` line per unmapped file and one indented +# ` ()` line per suite. With `--against ` it also runs the +# selector at , with 's own lists, on the same tree, and prints only +# the suites that differ (`+` this script only, `-` only, each with its +# reason) after a `commit ` line +# and its `unmapped` lines, +# so a selector change shows its blast radius. Both sides run with +# --allow-unmapped; a summary on stderr counts suites per commit (p50, p95, +# max, total) and the commits with an unmapped file on each side. The replay +# points each run at the scratch clone with AFFECTED_TESTS_ROOT and hands it a +# test-scope table with AFFECTED_TESTS_SCOPES; AFFECTED_TESTS_NO_SUITE names +# the no-suite list. # -# A basename any frontier file carries OUTSIDE a skill directory keeps the plain -# basename rule for every hit, and that is what leaves R5 untouched: its shared -# sources live outside skill directories, and a source that sits in one enters -# the frontier on the same level as its copies, so each copy's own skill is one -# of the owners and that skill's suites still match it bare. A copy changed on -# its own, apart from its source, no longer reaches the other copies' suites; R5 -# fans out from the source, and the sync lane gates a copy that drifts from it. -# -# Three things stay deliberately generous, all in the over-selecting direction -# (the comment rule above is the one deliberate narrowing): -# - a token match on the SAME basename in ANOTHER directory counts for any -# basename a frontier file carries outside a skill directory. There it is a -# basename rule and has to stay one: R5's entire fan-out is copies that share -# a basename across directories (lib/hook-utils.sh -> -# plugins/*/hooks/hook-utils.sh), so a suite naming its own plugin's copy is -# naming the shared source. Requiring the whole repo-relative path would cut -# that, which is under-selection. -# - a mention in a suite's comment counts, as does a trailing comment on a -# code line in any file. -# - a basename the token rule cannot express — one carrying a character -# outside [A-Za-z0-9_.-], which no tracked path in this repo does today — -# falls back to the old substring test rather than to no coverage at all. A -# name this rule cannot spell must over-select, never quietly stop matching. -# -# MECHANICALLY this is two stages, and the split is a measured cost rather than -# taste. `git grep -F` keeps its fixed-string fast path (~4s over this corpus -# for a 559-basename level); spelling the boundaries as an ERE alternation -# instead — `git grep -o -E`, one bounded pattern per basename — took over two -# minutes for that same level, which is not a usable per-level cost. So git grep -# still finds the candidate LINES with the substring test, and one awk pass over -# those lines (~0.06s) drops the lines COMMENTS excludes, splits each remaining -# line into path tokens and keeps only the pairs whose token IS one of the -# basenames asked about and that SKILL OWNERSHIP lets stand. +# MECHANICALLY the reverse lookup is two stages. `git grep -F` finds the +# candidate LINES with the substring test, which keeps git's fixed-string fast +# path (an ERE alternation of bounded basenames took minutes per level); one +# awk pass then drops comment lines, splits each line into tokens, and keeps the +# (file, name) pairs MATCHING and AMBIGUOUS NAMES let stand, marking each with +# whether its line runs or loads the file. # Both stages fail loud; see the call site in select_for. set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" || exit 2 -cd "$SCRIPT_DIR/.." || exit 2 +SELF="$SCRIPT_DIR/${BASH_SOURCE[0]##*/}" +cd "${AFFECTED_TESTS_ROOT:-$SCRIPT_DIR/..}" || exit 2 # shellcheck source=lib/changed-files.sh . "$SCRIPT_DIR/lib/changed-files.sh" || exit 2 # shellcheck source=lib/read-list.sh . "$SCRIPT_DIR/lib/read-list.sh" || exit 2 NO_SUITE_LIST="${AFFECTED_TESTS_NO_SUITE:-scripts/affected-tests-no-suite.txt}" -ALWAYS_LIST="${AFFECTED_TESTS_ALWAYS:-scripts/affected-tests-always.txt}" -# Basenames that name a structural role rather than one artifact. R3/R4 ignore -# them; see the note above. -STRUCTURAL_BASENAMES="README.md SKILL.md AGENTS.md CLAUDE.md CHANGELOG.md -plugin.json marketplace.json settings.json hooks.json package.json -package-lock.json index.md LICENSE" +# Basenames that name a repository-wide role, reached only through a resolved +# mention (AMBIGUOUS NAMES in the header), however few files carry them today. +STRUCTURAL_BASENAMES=" README.md SKILL.md AGENTS.md CLAUDE.md index.md CHANGELOG.md LICENSE " +STRUCTURAL_BASENAMES+="plugin.json marketplace.json settings.json hooks.json package.json package-lock.json " # Print the header block (everything after the shebang up to the first -# non-comment line) with its comment markers stripped. Derived rather than a -# hardcoded line range, which silently truncated as the header grew. +# non-comment line) with its comment markers stripped. usage() { awk 'NR == 1 { next } /^#/ { sub(/^# ?/, ""); print; next } { exit }' \ "${BASH_SOURCE[0]}" @@ -355,24 +204,24 @@ usage() { base_ref="" do_run=0 allow_unmapped=0 +unmapped_corpus=0 explain=0 -with_always=0 print_fanout="" shard_spec="" +replay_range="" +against_ref="" jobs=1 -# Whether --shard was SUPPLIED, tracked apart from its value. `--shard=` with an -# empty right-hand side is what an environment variable that expanded to nothing -# produces, and a presence test on the value alone would read it as "no shard -# requested" and run the whole selection on every leg while reporting success. +# Whether --shard was SUPPLIED, tracked apart from its value: `--shard=` with an +# empty right-hand side is what an unset environment variable produces, and it +# must not read as "no shard requested" and run everything on every leg. shard_given=0 shard_index=0 shard_total=1 declare -a explicit_paths=() # parse_shard : accept exactly `/` with i and n decimal, n >= 1 and -# 0 <= i < n. Rejected whole-string rather than by prefix: a spec this function -# cannot read must never silently become leg 0 of 1, because that leg RUNS -# EVERYTHING and would report a full pass from a typo'd fan-out. +# 0 <= i < n. A spec this function cannot read must never become leg 0 of 1, +# which RUNS EVERYTHING and would report a full pass from a typo'd fan-out. parse_shard() { local spec="$1" i n [[ "$spec" =~ ^[0-9]+/[0-9]+$ ]] || return 1 @@ -388,98 +237,76 @@ parse_shard() { return 0 } +# need_value : usage errors exit 2, never 1, which is +# spoken for by an unmapped file and a failing suite. +need_value() { + if [[ "$2" -lt 2 || -z "$3" ]]; then + echo "error: $1 needs a value ($4)." >&2 + exit 2 + fi +} + while [[ $# -gt 0 ]]; do case "$1" in -h | --help) usage exit 0 ;; - --run) - do_run=1 - shift - ;; - --allow-unmapped) - allow_unmapped=1 - shift - ;; - --explain) - explain=1 - shift - ;; - --with-always) - with_always=1 - shift - ;; + --run) do_run=1 ;; + --allow-unmapped) allow_unmapped=1 ;; + --unmapped-corpus) unmapped_corpus=1 ;; + --explain) explain=1 ;; + --with-always) ;; --jobs) - # Same as --base below: usage errors exit 2, not 1. - if [[ $# -lt 2 || -z "$2" ]]; then - echo "error: --jobs needs a positive integer." >&2 - exit 2 - fi + need_value "$1" $# "${2:-}" "a positive integer" jobs="$2" - shift 2 - ;; - --jobs=*) - jobs="${1#--jobs=}" shift ;; + --jobs=*) jobs="${1#--jobs=}" ;; --base) - # Checked explicitly rather than with `${2:?...}`, which exits 1. Exit 1 is - # already spoken for twice here — an unmapped changed file, and a failing - # suite under --run — and the header documents usage errors as 2. A caller - # that branches on the code cannot tell a typo'd flag from a real finding. - if [[ $# -lt 2 || -z "$2" ]]; then - echo "error: --base needs a ref." >&2 - exit 2 - fi + need_value "$1" $# "${2:-}" "a ref" base_ref="$2" - shift 2 - ;; - --base=*) - base_ref="${1#--base=}" shift ;; + --base=*) base_ref="${1#--base=}" ;; --shard) - # Same as --base above: usage errors exit 2, not 1. - if [[ $# -lt 2 || -z "$2" ]]; then - echo "error: --shard needs /." >&2 - exit 2 - fi + need_value "$1" $# "${2:-}" "/" shard_spec="$2" shard_given=1 - shift 2 + shift ;; --shard=*) shard_spec="${1#--shard=}" shard_given=1 - shift ;; --print-fanout) - # Same as --base above: usage errors exit 2, not 1. - if [[ $# -lt 2 || -z "$2" ]]; then - echo "error: --print-fanout needs a path." >&2 - exit 2 - fi + need_value "$1" $# "${2:-}" "a path" print_fanout="$2" - shift 2 + shift + ;; + --replay) + need_value "$1" $# "${2:-}" "a revision range" + replay_range="$2" + shift + ;; + --against) + need_value "$1" $# "${2:-}" "a ref" + against_ref="$2" + shift ;; --) shift - while [[ $# -gt 0 ]]; do - explicit_paths+=("$1") - shift - done + explicit_paths+=("$@") + break ;; -*) echo "error: unknown option: $1" >&2 usage >&2 exit 2 ;; - *) - explicit_paths+=("$1") - shift - ;; + *) explicit_paths+=("$1") ;; esac + shift done if [[ ! "$jobs" =~ ^[1-9][0-9]*$ ]]; then @@ -490,6 +317,18 @@ if [[ "$shard_given" -eq 1 ]] && ! parse_shard "$shard_spec"; then echo "error: --shard wants / with total >= 1 and 0 <= index < total; got: $shard_spec" >&2 exit 2 fi +if [[ "$allow_unmapped" -eq 1 && "$unmapped_corpus" -eq 1 ]]; then + echo "error: --allow-unmapped and --unmapped-corpus answer the same question two ways; pass one." >&2 + exit 2 +fi +if [[ -n "$against_ref" && -z "$replay_range" ]]; then + echo "error: --against only applies to --replay." >&2 + exit 2 +fi +if [[ -n "$replay_range" ]] && [[ ${#explicit_paths[@]} -gt 0 || -n "$base_ref" || "$do_run" -eq 1 || "$shard_given" -eq 1 ]]; then + echo "error: --replay takes its changed files from each commit; it combines with no paths, --base, --run or --shard." >&2 + exit 2 +fi WORK_DIR="$(mktemp -d "${TMPDIR:-/tmp}/affected-tests.XXXXXX")" || exit 2 trap 'rm -rf "$WORK_DIR"' EXIT @@ -524,9 +363,7 @@ register_sync_block() { fi SYNC_SCRIPT_SRC["$script"]+="$src"$'\n' # A src with NO copy key at all is a canonical-only cluster: the lib has - # landed and no plugin carries it yet. That is not the rot the zero-yield - # guard below catches (copy patterns declared, none matching anything), so - # it registers with an empty copy set and R5/R6 resolve to the src alone. + # landed and no plugin carries it yet. It registers with an empty copy set. if ((has_copy_key == 0)); then SYNC_SRC_COPIES["$src"]="" return 0 @@ -607,13 +444,10 @@ build_sync_map() { exit 2 fi - # A script matching scripts/sync-*.sh that publishes NEITHER key is not a - # copy manifest — it is a helper that happens to share the prefix. Skip it. - # Hard-exiting on it would be a repo-wide outage: this suite runs in the - # plugin-gate lane, so the first future `scripts/sync-something.sh` that is - # not a manifest would turn a REQUIRED check red for every PR, including - # ones that never touch this tool. Such a script opens no block, so the - # loop below registers nothing for it. Half a manifest is still fatal. + # A script matching scripts/sync-*.sh that publishes NEITHER key is a + # helper that happens to share the prefix, and opens no block. Exiting on + # it would turn a required lane red for every pull request the day such a + # helper lands. Half a manifest is still fatal. for i in "${!block_src[@]}"; do patterns=() while IFS= read -r line; do @@ -628,6 +462,80 @@ build_sync_map() { fi } +# --------------------------------------------------------------------------- +# Tree index: every file, the ambiguous basenames, the declared scopes +# --------------------------------------------------------------------------- + +# scope_table -> `\t` for every R8 declaration +# in the leading comment block of each suite the list names (paths relative to +# ). Reading stops at the first line that is neither blank nor a comment. +scope_table() { + awk -v root="$1" ' + { b = $0; sub(/.*\//, "", b) } + $0 ~ /\.(test\.(sh|js|mjs)|Tests\.ps1)$/ || b ~ /^test_.*\.py$/ { + f = root "/" $0 + while ((getline line < f) > 0) { + sub(/\r$/, "", line) + if (line ~ /^[ \t]*$/) continue + if (line !~ /^[ \t]*(#|\/\/|\/\*|\*)/) break + if (line ~ /^[ \t]*(#|\/\/)[ \t]*test-scope:/) { + sub(/^[^:]*:/, "", line) + n = split(line, g, /[ \t]+/) + for (i = 1; i <= n; i++) if (g[i] != "") print $0 "\t" g[i] + } + } + close(f) + }' "$2" +} + +declare -A AMBIGUOUS=() # basename -> 1 when two or more files carry it +declare -A SYNC_BASE=() # basename -> 1 when a shared library or its copy carries it +declare -a SCOPE_SUITES=() SCOPE_GLOBS=() +# build_tree_index: every tracked or untracked-unignored file, listed once for +# the ambiguous-name set, the reverse lookup's resolution, the declared scopes +# and the unmapped corpora. Fatal on a failed listing: a short list under-selects. +build_tree_index() { + local b src copy suite glob + if ! git ls-files --cached --others --exclude-standard >"$WORK_DIR/all-files" || + ! awk '{ sub(/.*\//, ""); if (++count[$0] == 2) print }' "$WORK_DIR/all-files" >"$WORK_DIR/ambiguous"; then + echo "error: listing the tree failed." >&2 + exit 2 + fi + while IFS= read -r b; do + AMBIGUOUS["$b"]=1 + done <"$WORK_DIR/ambiguous" + for src in "${!SYNC_SRC_COPIES[@]}"; do + SYNC_BASE["${src##*/}"]=1 + while IFS= read -r copy; do + [[ -n "$copy" ]] && SYNC_BASE["${copy##*/}"]=1 + done <<<"${SYNC_SRC_COPIES[$src]}" + done + + # Every Python import line, read once for PYTHON IMPORTS. Fatal on a git + # error for the same reason as the reverse lookup: no lines reads as no edges. + local rc=0 + git grep --untracked -I -E '^[[:space:]]*(from[[:space:]]+[.A-Za-z_][.A-Za-z0-9_]*[[:space:]]+import|import[[:space:]]+[A-Za-z_])' \ + -- '*.py' >"$WORK_DIR/py-imports" || rc=$? + if [[ "$rc" -gt 1 ]]; then + echo "error: 'git grep' failed (exit $rc) listing the Python import lines." >&2 + exit 2 + fi + + # R8. A replay supplies the table of the tree it was started from, since the + # commits it checks out predate the declarations. + if [[ -n "${AFFECTED_TESTS_SCOPES:-}" ]]; then + cp "$AFFECTED_TESTS_SCOPES" "$WORK_DIR/scopes" || exit 2 + elif ! scope_table . "$WORK_DIR/all-files" >"$WORK_DIR/scopes"; then + echo "error: reading the test-scope declarations failed." >&2 + exit 2 + fi + while IFS=$'\t' read -r suite glob; do + [[ -n "$glob" ]] || continue + SCOPE_SUITES+=("$suite") + SCOPE_GLOBS+=("$glob") + done <"$WORK_DIR/scopes" +} + # --------------------------------------------------------------------------- # Selection # --------------------------------------------------------------------------- @@ -636,18 +544,9 @@ declare -A SUITES=() # suite path -> reason declare -a UNMAPPED=() # changed paths that mapped to nothing declare -a DELETED=() # changed paths that mapped to nothing AND no longer exist -is_structural() { - local b="$1" s - for s in $STRUCTURAL_BASENAMES; do - [[ "$b" == "$s" ]] && return 0 - done - return 1 -} - # SEED_HITS counts the suites the CURRENT seed reached, whether or not an -# earlier seed had already selected them. Counting only NEWLY added suites was -# wrong and wrong in the dangerous direction: the second of two changed files -# that share a suite looked like it mapped to nothing and was reported unmapped. +# earlier seed had already selected them: counting only new ones reported the +# second of two files sharing a suite as unmapped. SEED_HITS=0 add_suite() { local suite="$1" reason="$2" @@ -659,11 +558,9 @@ add_suite() { return 0 } -# is_suite_path -> 0 when the path IS a test suite, in any ecosystem this -# repo actually carries. The conventions are READ from the corpus, not invented: -# shell and Node co-locate .test., Pester suffixes .Tests.ps1, -# and Python PREFIXES test_.py — which is why the Python arm has to test -# the basename rather than the whole path. +# is_suite_path -> 0 when the path IS a test suite, in any ecosystem +# this repo carries. Python PREFIXES test_.py, so its arm tests the +# basename rather than the whole path. is_suite_path() { case "$1" in *.test.sh | *.test.js | *.test.mjs | *.Tests.ps1) return 0 ;; @@ -676,13 +573,10 @@ is_suite_path() { return 1 } -# lang_family -> sets LANG_FAMILY to the ecosystem the path belongs to, -# for the one-hop rule above. A global rather than stdout for the same reason -# SEED_HITS is, plus one more: a command substitution forks a subshell, and this -# runs once per frontier path and once per reverse-lookup hit — a per-spawn cost -# the header's Windows note is about. Anything unrecognized gets its own bucket -# rather than a shared "other": two unrelated extensions must not read as the -# same language and license a walk between them. +# lang_family -> sets LANG_FAMILY to the ecosystem the path belongs to. +# A global rather than stdout: a command substitution forks, and this runs once +# per frontier path and per lookup hit. Anything unrecognized gets its own +# `ext:` bucket rather than a shared "other", and an `ext:` origin is data (R4). LANG_FAMILY="" lang_family() { case "$1" in @@ -694,187 +588,224 @@ lang_family() { esac } -# token_hits -# Reduce `git grep`'s SUBSTRING hits to the TOKEN hits R3/R4 actually mean: keep -# a (file, basename) pair only where that basename stands in the matched line as -# a whole path token, and, for a basename only skill directories carry, only -# where SKILL OWNERSHIP lets the line mean that file. See MATCHING in the header -# for both rules and for why they live here instead of in the grep pattern. -# -# The frontier file lists the paths whose basenames this level asked about, so a -# basename traces back to the skill directories that own it; the skill-files -# list says which skills carry a file of that name of their own. The grep input -# is `:`; the output is `:`, deduplicated, which is the -# shape the caller already parses. Splitting the LINE rather than the path keeps -# a basename that appears only in the path prefix from counting as a mention of -# itself. +# token_hits +# Reduce `git grep`'s SUBSTRING hits to the mentions the rules mean. Inputs: +# the plain basenames this level asked about, the frontier paths whose names +# must RESOLVE (AMBIGUOUS NAMES), and the `:` grep output. Output, +# one line per pair: +# p<1 when a kept line runs or loads it, else 0> +# r token_hits() { - awk -v pat="$1" -v front="$2" -v skills="$3" ' - function skill_dir(p) { - if (match(p, "^plugins/[^/]+/skills/[^/]+/")) return substr(p, 1, RLENGTH) - return "" - } - # First file: the basenames this level asked about. A name that is itself a - # path token gets the exact test; anything else cannot be tokenized at all, - # so it keeps the old substring test rather than losing coverage silently. - FILENAME == pat { + awk -v plainf="$1" -v resf="$2" -v allf="$WORK_DIR/all-files" ' + function dir_of(p) { sub(/[^\/]*$/, "", p); return p } + function base_of(p) { sub(/.*\//, "", p); return p } + function ends(s, t) { return length(s) >= length(t) && substr(s, length(s) - length(t) + 1) == t } + # Plain basenames: a name that is itself a path token gets the exact test; + # anything else keeps the substring test rather than losing coverage. + FILENAME == plainf { if ($0 == "") next if ($0 ~ /^[A-Za-z0-9_.-]+$/) want[$0] = 1 else loose[$0] = 1 next } - # Second file: the frontier paths. A basename any of them carries outside a - # skill directory keeps the plain basename rule; the rest record the skill - # directories and skill names that own them. - FILENAME == front { + FILENAME == resf { if ($0 == "") next - n = split($0, c, "/") - d = skill_dir($0) - if (d == "") free[c[n]] = 1 - else { - k = ++nowner[c[n]] - odir[c[n], k] = d - oskill[c[n], k] = c[4] - } + b = base_of($0) + rt[b, ++nrt[b]] = $0 next } - # Third file: every file inside a skill directory, as . - FILENAME == skills { - carries[$0] = 1 + FILENAME == allf { + b = base_of($0) + if (b in nrt) same[b, ++nsame[b]] = $0 next } - # mine_tok: does this path token (leading slash added) name the file through - # an owning skill? A token that spells the plugin must match the owning - # plugin and skill both; one that spells only the skill matches by name. - function mine_tok(q, name, k, full) { - full = match(q, "/plugins/[^/]+/skills/[^/]+/") ? substr(q, RSTART, RLENGTH) : "" - for (k = 1; k <= nowner[name]; k++) { - if (full != "") { if (full == "/" odir[name, k]) return 1 } - else if (index(q, "/" oskill[name, k] "/")) return 1 + # uniq_suffix: the shortest path suffix, two components or more, that no + # other file of the same basename ends in; empty when there is none. + function uniq_suffix(t, n, pa, k, j, suf, b, i, o, clash) { + n = split(t, pa, "/") + b = pa[n] + for (k = 2; k <= n; k++) { + suf = pa[n - k + 1] + for (j = n - k + 2; j <= n; j++) suf = suf "/" pa[j] + clash = 0 + for (i = 1; i <= nsame[b]; i++) { + o = same[b, i] + if (o != t && (o == suf || ends(o, "/" suf))) { clash = 1; break } + } + if (!clash) return suf } - return 0 + return "" } - # owned: may this line in this file stand for a frontier file of that name? - function owned(path, name, text, hd, k, n, j, pt, q, named, plain, mine, own) { - if ((name in free) || !(name in nowner)) return 1 - hd = skill_dir(path) - own = 0 - for (k = 1; k <= nowner[name]; k++) - if (hd == odir[name, k]) own = 1 - named = 0 - plain = 0 - mine = 0 - n = split(text, ptok, "[^A-Za-z0-9_./-]+") - for (j = 1; j <= n; j++) { - pt = ptok[j] - sub(/\.+$/, "", pt) - sub(/^[-+=?.]+/, "", pt) - if (pt != name && substr(pt, length(pt) - length(name)) != "/" name) continue - named = 1 - q = "/" pt - if (mine_tok(q, name)) mine = 1 - else if (!index(q, "/skills/")) plain = 1 + # resolves: does path token pt, written in file namer, mean target t? Any + # mention from the directory of t does; elsewhere a bare name never does, + # and a path does when it ends in the shortest unique suffix of t, or in the + # path of t relative to a directory holding both files ($SCRIPT_DIR/lib/x.sh, + # $PLUGIN_DIR/skills//SKILL.md). + function resolves(namer, pt, t, u, a) { + a = dir_of(namer) + if (a == dir_of(t)) return 1 + if (!index(pt, "/")) return 0 + if (!(t in usuf)) usuf[t] = uniq_suffix(t) + u = usuf[t] + if (u != "" && (pt == u || ends(pt, "/" u))) return 1 + while (1) { + if ((a == "" || index(t, a) == 1) && (pt == substr(t, length(a) + 1) || ends(pt, "/" substr(t, length(a) + 1)))) return 1 + if (a == "") return 0 + sub(/[^\/]*\/$/, "", a) } - if (mine) return 1 - # A mention this split cannot place counts as plain: over-select. Inside an - # owning skill, a plain mention is that skill citing its own file. - if (own) return plain || !named - return (plain || !named) && !((hd name) in carries) } - function emit(path, name) { - if ((path SUBSEP name) in seen) return - seen[path SUBSEP name] = 1 - print path ":" name + # runs_or_loads: R4. An interpreter or process API on the line, a path to + # the file, or a shell script as the named file. + function runs_or_loads(name, n, j) { + if (exec_line || name ~ /\.(sh|bash)$/) return 1 + for (j = 1; j <= n; j++) if (ends(ptok[j], "/" name)) return 1 + return 0 } - # A rejected line does not mark the pair seen: a later line may qualify it. - function keep(path, name, text) { - if ((path SUBSEP name) in seen) return - if (owned(path, name, text)) emit(path, name) + function keep(path, name, n) { + key = path SUBSEP name + if (!(key in kept)) { kept[key] = 0; order[++nkept] = key } + if (!kept[key] && runs_or_loads(name, n)) kept[key] = 1 } - # comment_only: is this a whole-line comment in a NON-suite file? Such a line - # makes no R4 dependent; see COMMENTS in the header. Suites keep every line - # (R3), and a shellcheck source directive or a JSDoc type import declares a - # real edge. The suite test mirrors is_suite_path. - function comment_only(path, text, b) { - b = path - sub(/.*\//, "", b) - if (path ~ /\.(test\.(sh|js|mjs)|Tests\.ps1)$/ || b ~ /^test_.*\.py$/) return 0 + # comment_only: a whole-line comment names nothing (COMMENTS in the header), + # except a shellcheck source directive and a JSDoc type import. + function comment_only(path, text) { if (path ~ /\.(js|mjs|cjs)$/) return text ~ /^[ \t]*(\/\/|\/\*|\*([ \t\/]|$))/ && text !~ /@import|import\(/ if (text ~ /^[ \t]*#[ \t]*shellcheck[ \t]+source=/) return 0 - if (text !~ /^[ \t]*#/) return 0 - return !(path ~ /\.py$/ && py_imports_named(path, text)) - } - # py_imports_named: does this .py import, by module name, a .py that - # this comment line names? A Python import never spells the .py, so such a - # comment is the only text edge to the module and has to keep counting. - function py_imports_named(path, text, n, j, t, stem, line, found) { - n = split(text, ptk, /[^A-Za-z0-9_.-]+/) - for (j = 1; j <= n; j++) { - t = ptk[j] - sub(/\.+$/, "", t) - if (t !~ /^[A-Za-z_][A-Za-z0-9_]*\.py$/) continue - stem = substr(t, 1, length(t) - 3) - if (!((path SUBSEP stem) in pyimp)) { - found = 0 - while ((getline line < path) > 0) - if (line ~ ("^[ \t]*(from[ \t]+\\.*" stem "[ \t]+import|import[ \t]+([A-Za-z0-9_.]+[ \t]*,[ \t]*)*" stem "([ \t,]|$))")) { - found = 1 - break - } - close(path) - pyimp[path, stem] = found - } - if (pyimp[path, stem]) return 1 - } - return 0 + return text ~ /^[ \t]*#/ } { i = index($0, ":") - # No separator means no path: git grep says "Binary file X matches" that - # way, and a hit with no readable path must not become a frontier entry. + # No separator means no path: git grep says "Binary file X matches" that way. if (i == 0) next path = substr($0, 1, i - 1) text = substr($0, i + 1) if (comment_only(path, text)) next + exec_line = text ~ /(^|[^A-Za-z0-9_-])(bash|sh|zsh|python3?|node|deno|pwsh|powershell|uv|npx|source|subprocess|Popen|check_output|check_call|spawn[A-Za-z0-9_]*|exec[A-Za-z0-9_]*|execa|child_process|Start-Process|Invoke-Expression)([^A-Za-z0-9_-]|$)/ + # Path tokens. A leading `.` stays: `./x`, `../x` and `.claude-plugin/x` + # are paths, not punctuation. + np = split(text, ptok, /[^A-Za-z0-9_.\/-]+/) + for (j = 1; j <= np; j++) { + sub(/\.+$/, "", ptok[j]) + sub(/^[-+=?]+/, "", ptok[j]) + b = base_of(ptok[j]) + if (!(b in nrt)) continue + for (k = 1; k <= nrt[b]; k++) + if (rt[b, k] != path && resolves(path, ptok[j], rt[b, k])) { + key = path SUBSEP rt[b, k] + if (!(key in rhit)) { rhit[key] = 1; print "r\t" path "\t" rt[b, k] } + } + } n = split(text, tok, /[^A-Za-z0-9_.-]+/) for (j = 1; j <= n; j++) { t = tok[j] if (t == "") continue - if (t in want) { keep(path, t, text); continue } - # A trailing dot run is sentence punctuation (a comment ending "... in - # that suite."), and a leading one is an ellipsis butted against the - # name. Neither is part of a filename. The leading arm is the narrow - # one: a relative path needs no stripping, because the `/` in `./x` - # already delimits the token, so it fires only on a literal `...x`. + if (t in want) { keep(path, t, np); continue } sub(/\.+$/, "", t) - if (t in want) { keep(path, t, text); continue } - # The leading strip also drops a run of parameter-expansion operator - # characters, so `${V:-}` names . Only `-` is load-bearing: - # it is the one operator character inside the token class, so it glues - # onto the name instead of ending the token. See MATCHING in the header. + if (t in want) { keep(path, t, np); continue } sub(/^[-+=?.]+/, "", t) - if (t in want) keep(path, t, text) + if (t in want) keep(path, t, np) } for (name in loose) - if (index(text, name) > 0) emit(path, name) + if (index(text, name) > 0) { + key = path SUBSEP name + if (!(key in kept)) order[++nkept] = key + kept[key] = 1 + } + } + END { + for (k = 1; k <= nkept; k++) { + split(order[k], kv, SUBSEP) + print "p\t" kv[1] "\t" kv[2] "\t" kept[order[k]] + } + } + ' "$1" "$2" "$WORK_DIR/all-files" "$3" >"$4" +} + +# py_hits -> PYTHON IMPORTS: append one +# r +# line for every .py whose import line imports a frontier module, read from the +# import lines build_tree_index listed. +py_hits() { + awk -v front="$1" -v allf="$WORK_DIR/all-files" ' + function dir_of(p) { sub(/[^\/]*$/, "", p); return p } + function root_of(p, c) { split(p, c, "/"); return c[1] == "plugins" ? "plugins/" c[2] "/" : c[1] "/" } + function ends(s, t) { return length(s) >= length(t) && substr(s, length(s) - length(t) + 1) == t } + # The module a file is: foo for foo.py, pkg for pkg/__init__.py. + function modname(p) { + if (p ~ /(^|\/)__init__\.py$/) { p = dir_of(p); sub(/\/$/, "", p) } + sub(/.*\//, "", p) + sub(/\.py$/, "", p) + return p + } + # imports: does dotted name D, imported in P, mean module file t? From the + # directory t is imported from, or below it (a tests/ directory); by a + # dotted path that spells t; or anywhere in the same plugin when t is the + # only module of that name there. + function imports(P, D, t, m, home, pd, path) { + home = dir_of(t) + if (t ~ /(^|\/)__init__\.py$/) { sub(/\/$/, "", home); home = dir_of(home) } + pd = dir_of(P) + if (pd == home || (home != "" && index(pd, home) == 1)) return 1 + if (index(D, ".")) { + path = D + gsub(/\./, "/", path) + if (ends("/" t, "/" path ".py") || ends("/" t, "/" path "/__init__.py")) return 1 + } + return root_of(P) == root_of(t) && cnt[root_of(t), m] == 1 + } + function cand(P, D, m, k, t) { + m = D + sub(/.*\./, "", m) + for (k = 1; k <= nt[m]; k++) { + t = tg[m, k] + if (t != P && imports(P, D, t, m) && !((P SUBSEP t) in seen)) { + seen[P, t] = 1 + print "r\t" P "\t" t + } + } } - ' "$1" "$2" "$3" "$4" >"$5" + FILENAME == front { if ($0 != "") { m = modname($0); tg[m, ++nt[m]] = $0 } next } + FILENAME == allf { if ($0 ~ /\.py$/) cnt[root_of($0), modname($0)]++; next } + { + i = index($0, ":") + if (i == 0) next + P = substr($0, 1, i - 1) + s = substr($0, i + 1) + sub(/#.*/, "", s) + gsub(/[()\\]/, " ", s) + if (s ~ /^[ \t]*from[ \t]/) { + sub(/^[ \t]*from[ \t]+/, "", s) + base = s + sub(/[ \t].*/, "", base) + sub(/^[^ \t]+[ \t]+import[ \t]+/, "", s) + sub(/^\.+/, "", base) + if (base != "") cand(P, base) + } else { + sub(/^[ \t]*import[ \t]+/, "", s) + base = "" + } + n = split(s, ys, /,/) + for (k = 1; k <= n; k++) { + y = ys[k] + sub(/^[ \t]+/, "", y) + sub(/[ \t].*/, "", y) + if (y !~ /^[A-Za-z_][A-Za-z0-9_.]*$/) continue + cand(P, base == "" ? y : base "." y) + } + } + ' "$1" "$WORK_DIR/all-files" "$WORK_DIR/py-imports" >>"$2" } # colocated_suites -> every sibling suite covering it, one per line. -# PLURAL on purpose: one source file can carry suites in two ecosystems at once -# — a .py with both a co-located test_.py and a wrapping .test.sh -# that drives it — and returning only the first is an under-selection, the one -# direction this tool treats as unsafe. +# PLURAL on purpose: a .py can carry a co-located test_.py and a +# wrapping .test.sh at once, and returning one under-selects. colocated_suites() { local p="$1" stem dir base candidate if is_suite_path "$p"; then printf '%s\n' "$p" return 0 fi - # An extensionless path keeps its whole name as the stem, which is what - # ${p%.*} already yields when there is no dot to strip. stem="${p%.*}" dir="${p%/*}" [[ "$dir" == "$p" ]] && dir="." @@ -885,15 +816,10 @@ colocated_suites() { "$stem.test.mjs" "$stem.Tests.ps1" "$dir/test_$base.py" - # The one suite this repo puts in a SUBDIRECTORY instead of beside its - # subject. It has to be a path rule: the suites in question reach their - # subject with `import `, which never spells the filename, so the - # reference rule has no text to match and the file looks uncovered. + # A Python suite reaches its subject with `import `, which never + # spells the filename, so the tests/ form has to be a path rule. "$dir/tests/test_$base.py" ) - # The hyphen fold is a SECOND candidate only when there is a hyphen to fold; - # otherwise it is character-for-character the previous entry and would emit - # the same path twice. [[ "$base" == *-* ]] && candidates+=( "$dir/test_${base//-/_}.py" "$dir/tests/test_${base//-/_}.py" @@ -905,29 +831,19 @@ colocated_suites() { } # select_for -> populate SUITES, and set SEED_HITS to the number -# of suites THIS seed reached. The count comes back through a global rather than -# stdout on purpose: a command substitution would run the whole walk in a -# SUBSHELL and every SUITES mutation would be discarded with it. +# of suites THIS seed reached. The count comes back through a global: a command +# substitution would run the walk in a subshell and lose every SUITES entry. select_for() { local seed="$1" - local -a frontier=() - local -a next=() - local p b sib copy line matched_path matched_name grep_rc - local origin_family matched_family hop_state - # PATTERN_ORIGIN maps a batched basename back to the ecosystem of the file that - # contributed it, and PATTERN_CROSSED records whether that file had already - # spent its one language transition. When two files in DIFFERENT families - # contribute the same basename, the origin collapses to '*' and the pattern is - # treated as same-family against anything — the over-selecting direction, which - # is the safe one. - local -A PATTERN_ORIGIN=() - local -A PATTERN_CROSSED=() - local -A CROSSED=() - # The visited set is PER SEED, not shared across seeds. Sharing it made a - # changed file that an earlier seed had already walked past look unvisited- - # and-unmapped, which is the fail-open direction: the file would be reported - # as covered because someone else's walk touched it, or as unmapped because - # its own walk was short-circuited. Re-walking costs one batched grep. + local -a frontier=() next=() + local p b sib copy line kind hit_path hit_name hit_x grep_rc + local origin_family hit_family hop_state + # PATTERN_ORIGIN maps a plain basename back to the ecosystem of the file that + # contributed it ('*' when two families contributed it, which over-selects), + # and PATTERN_CROSSED records whether that file had spent its transition. + local -A PATTERN_ORIGIN=() PATTERN_CROSSED=() CROSSED=() + # PER SEED, not shared: a shared visited set made a changed file another + # seed had walked past look unmapped. local -A VISITED=() SEED_HITS=0 @@ -951,11 +867,14 @@ select_for() { while [[ ${#frontier[@]} -gt 0 ]]; do : >"$WORK_DIR/patterns" - : >"$WORK_DIR/frontier" + : >"$WORK_DIR/plain" + : >"$WORK_DIR/resolve" + : >"$WORK_DIR/pyfront" next=() for p in "${frontier[@]}"; do [[ -n "${VISITED[$p]:-}" ]] && continue VISITED["$p"]=1 + [[ "$p" == *.py ]] && printf '%s\n' "$p" >>"$WORK_DIR/pyfront" # R1/R2 while IFS= read -r sib; do [[ -n "$sib" ]] || continue @@ -966,8 +885,7 @@ select_for() { fi done < <(colocated_suites "$p") # R7. The suite path is joined from two pieces so this file never carries - # its basename as one token: that would make this file an R4 dependent of - # the suite and fan every edit to it out to whatever names this file. + # its basename as one token, which would make this file name the suite. case "$p" in plugins/autonomy/reference/*) add_suite "scripts/validate-plugin-contracts"".test.sh" \ @@ -976,7 +894,12 @@ select_for() { *) ;; esac b="${p##*/}" - is_structural "$b" && continue + printf '%s\n' "$b" >>"$WORK_DIR/patterns" + if [[ "$STRUCTURAL_BASENAMES" == *" $b "* ]] || + [[ -n "${AMBIGUOUS[$b]:-}" && -z "${SYNC_BASE[$b]:-}" ]]; then + printf '%s\n' "$p" >>"$WORK_DIR/resolve" + continue + fi lang_family "$p" origin_family="$LANG_FAMILY" if [[ -z "${PATTERN_ORIGIN[$b]:-}" ]]; then @@ -984,99 +907,128 @@ select_for() { PATTERN_CROSSED["$b"]="${CROSSED[$p]:-0}" else [[ "${PATTERN_ORIGIN[$b]}" == "$origin_family" ]] || PATTERN_ORIGIN["$b"]='*' - # Any contributor that has NOT yet crossed wins: the walk keeps its - # budget. Over-selection is the safe direction, so a tie resolves toward - # more walking. + # Any contributor that has NOT yet crossed wins: over-select. [[ "${CROSSED[$p]:-0}" == "0" ]] && PATTERN_CROSSED["$b"]=0 fi - printf '%s\n' "$b" >>"$WORK_DIR/patterns" - printf '%s\n' "$p" >>"$WORK_DIR/frontier" + printf '%s\n' "$b" >>"$WORK_DIR/plain" done [[ -s "$WORK_DIR/patterns" ]] || break - # R3/R4: one batched reverse lookup per level, in the two stages MATCHING - # describes — git grep finds the candidate lines, token_hits keeps only the - # ones that NAME a basename rather than merely containing it. The hits go - # through a FILE, not a process substitution, so this lookup can FAIL LOUD - # like every other step here. `done < <(git grep ... 2>/dev/null || true)` could not: after - # the loop `$?` holds the LOOP's status, git's stderr was discarded, and - # `|| true` erased the code, so a git ERROR (exit >= 2 — a bad flag, an - # unreadable pattern file, a corrupt index) was indistinguishable from NO - # MATCH (exit 1). Both produced an empty read and the walk simply found no - # dependents. That is the fail-open this tool exists to refuse, and it - # lands in the UNDER-selection direction the DIRECTION note above calls - # unsafe: a broken lookup reported a narrow selection, or "no suites - # selected", at exit 0. Same reasoning as changed_from_diff below. + # One batched reverse lookup per level, in the two stages the header + # describes. The hits go through a FILE rather than a process substitution + # so a git ERROR (exit >= 2) is not read as NO MATCH (exit 1): both would + # come back as "no dependents", an under-selection at exit 0. Every + # extension lang_family() names is searched, or a family it classifies + # would be under-covered while looking supported. grep_rc=0 - # Every extension lang_family() names must appear here. If it classifies a - # path into a family but nothing ever greps that path's content, the file is - # silently under-covered while LOOKING supported — fail-open, and the reason - # .bash and .cjs are in this list despite the repo carrying none today. git grep --untracked -F -f "$WORK_DIR/patterns" \ -- '*.sh' '*.bash' '*.js' '*.mjs' '*.cjs' '*.py' '*.ps1' '*.psm1' \ >"$WORK_DIR/matched-lines" || grep_rc=$? - # Exit 1 is "no match" and is completely ordinary — most seeds reach a level - # with no further dependents. Only above 1 is git itself failing. if [[ "$grep_rc" -gt 1 ]]; then echo "error: 'git grep' failed (exit $grep_rc) resolving dependents of the current level." >&2 echo " Refusing to continue: an unreadable reverse lookup silently UNDER-selects, and" >&2 echo " under-selection is reported as success by everything downstream." >&2 exit 2 fi - # The boundary half of the lookup, and fatal for the same reason: a filter - # that dies mid-stream hands the walk a TRUNCATED hit set, which is an - # under-selection wearing a successful exit code. - if ! token_hits "$WORK_DIR/patterns" "$WORK_DIR/frontier" "$WORK_DIR/skill-files" \ - "$WORK_DIR/matched-lines" "$WORK_DIR/hits"; then + # Fatal for the same reason: a filter that dies mid-stream hands the walk a + # TRUNCATED hit set. + if ! token_hits "$WORK_DIR/plain" "$WORK_DIR/resolve" "$WORK_DIR/matched-lines" "$WORK_DIR/hits" || + { [[ -s "$WORK_DIR/pyfront" ]] && ! py_hits "$WORK_DIR/pyfront" "$WORK_DIR/hits"; }; then echo "error: the token filter over the reverse lookup failed on the current level." >&2 echo " Refusing to continue: a partial filter silently UNDER-selects, and" >&2 echo " under-selection is reported as success by everything downstream." >&2 exit 2 fi - while IFS= read -r line; do - matched_path="${line%:*}" - matched_name="${line##*:}" - [[ -n "$matched_path" ]] || continue - if is_suite_path "$matched_path"; then - # A suite is taken whatever language it is written in: a .test.sh that - # drives a .py helper is exactly the cross-language coverage this must - # keep finding. - add_suite "$matched_path" "references $matched_name" || true + while IFS=$'\t' read -r kind hit_path hit_name hit_x; do + [[ -n "$hit_path" ]] || continue + if [[ "$kind" == r ]]; then + # A mention that resolves to an ambiguous or structural file is a real + # reference to it, from any language, and spends no transition. + if is_suite_path "$hit_path"; then + add_suite "$hit_path" "references ${hit_name##*/} (resolves to $hit_name)" || true + elif [[ -z "${VISITED[$hit_path]:-}" ]]; then + CROSSED["$hit_path"]=0 + next+=("$hit_path") + fi continue fi - lang_family "$matched_path" - matched_family="$LANG_FAMILY" - origin_family="${PATTERN_ORIGIN[$matched_name]:-*}" - if [[ "$origin_family" == '*' || "$matched_family" == "$origin_family" ]]; then - # Same-family edge: a real dependency, and it spends no budget. - hop_state="${PATTERN_CROSSED[$matched_name]:-0}" - elif [[ "${PATTERN_CROSSED[$matched_name]:-0}" == "1" ]]; then - # This would be a SECOND language transition for the chain reaching here. + lang_family "$hit_path" + hit_family="$LANG_FAMILY" + origin_family="${PATTERN_ORIGIN[$hit_name]:-*}" + if [[ "$origin_family" == '*' || "$hit_family" == "$origin_family" ]]; then + # R3: same language, spends no transition. + hop_state="${PATTERN_CROSSED[$hit_name]:-0}" + elif [[ "$origin_family" == ext:* ]]; then + # R4: data reaches code of any language, and spends no transition. + hop_state=0 + elif [[ "$hit_x" != 1 ]]; then + # R4: another language that neither runs nor loads the file. + continue + elif ! is_suite_path "$hit_path" && [[ "${PATTERN_CROSSED[$hit_name]:-0}" == 1 ]]; then + # R4: a second transition. A suite that runs or loads the file is still + # taken: the budget bounds the walk, not the suites it ends in. continue else hop_state=1 fi - [[ -n "${VISITED[$matched_path]:-}" ]] && continue - # AGGREGATE, never assign. One path can be hit several times in a single - # round by different patterns — a shell wrapper naming both a shell helper - # and a JS file — and those hits can disagree about whether the chain - # reaching it has already crossed. A plain assignment let whichever hit - # `git grep` happened to emit last decide, so an unrelated cross-family - # mention could spend a path's budget and cut its own same-family walk - # short: order-dependent UNDER-selection, the direction this file calls - # unsafe. The path keeps the most permissive state any contributor gives - # it, matching how PATTERN_CROSSED already resolves its own ties. - if [[ "${CROSSED[$matched_path]:-1}" != "0" ]]; then - CROSSED["$matched_path"]="$hop_state" + if is_suite_path "$hit_path"; then + add_suite "$hit_path" "references $hit_name" || true + continue + fi + [[ -n "${VISITED[$hit_path]:-}" ]] && continue + # AGGREGATE, never assign: one path can be hit by several patterns in one + # round that disagree about whether its chain has crossed, and the most + # permissive state wins, whatever order git grep printed them in. + if [[ "${CROSSED[$hit_path]:-1}" != "0" ]]; then + CROSSED["$hit_path"]="$hop_state" fi - next+=("$matched_path") + next+=("$hit_path") done <"$WORK_DIR/hits" frontier=(${next[@]+"${next[@]}"}) done } +# check_declarations -> exit 2 when a test-scope line of the +# suite sits below its leading comment block, where R8 never reads it, or when +# one of its globs matches no file of the tree. Either mistake silently drops +# the suite from the changes it reads, so the pull request that makes it fails. +check_declarations() { + local suite="$1" i f hit + if ! awk 'BEGIN { hdr = 1 } { sub(/\r$/, "") } + hdr && !/^[ \t]*$/ && !/^[ \t]*(#|\/\/|\/\*|\*)/ { hdr = 0 } + !hdr && /^[ \t]*(#|\/\/)[ \t]*test-scope:/ { printf "%s:%d\n", FILENAME, FNR; bad = 1 } + END { exit bad }' "$suite" >"$WORK_DIR/misplaced"; then + echo "error: a test-scope declaration below the leading comment block is never read:" >&2 + sed 's/^/ - /' "$WORK_DIR/misplaced" >&2 + exit 2 + fi + for i in "${!SCOPE_GLOBS[@]}"; do + [[ "${SCOPE_SUITES[i]}" == "$suite" ]] || continue + hit=0 + while IFS= read -r f; do + # shellcheck disable=SC2053 # the right-hand side is a glob pattern by design. + [[ "$f" == ${SCOPE_GLOBS[i]} ]] && hit=1 && break + done <"$WORK_DIR/all-files" + if [[ "$hit" -eq 0 ]]; then + echo "error: $suite declares test-scope ${SCOPE_GLOBS[i]}, which matches no file of the tree." >&2 + exit 2 + fi + done +} + +# select_scoped -> R8: add every suite whose declared test-scope +# matches the path, counting each as a hit of the current seed. +select_scoped() { + local p="$1" i + for i in "${!SCOPE_GLOBS[@]}"; do + # shellcheck disable=SC2053 # the right-hand side is a glob pattern by design. + [[ "$p" == ${SCOPE_GLOBS[i]} ]] || continue + add_suite "${SCOPE_SUITES[i]}" "test-scope ${SCOPE_GLOBS[i]}" || true + done +} + # --------------------------------------------------------------------------- # No-suite classification # --------------------------------------------------------------------------- @@ -1113,11 +1065,9 @@ is_no_suite() { changed_from_diff() { local base="" mb - # An explicit --base is resolved HERE, not through the shared ladder: a ref - # the user typed and got wrong must be an error naming that ref, never a - # silent fall-through to origin/main that would report suites for a diff they - # did not ask for. The fallback ladder (origin/main, origin/master, main, - # master) is the part shared with the checker gates. + # An explicit --base must resolve or be an error naming it, never a silent + # fall-through to origin/main that reports suites for a diff nobody asked + # for. The fallback ladder is shared with the checker gates. if [[ -n "$base_ref" ]]; then if ! changed_files::verify_base "$base_ref"; then echo "error: base ref '$base_ref' does not resolve to a commit." >&2 @@ -1129,16 +1079,10 @@ changed_from_diff() { echo " Pass --base , or give explicit paths." >&2 exit 2 fi - # Compare the WORKING TREE against the merge base: a local developer wants the - # suites covering the work in front of them, including uncommitted edits, and - # not the suites for whatever else landed on the base branch meanwhile. - # - # Mid-merge, HEAD is still the pre-merge commit while the working tree already - # holds the incoming side, so merge-base(base, HEAD) would charge this change - # with every file the incoming side brought. Passing the MERGE_HEAD commits as - # further arguments makes git compute the base against a hypothetical merge of - # HEAD and them: the same base the selector reports once the merge commit - # lands. --git-path resolves the file in a linked worktree too. + # The WORKING TREE against the merge base: uncommitted edits count, and so + # does nothing that landed on the base branch meanwhile. Mid-merge, the + # MERGE_HEAD commits join the merge-base computation, so the incoming side's + # files are not charged to this change. local merge_head_file="" local -a merge_heads=() merge_head_file="$(git rev-parse --git-path MERGE_HEAD 2>/dev/null)" || merge_head_file="" @@ -1146,12 +1090,9 @@ changed_from_diff() { mapfile -t merge_heads <"$merge_head_file" fi mb="$(git merge-base "$base" HEAD "${merge_heads[@]}" 2>/dev/null)" || mb="$base" - # Every failure here is fatal, never an empty list. An empty change set is - # indistinguishable from "the diff blew up" downstream, and downstream reports - # it as "nothing to select, exit 0" — the fail-open this whole tool exists to - # refuse. This function must therefore run in the CURRENT shell (see the call - # site): from inside a process substitution these exits would kill only the - # subshell and `mapfile` would happily succeed with nothing. + # Every failure is fatal, never an empty list, which downstream reads as + # "nothing to select, exit 0". This runs in the CURRENT shell (see the call + # site) so these exits are the script's. if ! git diff --name-only "$mb" --; then echo "error: 'git diff --name-only $mb' failed; refusing to report an empty change set." >&2 exit 2 @@ -1177,15 +1118,159 @@ if [[ -n "$print_fanout" ]]; then exit 0 fi +# --------------------------------------------------------------------------- +# Replay +# --------------------------------------------------------------------------- + +# replay_select +# [...] -- ... +# One selection in the replay tree, through a fresh process. Writes +# .sel (`\t`) and .unmapped, and fails +# loud on any exit but 0, because a broken selection counted as an empty one +# would understate the side it ran for. An empty lets the +# selector read the tree's own declarations. +replay_select() { + local out="$1" tree="$2" sel="$3" list="$4" scopes="$5" rc=0 + shift 5 + AFFECTED_TESTS_ROOT="$tree" AFFECTED_TESTS_NO_SUITE="$list" AFFECTED_TESTS_SCOPES="$scopes" \ + bash "$sel" --explain --allow-unmapped "$@" >/dev/null 2>"$out.err" || rc=$? + if [[ "$rc" -ne 0 ]]; then + echo "error: the selector $sel failed (exit $rc) on a replayed commit:" >&2 + cat "$out.err" >&2 + exit 2 + fi + awk '/^select: / { sub(/^select: /, ""); i = index($0, " ("); print substr($0, 1, i - 1) "\t" substr($0, i + 3, length($0) - i - 3) }' \ + "$out.err" | sort >"$out.sel" + awk '/^UNMAPPED:/ { f = 1; next } f && /^ - / { sub(/^ - /, ""); print; next } { f = 0 }' "$out.err" >"$out.unmapped" +} + +run_replay() { + local tree="$WORK_DIR/replay-tree" against="$WORK_DIR/against" against_scopes="" + local c subject n_new n_old u_new u_old + local -a commits=() changed=() against_flags=() + if ! git rev-list --first-parent --reverse "$replay_range" >"$WORK_DIR/commits"; then + echo "error: '$replay_range' is not a revision range git can list." >&2 + exit 2 + fi + mapfile -t commits <"$WORK_DIR/commits" + if [[ ${#commits[@]} -eq 0 ]]; then + echo "error: '$replay_range' holds no commits to replay." >&2 + exit 2 + fi + # This tree's rules travel with the replay: its no-suite list and its + # test-scope table, read once here. + build_sync_map + build_tree_index + cp "$NO_SUITE_LIST" "$WORK_DIR/replay-no-suite" || exit 2 + cp "$WORK_DIR/scopes" "$WORK_DIR/replay-scopes" || exit 2 + if ! git clone -q --shared --no-checkout . "$tree"; then + echo "error: could not make the scratch clone for the replay." >&2 + exit 2 + fi + if [[ -n "$against_ref" ]]; then + # 's selector, its scripts/lib/ and its lists, pointed at the replay + # tree through AFFECTED_TESTS_ROOT. A selector without that override gets + # its one `cd` line rewritten; one with neither form cannot be pointed. + mkdir -p "$against/scripts" "$against/tree" || exit 2 + if ! git show "$against_ref:scripts/affected-tests.sh" >"$against/selector.orig" || + ! git archive "$against_ref" scripts/lib | tar -x -C "$against" || + ! git show "$against_ref:scripts/affected-tests-no-suite.txt" >"$against/no-suite.txt"; then + echo "error: could not read the selector, its scripts/lib/ or its no-suite list at '$against_ref'." >&2 + exit 2 + fi + awk '$0 == "cd \"$SCRIPT_DIR/..\" || exit 2" { print "cd \"${AFFECTED_TESTS_ROOT:-$SCRIPT_DIR/..}\" || exit 2"; n++; next } + index($0, "AFFECTED_TESTS_ROOT") { n++ } { print } END { exit n ? 0 : 1 }' \ + "$against/selector.orig" >"$against/scripts/affected-tests.sh" || { + echo "error: the selector at '$against_ref' cannot be pointed at another tree." >&2 + exit 2 + } + grep -q -- '--with-always)' "$against/scripts/affected-tests.sh" && against_flags+=(--with-always) + git show "$against_ref:scripts/affected-tests-always.txt" >"$against/always.txt" 2>/dev/null || + rm -f "$against/always.txt" + # A selector that reads test-scope declarations gets 's own table. + if grep -q 'AFFECTED_TESTS_SCOPES' "$against/scripts/affected-tests.sh"; then + if ! git archive "$against_ref" | tar -x -C "$against/tree" || + ! git ls-tree -r --name-only "$against_ref" >"$against/files" || + ! scope_table "$against/tree" "$against/files" >"$against/scopes"; then + echo "error: could not read the test-scope declarations at '$against_ref'." >&2 + exit 2 + fi + against_scopes="$against/scopes" + fi + fi + + for c in "${commits[@]}"; do + git -C "$tree" -c advice.detachedHead=false checkout -q --detach "$c" || exit 2 + subject="$(git -C "$tree" log -1 --format=%s "$c")" || exit 2 + if ! git -C "$tree" diff --no-renames --name-only "$c^" "$c" >"$WORK_DIR/replay-changed"; then + echo "error: could not diff $c against its parent." >&2 + exit 2 + fi + mapfile -t changed <"$WORK_DIR/replay-changed" + [[ ${#changed[@]} -gt 0 ]] || continue + replay_select "$WORK_DIR/new" "$tree" "$SELF" "$WORK_DIR/replay-no-suite" "$WORK_DIR/replay-scopes" \ + -- "${changed[@]}" + n_new=$(grep -c . "$WORK_DIR/new.sel") + u_new=$(grep -c . "$WORK_DIR/new.unmapped") + if [[ -z "$against_ref" ]]; then + printf 'commit %s %s %s %s\n' "$c" "$n_new" "$u_new" "$subject" + sed 's/^/ unmapped /' "$WORK_DIR/new.unmapped" + awk -F '\t' '{ print " " $1 " (" $2 ")" }' "$WORK_DIR/new.sel" + printf '%s\t%s\t-\t%s\t-\n' "$c" "$n_new" "$u_new" >>"$WORK_DIR/replay-counts" + continue + fi + # The always list as has it, narrowed to the suites this commit has: + # its selector refuses an entry naming no suite. + if [[ -f "$against/always.txt" ]]; then + awk -v root="$tree" '/^#/ || NF == 0 { print; next } { if ((getline _ < (root "/" $1)) >= 0) print; close(root "/" $1) }' \ + "$against/always.txt" >"$WORK_DIR/always-now" + export AFFECTED_TESTS_ALWAYS="$WORK_DIR/always-now" + fi + replay_select "$WORK_DIR/old" "$tree" "$against/scripts/affected-tests.sh" "$against/no-suite.txt" \ + "$against_scopes" ${against_flags[@]+"${against_flags[@]}"} -- "${changed[@]}" + n_old=$(grep -c . "$WORK_DIR/old.sel") + u_old=$(grep -c . "$WORK_DIR/old.unmapped") + printf 'commit %s %s %s %s %s %s\n' "$c" "$n_new" "$n_old" "$u_new" "$u_old" "$subject" + sed 's/^/ unmapped /' "$WORK_DIR/new.unmapped" + awk -F '\t' 'FNR == 1 { side++ } side == 1 { n[$1] = $2; next } { o[$1] = $2 } + END { + for (s in n) if (!(s in o)) print " + " s " (" n[s] ")" + for (s in o) if (!(s in n)) print " - " s " (" o[s] ")" + }' "$WORK_DIR/new.sel" "$WORK_DIR/old.sel" | sort -k2 + printf '%s\t%s\t%s\t%s\t%s\n' "$c" "$n_new" "$n_old" "$u_new" "$u_old" >>"$WORK_DIR/replay-counts" + done + + [[ -s "$WORK_DIR/replay-counts" ]] || { + echo "Replayed ${#commits[@]} commit(s); none changed a file." >&2 + return 0 + } + awk -F '\t' -v against="$against_ref" ' + function pct(a, n, q, i) { i = int(q * n); if (i >= n) i = n - 1; return a[i + 1] } + function sortn(a, n, i, j, t) { for (i = 2; i <= n; i++) { t = a[i]; for (j = i - 1; j >= 1 && a[j] > t; j--) a[j + 1] = a[j]; a[j + 1] = t } } + { nn[++c] = $2; tn += $2; un += ($4 > 0); if (against != "") { no[c] = $3; to += $3; uo += ($5 > 0) } } + END { + sortn(nn, c) + printf "replay: %d commit(s); this selector: suites per commit p50 %d, p95 %d, max %d, total %d; commits with an unmapped file %d\n", c, pct(nn, c, .5), pct(nn, c, .95), nn[c], tn, un + if (against != "") { + sortn(no, c) + printf "replay: %s: suites per commit p50 %d, p95 %d, max %d, total %d; commits with an unmapped file %d\n", against, pct(no, c, .5), pct(no, c, .95), no[c], to, uo + } + }' "$WORK_DIR/replay-counts" >&2 +} + +if [[ -n "$replay_range" ]]; then + run_replay + exit 0 +fi + declare -a changed=() if [[ ${#explicit_paths[@]} -gt 0 ]]; then for p in "${explicit_paths[@]}"; do p="${p#./}" # Every rule is repo-relative, and so is every sync-manifest key. An - # absolute path that stayed absolute would miss those keys and then fall - # through to a broad no-suite pattern — reporting success with no suites, - # which is the fail-open direction. Normalize what can be normalized and - # refuse the rest out loud. + # absolute path that stayed absolute would miss them and fall through to a + # broad no-suite pattern: success with no suites. Normalize what can be + # normalized and refuse the rest out loud. case "$p" in "$PWD"/*) p="${p#"$PWD"/}" ;; /* | [A-Za-z]:[/\\]*) @@ -1200,9 +1285,8 @@ if [[ ${#explicit_paths[@]} -gt 0 ]]; then else # Run the producer in the CURRENT shell so its fatal exits are the script's. changed_from_diff >"$WORK_DIR/changed.raw" - # The sort is checked for the same reason the diff above is: reading it from a - # process substitution would let a failing `sort` yield an EMPTY change set at - # exit 0, which the very next block reports as "nothing to select". + # Checked: a failing `sort` read from a process substitution would yield an + # EMPTY change set at exit 0. if ! sort -u "$WORK_DIR/changed.raw" >"$WORK_DIR/changed"; then echo "error: sorting the changed-file list failed; refusing to report an empty change set." >&2 exit 2 @@ -1217,97 +1301,31 @@ fi build_sync_map load_no_suite_patterns -# Every file inside a skill directory, listed once for SKILL OWNERSHIP as -# `plugins//skills//`: the file set the reverse lookup -# greps, and fatal on failure for the same reason, since a missing list would -# read as "no skill carries its own copy of any name". -if ! git ls-files --cached --others --exclude-standard -- 'plugins/*/skills/*' \ - >"$WORK_DIR/skill-files.raw" || - ! awk '{ - n = split($0, c, "/") - if (match($0, "^plugins/[^/]+/skills/[^/]+/")) print substr($0, 1, RLENGTH) c[n] - }' "$WORK_DIR/skill-files.raw" >"$WORK_DIR/skill-files"; then - echo "error: listing the skill directories failed." >&2 - exit 2 -fi - -# R8: every shell suite of a plugin the change touches, once per plugin. Read -# after the seed's own walk and its mapped/unmapped verdict, so it adds suites -# without ever counting as the rule that mapped a file. Fatal on a failed -# listing, for the same reason as the reverse lookup: a short list is an -# under-selection that reports success. -declare -A R8_PLUGINS=() -select_plugin_suites() { - local p="$1" suite - case "$p" in - plugins/*/*) ;; - *) return 0 ;; - esac - p="${p#plugins/}" - p="${p%%/*}" - [[ -z "${R8_PLUGINS[$p]:-}" ]] || return 0 - R8_PLUGINS["$p"]=1 - if ! git ls-files --cached --others --exclude-standard -- "plugins/$p/*.test.sh" \ - >"$WORK_DIR/r8-suites"; then - echo "error: listing the shell suites under plugins/$p/ failed." >&2 - exit 2 - fi - while IFS= read -r suite; do - [[ -n "$suite" ]] || continue - add_suite "$suite" "R8: plugins/$p/ changed" || true - done <"$WORK_DIR/r8-suites" -} +build_tree_index declare -a NO_SUITE_FILES=() for f in "${changed[@]}"; do [[ -n "$f" ]] || continue + # A replay's table comes from another tree, so only a tree's own is checked. + if [[ -z "${AFFECTED_TESTS_SCOPES:-}" && -f "$f" ]] && is_suite_path "$f"; then + check_declarations "$f" + fi select_for "$f" + select_scoped "$f" if [[ "$SEED_HITS" -eq 0 ]]; then if is_no_suite "$f"; then NO_SUITE_FILES+=("$f") elif [[ ! -e "$f" ]]; then - # A deletion that maps to nothing needs no suite: the file has no content - # left to cover, and anything that still referenced it selects through - # its own changed path or a suite that names the dead path (both handled - # by select_for above, which runs for deletions too). Only the terminal - # would-be-UNMAPPED case lands here, reported visibly rather than as the - # loud unknown-coverage error that exists for files that DO have content. + # A deletion that maps to nothing needs no suite: it has no content left + # to cover, and anything that still referenced it selects through its own + # changed path or a suite naming the dead path. DELETED+=("$f") else UNMAPPED+=("$f") fi fi - select_plugin_suites "$f" done -if [[ "$with_always" -eq 1 ]]; then - declare -a ALWAYS_ENTRIES=() - read_list::into ALWAYS_ENTRIES "$ALWAYS_LIST" --comments leading || exit 2 - for entry in ${ALWAYS_ENTRIES[@]+"${ALWAYS_ENTRIES[@]}"}; do - entry="${entry%%[[:blank:]]*}" - if ! add_suite "$entry" "always: asserts against the live repository"; then - echo "error: $ALWAYS_LIST names '$entry', which is not a suite; remove the stale entry." >&2 - exit 2 - fi - done -fi - -# `${!SUITES[@]}` cannot carry a `+` default-guard: bash parses `${!NAME...}` as -# an indirect reference and rejects the expanded key list as a variable name. -declare -a selected=() -if [[ ${#SUITES[@]} -gt 0 ]]; then - # Checked, and read from a file, for the third time and the same reason: a - # failing `sort` here would empty a NON-EMPTY selection, and the block below - # would then print "every changed file is a recorded no-suite class" — a - # statement that is affirmatively false — and exit 0. - if ! printf '%s\n' "${!SUITES[@]}" | sort -u >"$WORK_DIR/selected"; then - echo "error: sorting the selected-suite list failed; refusing to report an empty selection" >&2 - echo " when ${#SUITES[@]} suite(s) were selected." >&2 - exit 2 - fi - mapfile -t selected <"$WORK_DIR/selected" -fi - if [[ ${#NO_SUITE_FILES[@]} -gt 0 && "$explain" -eq 1 ]]; then for f in "${NO_SUITE_FILES[@]}"; do echo "no-suite: $f (recorded in $NO_SUITE_LIST; a non-shell CI lane covers it)" >&2 @@ -1320,19 +1338,58 @@ if [[ ${#DELETED[@]} -gt 0 ]]; then done fi +corpus_used=0 if [[ ${#UNMAPPED[@]} -gt 0 ]]; then echo "UNMAPPED: ${#UNMAPPED[@]} changed file(s) map to no test suite:" >&2 for f in "${UNMAPPED[@]}"; do echo " - $f" >&2 done echo "This is NOT 'nothing to run' — it is 'this tool does not know what covers these'." >&2 - echo "Fix one of: add a co-located .test.sh; make a suite name the file; or record the" >&2 - echo "path class in $NO_SUITE_LIST with the CI lane that does cover it." >&2 - if [[ "$allow_unmapped" -eq 0 ]]; then + echo "Fix one of: add a co-located .test.sh; make a suite name the file; declare a" >&2 + echo "test-scope on the suite that reads it; or record the path class in $NO_SUITE_LIST" >&2 + echo "with the CI lane that does cover it." >&2 + if [[ "$unmapped_corpus" -eq 1 ]]; then + declare -A CORPORA=() + for f in "${UNMAPPED[@]}"; do + lang_family "$f" + case "$LANG_FAMILY" in + py | node | ps) CORPORA["$LANG_FAMILY"]=1 ;; + *) CORPORA[sh]=1 ;; + esac + done + awk -v want=" ${!CORPORA[*]} " ' + { b = $0; sub(/.*\//, "", b) } + /\.test\.sh$/ { e = "sh" } + /\.test\.m?js$/ { e = "node" } + /\.Tests\.ps1$/ { e = "ps" } + b ~ /^test_.*\.py$/ { e = "py" } + e != "" && index(want, " " e " ") { print e "\t" $0 } + { e = "" }' "$WORK_DIR/all-files" >"$WORK_DIR/corpus" + while IFS=$'\t' read -r lang suite; do + add_suite "$suite" "unmapped-corpus: the $lang corpus of an unmapped file" || true + done <"$WORK_DIR/corpus" + echo "Selecting the whole corpus of each unmapped file's language under --unmapped-corpus: ${!CORPORA[*]}." >&2 + corpus_used=1 + elif [[ "$allow_unmapped" -eq 0 ]]; then echo "Re-run with --allow-unmapped to proceed anyway." >&2 exit 1 + else + echo "Proceeding under --allow-unmapped." >&2 fi - echo "Proceeding under --allow-unmapped." >&2 +fi + +# `${!SUITES[@]}` cannot carry a `+` default-guard: bash parses `${!NAME...}` as +# an indirect reference and rejects the expanded key list as a variable name. +declare -a selected=() +if [[ ${#SUITES[@]} -gt 0 ]]; then + # Checked, and read from a file: a failing `sort` here would empty a + # NON-EMPTY selection and report "every changed file is a no-suite class". + if ! printf '%s\n' "${!SUITES[@]}" | sort -u >"$WORK_DIR/selected"; then + echo "error: sorting the selected-suite list failed; refusing to report an empty selection" >&2 + echo " when ${#SUITES[@]} suite(s) were selected." >&2 + exit 2 + fi + mapfile -t selected <"$WORK_DIR/selected" fi if [[ ${#selected[@]} -eq 0 ]]; then @@ -1340,12 +1397,10 @@ if [[ ${#selected[@]} -eq 0 ]]; then exit 0 fi -# The partition. It happens HERE, after the unmapped check, after the deletion -# and no-suite reporting, and after the sort, so every leg derives the same -# full selection from the same diff and then keeps its own slice of it. Doing it -# earlier (partitioning the CHANGED FILES) would give each leg a different -# derivation, and the unmapped check would then fire on whichever leg happened -# to receive the unmapped file rather than on all of them. +# The partition happens HERE, after the unmapped check and the sort, so every +# leg derives the same full selection from the same diff and keeps its own +# slice; partitioning the changed files would let the unmapped check fire on +# only one leg. if [[ "$shard_total" -gt 1 ]]; then declare -a leg=() for ((si = shard_index; si < ${#selected[@]}; si += shard_total)); do @@ -1367,34 +1422,25 @@ fi if [[ "$do_run" -eq 0 ]]; then printf '%s\n' "${selected[@]}" + [[ "$corpus_used" -eq 1 ]] && exit 4 exit 0 fi -# SEQUENTIAL BY DEFAULT, --jobs N ON REQUEST. The default stays 1 because that -# is the measurement this file was written from: on a Windows Git Bash host a -# parallel run was sublinear (the suites are spawn-bound and the box saturates -# on process creation). On a Linux CI runner the same measurement came out the -# other way, which is why scripts/run-plugin-tests.sh has carried --jobs since -# it was written and why CI passes a count explicitly there. -# -# --jobs N > 1 hands the selection to run-plugin-tests.sh rather than spawning -# anything here: that runner already owns the worker, the bounded xargs -# dispatch, the per-suite print lock that keeps concurrent output from -# interleaving, and scripts/run-plugin-tests-serial.txt, the suites that assert -# wall-clock ceilings or drive concurrency probes and so must never overlap -# anything. A second parallel runner in this file would be a second copy of all -# four, and the serial allowlist is the one that must not be forgotten. Three -# is the proven ceiling on a 4-vCPU runner: at four, suites failed by producing -# empty output from an external command (#3694). +# SEQUENTIAL BY DEFAULT, --jobs N ON REQUEST. On a Windows Git Bash host a +# parallel run was sublinear (the suites are spawn-bound); on a Linux CI runner +# it pays, which is why CI passes a count. --jobs N > 1 hands the selection to +# run-plugin-tests.sh, which owns the worker, the bounded xargs dispatch, the +# per-suite print lock and scripts/run-plugin-tests-serial.txt, the suites that +# must never overlap anything. Three is the proven ceiling on a 4-vCPU runner: +# at four, suites failed by producing empty output from an external command +# (#3694). # # Only *.test.sh is executable HERE. The other three ecosystems are run by their -# own lanes, with lane-specific invocations this script cannot derive from a -# suite path: `python -m unittest` against a named module, `npm test`, a bare -# `node --test`, vitest, Pester. Guessing one is strictly worse than declining -# to: the wrong runner either errors in a way that reads as the SUITE failing, -# or exits 0 having run nothing, which is a PASS the change never earned. So -# they are named and NOT run, and the exit code says so rather than reporting -# success over suites that never executed. +# own lanes, with invocations this script cannot derive from a suite path +# (`python -m unittest` against a named module, `npm test`, `node --test`, +# vitest, Pester), and a guessed runner either errors as though the suite failed +# or exits 0 having run nothing. So they are named and NOT run, and the exit +# code says so. declare -a runnable=() delegated=() for s in "${selected[@]}"; do case "$s" in @@ -1408,7 +1454,7 @@ if [[ ${#runnable[@]} -gt 0 ]] && [[ "$jobs" -gt 1 ]]; then echo "Running ${#runnable[@]} selected shell suite(s) across up to $jobs job(s)." >&2 list="$WORK_DIR/selection.txt" printf '%s\n' "${runnable[@]}" >"$list" - bash "$(dirname "${BASH_SOURCE[0]}")/run-plugin-tests.sh" --jobs "$jobs" --suites-from "$list" || failed=1 + bash "$SCRIPT_DIR/run-plugin-tests.sh" --jobs "$jobs" --suites-from "$list" || failed=1 elif [[ ${#runnable[@]} -gt 0 ]]; then echo "Running ${#runnable[@]} selected shell suite(s) sequentially." >&2 for s in "${runnable[@]}"; do @@ -1439,4 +1485,8 @@ if [[ ${#delegated[@]} -gt 0 ]]; then echo "${#runnable[@]} shell suite(s) passed or were skipped; ${#delegated[@]} still need their own lane." >&2 exit 3 fi +if [[ "$corpus_used" -eq 1 ]]; then + echo "All ${#runnable[@]} selected suites passed or were skipped, including an unmapped file's corpus." >&2 + exit 4 +fi echo "All ${#runnable[@]} selected suites passed or were skipped." diff --git a/scripts/affected-tests.test.sh b/scripts/affected-tests.test.sh index ca3b83a99b..2b089d08b1 100755 --- a/scripts/affected-tests.test.sh +++ b/scripts/affected-tests.test.sh @@ -8,6 +8,9 @@ # against the LIVE repo — the derived shared-lib copy set and the real no-suite # list — because a synthetic fixture cannot show that the derivation still # tracks reality, which is the whole failure mode this tool exists to avoid. +# +# The live cases run every sync manifest and read the selector's own lists: +# test-scope: scripts/affected-tests* scripts/sync-*.sh scripts/lib/sync-*.sh set -uo pipefail TMP_ROOT="$(mktemp -d)" @@ -119,19 +122,6 @@ run_sel() { RC=$? } -# run_sel_rules : like run_sel, but OUT keeps only the suites -# R1-R7 select. R8 adds every shell suite of a touched plugin, so a case about -# what NAMES a file reads the selection through --explain and drops the suites -# whose recorded reason is R8's. A suite another rule also reached keeps that -# rule's reason, because a seed's own walk runs before R8, so it stays. -run_sel_rules() { - local repo="$1" err - shift - err="$(cd "$repo" && bash scripts/affected-tests.sh --explain "$@" 2>&1 >/dev/null)" - RC=$? - OUT="$(awk '/^select: / && !/ \(R8: / { sub(/^select: /, ""); sub(/ \(.*$/, ""); print }' <<<"$err")" -} - # Captured output is matched in-shell, never piped into a reader: under pipefail # an early-exit reader can kill the writer with SIGPIPE (see the pin below). # has_line : is one whole line of , matched literally. @@ -1055,28 +1045,36 @@ else fail "dual-ecosystem coverage (rc=$RC): $OUT" fi -# --- cross-language matches are followed exactly one hop ------------------- -# Widening the corpus to four ecosystems introduced an edge that never existed -# when the reverse lookup was `-- '*.sh'`: a match ACROSS languages. Left -# uncapped, those coincidental matches chained (js -> ps1 -> sh) and saturated -# on the far-end hubs. One hop still reaches the suite covering the crossed-to file, -# which is what keeps a .sh wrapper around a .py helper working; what it must -# NOT do is keep walking from there and drag in that file's own dependents. +# --- R4: another language counts only where it runs or loads the file ------ +# A file in another language that merely contains the name (a string, a log +# message) is not a dependent and its suite is not selected: across languages +# the text says nothing about a dependency unless the line runs or loads the +# file. An interpreter on the line, or a path to the file, does. mkdir -p "$repo/eco/hop" printf 'export const c = 3;\n' >"$repo/eco/hop/origin.js" -# A .ps1 that merely MENTIONS the js basename in a string — not a real -# dependency, but a code line, so the walk still crosses into it. A comment-only -# mention would not, and the case would pass without testing the crossing rule. -printf "\$null = 'origin.js'\nfunction Get-Far { 2 }\n" >"$repo/eco/hop/Far.ps1" -# A shell file that depends on the .ps1, with its own suite. Reaching this suite -# would require a SECOND cross-language hop, which the rule forbids. -printf 'echo "runs Far.ps1"\n' >"$repo/eco/hop/far-runner.sh" +printf 'echo "origin.js is the entry point"\n' >"$repo/eco/hop/mention.test.sh" +# shellcheck disable=SC2016 # deliberate: the emitted fixture must expand these +printf 'node "$(dirname "$0")/origin.js"\n' >"$repo/eco/hop/node-runs.test.sh" +# shellcheck disable=SC2016 # deliberate: the emitted fixture must expand these +printf 'cp "$SRC/eco/hop/origin.js" "$DEST"\n' >"$repo/eco/hop/path-loads.test.sh" +# A .ps1 that runs the js, so the walk crosses into it once, and a shell file +# that runs the .ps1: reaching ITS suite takes a second transition. +printf "node origin.js\nfunction Get-Far { 2 }\n" >"$repo/eco/hop/Far.ps1" +suite_body far >"$repo/eco/hop/Far.Tests.ps1" +printf 'pwsh -File Far.ps1\n' >"$repo/eco/hop/far-runner.sh" suite_body far-runner >"$repo/eco/hop/far-runner.test.sh" run_sel "$repo" eco/hop/origin.js -if ! has_line "$OUT" eco/hop/far-runner.test.sh; then - ok "a chain that already crossed languages cannot cross a second time" +if ! has_line "$OUT" eco/hop/mention.test.sh && + has_line "$OUT" eco/hop/node-runs.test.sh && + has_line "$OUT" eco/hop/path-loads.test.sh; then + ok "R4: another language's suite runs only where its line runs or loads the file" else - fail "cross-language walk took a second transition (rc=$RC): $OUT" + fail "R4: cross-language selection wrong (rc=$RC): $OUT" +fi +if has_line "$OUT" eco/hop/Far.Tests.ps1 && ! has_line "$OUT" eco/hop/far-runner.test.sh; then + ok "R4: a chain crosses languages once and cannot cross a second time" +else + fail "R4: the transition budget was not applied (rc=$RC): $OUT" fi # --- --run refuses to guess a runner for another ecosystem ----------------- @@ -1117,7 +1115,8 @@ mk_repo repo mkdir -p "$repo/eco/chain" printf 'def helper():\n return 1\n' >"$repo/eco/chain/helper.py" # The shell wrapper that drives the Python helper: one crossing, py -> sh. -printf 'echo "runs helper.py"\n' >"$repo/eco/chain/runner.sh" +# shellcheck disable=SC2016 # deliberate: the emitted fixture must expand these, not this shell +printf 'python3 "$(dirname "$0")/helper.py"\n' >"$repo/eco/chain/runner.sh" # A shell dependent of the wrapper: the SECOND edge, shell -> shell. # shellcheck disable=SC2016 # deliberate: the emitted fixture must expand these, not this shell printf 'source "$(dirname "$0")/runner.sh"\n' >"$repo/eco/chain/command.sh" @@ -1130,6 +1129,52 @@ else fail "py -> sh -> sh chain lost its suite (rc=$RC): $OUT" fi +# --- Python imports name the module they load --------------------------------- +# `import tool` never spells tool.py, so the import line is the edge: from the +# module's directory or below it, by a dotted path that spells it, or from +# anywhere in the plugin when the module name is unique there. A module of the +# same name elsewhere in the plugin makes the bare import ambiguous, and a +# module nothing imports selects only its own suites. +mkdir -p "$repo/plugins/alpha/scripts/tests" "$repo/plugins/alpha/skills/one/scripts" \ + "$repo/plugins/beta/scripts" "$repo/plugins/alpha/pkg/sub" +printf 'def run():\n return 1\n' >"$repo/plugins/alpha/scripts/tool.py" +printf 'from tool import run\n' >"$repo/plugins/alpha/scripts/runner.py" +printf 'import runner\n' >"$repo/plugins/alpha/scripts/test_runner.py" +printf 'import sys\nimport tool as t\n' >"$repo/plugins/alpha/scripts/tests/test_tool_behavior.py" +printf 'import tool\n' >"$repo/plugins/alpha/skills/one/scripts/use_tool.py" +printf 'import use_tool\n' >"$repo/plugins/alpha/skills/one/scripts/test_use_tool.py" +printf 'import tool\n' >"$repo/plugins/beta/scripts/other.py" +printf 'import other\n' >"$repo/plugins/beta/scripts/test_other.py" +printf 'X = 1\n' >"$repo/plugins/alpha/pkg/sub/deep.py" +printf 'from pkg.sub.deep import X\n' >"$repo/plugins/alpha/dotted.py" +printf 'import dotted\n' >"$repo/plugins/alpha/test_dotted.py" +git_test_config "$repo" add plugins >/dev/null +git_test_config "$repo" commit -qm pyimports >/dev/null +run_sel "$repo" plugins/alpha/scripts/tool.py +if [[ "$RC" -eq 0 ]] && has_line "$OUT" plugins/alpha/scripts/test_runner.py && + has_line "$OUT" plugins/alpha/scripts/tests/test_tool_behavior.py && + has_line "$OUT" plugins/alpha/skills/one/scripts/test_use_tool.py && + ! has_line "$OUT" plugins/beta/scripts/test_other.py; then + ok "python: an import selects from the module's directory, below it, and across its plugin" +else + fail "python: import selection wrong for tool.py (rc=$RC): $OUT" +fi +run_sel "$repo" plugins/alpha/pkg/sub/deep.py +if [[ "$RC" -eq 0 ]] && has_line "$OUT" plugins/alpha/test_dotted.py; then + ok "python: a dotted import that spells the module's path selects" +else + fail "python: dotted import lost (rc=$RC): $OUT" +fi +printf 'def run():\n return 2\n' >"$repo/plugins/alpha/skills/one/scripts/tool.py" +run_sel "$repo" plugins/alpha/scripts/tool.py +if [[ "$RC" -eq 0 ]] && has_line "$OUT" plugins/alpha/scripts/test_runner.py && + ! has_line "$OUT" plugins/alpha/skills/one/scripts/test_use_tool.py; then + ok "python: a module name two directories of a plugin carry resolves by directory only" +else + fail "python: an ambiguous module name still selected across the plugin (rc=$RC): $OUT" +fi +rm -f "$repo/plugins/alpha/skills/one/scripts/tool.py" + # --- the crossing budget is aggregated per path, never assigned ------------ # One path can be hit several times in a single round by different patterns, and # those hits can disagree about whether the chain reaching it has already @@ -1148,9 +1193,9 @@ printf 'mixed_helper() { echo mixed; }\n' >"$repo2/lib/mixed.sh" printf 'export const mixed = 1;\n' >"$repo2/plugins/alpha/hooks/mixed.js" # Names the shell source FIRST, the JS copy SECOND — so the cross-family hit is -# the one a last-write-wins bug would keep. The second mention is a `:` no-op -# rather than a comment, because a comment-only line makes no dependent at all. -printf 'source "lib/mixed.sh"\n: also mirrors mixed.js\n' >"$repo2/eco/agg/zed.sh" +# the one a last-write-wins bug would keep. The second line runs the copy, so +# R4 takes it as a crossing; a bare mention would make no dependent at all. +printf 'source "lib/mixed.sh"\nnode mixed.js\n' >"$repo2/eco/agg/zed.sh" # A genuine crossing OUT of zed.sh, which is exactly what a wrongly-spent budget # would block. Its suite is the assertion. printf 'import subprocess # drives zed.sh\n' >"$repo2/eco/agg/zed_user.py" @@ -1190,7 +1235,7 @@ printf 'echo target\n' >"$repo3/eco/name/deep-target.sh" printf 'echo prose\n' >"$repo3/eco/name/prose-target.sh" { printf '#!/usr/bin/env bash\n' - printf '# Every rejected shape is covered by prose-target.sh.\n' + printf 'echo "Every rejected shape is covered by prose-target.sh."\n' } >"$repo3/eco/name/prose.test.sh" # A file named with an ELLIPSIS butted straight against it. The mirror of the @@ -1201,7 +1246,7 @@ printf 'echo prose\n' >"$repo3/eco/name/prose-target.sh" printf 'echo ellipsis\n' >"$repo3/eco/name/ellipsis-target.sh" { printf '#!/usr/bin/env bash\n' - printf '# the rest of that argument lives in ...ellipsis-target.sh\n' + printf 'echo "the rest of that argument lives in ...ellipsis-target.sh"\n' } >"$repo3/eco/name/ellipsis.test.sh" # A basename the token rule cannot spell, because `+` is not a path-token @@ -1250,9 +1295,8 @@ fi # The exit code alone is not the assertion: what must be gone is the SELECTION # the substring match handed it. Under --allow-unmapped the run proceeds, so an -# empty rule selection is direct evidence that no unrelated suite was borrowed -# (R8's plugin suites are a different rule, and never map a file). -run_sel_rules "$repo3" --allow-unmapped plugins/alpha/hooks/get.sh +# empty selection is direct evidence that no unrelated suite was borrowed. +run_sel "$repo3" --allow-unmapped plugins/alpha/hooks/get.sh if [[ "$RC" -eq 0 && -z "$OUT" ]]; then ok "the borrowed suites are gone, not merely re-labeled" else @@ -1310,11 +1354,12 @@ else fi rm -rf "$repo3" -# --- a comment-only mention in a NON-suite file makes no dependent ----------- -# Hub files cite neighboring scripts in prose, and counting those as R4 edges -# fanned one plugin's change out to most of the corpus. A suite's comment still -# names the file (R3), and so does every code line, a trailing comment on one, -# a shellcheck source directive and a JSDoc type import. +# --- a comment-only mention makes no dependent and selects no suite ---------- +# Hub files cite neighboring scripts in prose, and counting those as edges +# fanned one plugin's change out to most of the corpus; a suite citing a file +# in prose does not run it either. Every code line still names the file, as do +# a trailing comment on one, a shellcheck source directive and a JSDoc type +# import. mk_repo repo3 mkdir -p "$repo3/eco/cmt" printf 'echo hub\n' >"$repo3/eco/cmt/hub-target.sh" @@ -1365,10 +1410,10 @@ else fail "a non-comment mention lost its dependent (rc=$RC): $OUT" fi -if has_line "$OUT" eco/cmt/hub-prose.test.sh; then - ok "a suite's comment mention still selects it" +if ! has_line "$OUT" eco/cmt/hub-prose.test.sh; then + ok "a suite that names the file only in a comment is not selected" else - fail "a suite naming the file in a comment was dropped (rc=$RC): $OUT" + fail "a suite's comment-only mention still selected it (rc=$RC): $OUT" fi run_sel "$repo3" eco/cmt/hubmod.py @@ -1477,16 +1522,23 @@ else fi fi -# --- R8: a plugin change selects every shell suite of that plugin ------------- -# A suite that globs its own plugin directory never spells the changed file's -# name, so only a path rule reaches it from a SKILL.md edit. Pinned: the -# plugin's shell suites are in, its Python suite and another plugin's suites are -# out, the reason reads R8, and R8 never maps a file, so a plugin file nothing -# names is still UNMAPPED. +# --- R8: a declared test-scope selects the suite that scans a directory ------ +# A suite that greps or globs a directory never spells the files it reads, so +# it declares them in its leading comment block, and a matching change selects +# it and counts as mapped. Pinned: the glob crosses `/`, the plugin's other +# suites stay out, a declaration below the first code line is not read, the +# Node `//` form works, and a plugin file nothing names or declares is still +# UNMAPPED. mk_repo repo mkdir -p "$repo/plugins/alpha/skills/one" "$repo/plugins/alpha/tests" printf -- '---\nname: one\n---\n' >"$repo/plugins/alpha/skills/one/SKILL.md" -suite_body alpha-scan >"$repo/plugins/alpha/tests/scan.test.sh" +printf 'kind: probe\n' >"$repo/plugins/alpha/skills/one/probe.yaml" +{ + printf '#!/usr/bin/env bash\n# Scans every skill body.\n# test-scope: plugins/alpha/skills/*.md\n' + printf '# test-scope: plugins/alpha/*.yaml\n\nset -u\n# test-scope: plugins/beta/*\n' +} >"$repo/plugins/alpha/tests/scan.test.sh" +printf '// test-scope: plugins/alpha/skills/*/SKILL.md\nimport test from "node:test";\n' \ + >"$repo/plugins/alpha/tests/scan.test.mjs" printf 'import unittest\n' >"$repo/plugins/alpha/tests/test_scan.py" printf 'echo orphan\n' >"$repo/plugins/alpha/zzorphan-plugin.sh" git_test_config "$repo" add plugins >/dev/null @@ -1494,165 +1546,222 @@ git_test_config "$repo" commit -qm r8 >/dev/null run_sel "$repo" plugins/alpha/skills/one/SKILL.md if [[ "$RC" -eq 0 ]] && has_line "$OUT" plugins/alpha/tests/scan.test.sh && - has_line "$OUT" plugins/alpha/hooks/alpha-hook.test.sh && - ! has_line "$OUT" plugins/alpha/tests/test_scan.py && - ! has_line "$OUT" plugins/beta/hooks/beta-hook.test.sh; then - ok "R8: a SKILL.md edit selects its plugin's shell suites and nothing else" + has_line "$OUT" plugins/alpha/tests/scan.test.mjs && + ! has_line "$OUT" plugins/alpha/hooks/alpha-hook.test.sh && + ! has_line "$OUT" plugins/alpha/tests/test_scan.py; then + ok "R8: a SKILL.md edit selects the suites that declare it and no other suite of the plugin" else - fail "R8: plugin selection wrong for a SKILL.md edit (rc=$RC): $OUT" + fail "R8: declared-scope selection wrong for a SKILL.md edit (rc=$RC): $OUT" fi out="$(cd "$repo" && bash scripts/affected-tests.sh --explain plugins/alpha/skills/one/SKILL.md 2>&1)" -if contains "$out" "select: plugins/alpha/tests/scan.test.sh (R8: plugins/alpha/ changed)"; then - ok "R8: --explain reports the plugin reason" +if contains "$out" "select: plugins/alpha/tests/scan.test.sh (test-scope plugins/alpha/skills/*.md)"; then + ok "R8: --explain reports the declared glob" else - fail "R8: --explain lacks the plugin reason: $out" + fail "R8: --explain lacks the declared glob: $out" fi -run_sel "$repo" plugins/alpha/zzorphan-plugin.sh -if [[ "$RC" -eq 1 ]]; then - ok "R8: a plugin file no suite names is still UNMAPPED" +run_sel "$repo" plugins/alpha/skills/one/probe.yaml +if [[ "$RC" -eq 0 ]] && has_line "$OUT" plugins/alpha/tests/scan.test.sh; then + ok "R8: a declared file no other rule reaches is mapped, not UNMAPPED" else - fail "R8: R8 mapped a plugin file nothing names (rc=$RC): $OUT" + fail "R8: a declared file was not mapped by its scope (rc=$RC): $OUT" fi -rm -rf "$repo" - -# --- --with-always: the live-tree suites ride every selection ---------------- -# scripts/affected-tests-always.txt lists suites that assert against the live -# repository, so no rule can see the change that breaks them. Pinned: off by -# default, on under --with-always even when the diff selects nothing else, and -# a stale entry is an error rather than a quiet skip. -mk_repo repo -mkdir -p "$repo/scripts/lib" -suite_body live-scan >"$repo/scripts/lib/live-scan.test.sh" -printf '# reason-bearing entries\nscripts/lib/live-scan.test.sh scans every script\n' \ - >"$repo/scripts/affected-tests-always.txt" -git_test_config "$repo" add scripts >/dev/null -git_test_config "$repo" commit -qm always >/dev/null -run_sel "$repo" plugins/alpha/hooks/alpha-hook.sh -if [[ "$RC" -eq 0 ]] && ! has_line "$OUT" scripts/lib/live-scan.test.sh; then - ok "always-run: off by default" +run_sel "$repo" plugins/beta/hooks/beta-hook.sh +if [[ "$RC" -eq 0 ]] && ! has_line "$OUT" plugins/alpha/tests/scan.test.sh; then + ok "R8: a declaration below the leading comment block is not read" else - fail "always-run: a local run selected the live-tree suite (rc=$RC): $OUT" + fail "R8: a declaration after the first code line was honored (rc=$RC): $OUT" fi -run_sel "$repo" --with-always plugins/alpha/hooks/alpha-hook.sh -if [[ "$RC" -eq 0 ]] && has_line "$OUT" scripts/lib/live-scan.test.sh && - has_line "$OUT" plugins/alpha/hooks/alpha-hook.test.sh; then - ok "always-run: --with-always adds the listed suite to the selection" +run_sel "$repo" plugins/alpha/zzorphan-plugin.sh +if [[ "$RC" -eq 1 ]]; then + ok "R8: a plugin file no suite names or declares is still UNMAPPED" else - fail "always-run: --with-always did not add the listed suite (rc=$RC): $OUT" + fail "R8: a plugin file nothing names or declares was mapped (rc=$RC): $OUT" fi -run_sel "$repo" --with-always plugins/alpha/README.md -if [[ "$RC" -eq 0 ]] && has_line "$OUT" scripts/lib/live-scan.test.sh; then - ok "always-run: a no-suite diff still runs the listed suite" +# A changed suite's declarations are checked: one below the leading comment +# block is never read, and a glob matching no file has outlived what it read. +out="$(cd "$repo" && bash scripts/affected-tests.sh plugins/alpha/tests/scan.test.sh 2>&1)" +RC=$? +if [[ "$RC" -eq 2 ]] && contains "$out" 'plugins/alpha/tests/scan.test.sh:7'; then + ok "R8: a changed suite with a declaration below its header fails loud" else - fail "always-run: a no-suite diff dropped the listed suite (rc=$RC): $OUT" + fail "R8: a misplaced declaration was not refused (rc=$RC): $out" fi - -printf 'scripts/lib/gone.test.sh removed long ago\n' >>"$repo/scripts/affected-tests-always.txt" -out="$(cd "$repo" && bash scripts/affected-tests.sh --with-always plugins/alpha/hooks/alpha-hook.sh 2>&1)" +printf '#!/usr/bin/env bash\n# test-scope: plugins/nowhere/*\necho stale\n' >"$repo/plugins/alpha/tests/stale.test.sh" +out="$(cd "$repo" && bash scripts/affected-tests.sh plugins/alpha/tests/stale.test.sh 2>&1)" RC=$? -if [[ "$RC" -eq 2 ]] && contains "$out" "names 'scripts/lib/gone.test.sh'"; then - ok "always-run: a stale entry is an error, not a quiet skip" +if [[ "$RC" -eq 2 ]] && contains "$out" 'plugins/nowhere/*, which matches no file'; then + ok "R8: a changed suite declaring a glob that matches nothing fails loud" +else + fail "R8: a stale declaration was not refused (rc=$RC): $out" +fi +rm -f "$repo/plugins/alpha/tests/stale.test.sh" + +# --- --unmapped-corpus: an unmapped file selects its own language's corpus --- +# The report stays, the exit says so (4), and only the file's language runs: a +# shell file never starts the Python corpus, and the other way round. +run_sel "$repo" --unmapped-corpus plugins/alpha/zzorphan-plugin.sh +if [[ "$RC" -eq 4 ]] && has_line "$OUT" plugins/alpha/tests/scan.test.sh && + has_line "$OUT" plugins/beta/hooks/beta-hook.test.sh && has_line "$OUT" lib/widget.test.sh && + ! has_line "$OUT" plugins/alpha/tests/test_scan.py && ! has_line "$OUT" plugins/alpha/tests/scan.test.mjs; then + ok "--unmapped-corpus: an unmapped .sh selects the shell corpus only, at exit 4" +else + fail "--unmapped-corpus: wrong corpus or exit for an unmapped .sh (rc=$RC): $OUT" +fi +printf 'X = 1\n' >"$repo/plugins/alpha/zz_orphan_mod.py" +run_sel "$repo" --unmapped-corpus plugins/alpha/zz_orphan_mod.py +if [[ "$RC" -eq 4 ]] && has_line "$OUT" plugins/alpha/tests/test_scan.py && + ! has_line "$OUT" plugins/alpha/tests/scan.test.sh; then + ok "--unmapped-corpus: an unmapped .py selects the Python corpus only" +else + fail "--unmapped-corpus: wrong corpus for an unmapped .py (rc=$RC): $OUT" +fi +out="$(cd "$repo" && bash scripts/affected-tests.sh --unmapped-corpus plugins/alpha/zz_orphan_mod.py 2>&1 >/dev/null)" +if contains "$out" 'UNMAPPED: 1 changed file(s)'; then + ok "--unmapped-corpus: the unmapped report is still printed" else - fail "always-run: a stale entry was not refused (rc=$RC): $out" + fail "--unmapped-corpus: the unmapped report went missing: $out" +fi +run_sel "$repo" --unmapped-corpus --allow-unmapped plugins/alpha/zz_orphan_mod.py +if [[ "$RC" -eq 2 ]]; then + ok "--unmapped-corpus with --allow-unmapped is a usage error" +else + fail "--unmapped-corpus with --allow-unmapped should exit 2 (rc=$RC)" fi rm -rf "$repo" -# --- LIVE repo: every always-run entry names a suite today -------------------- -out="$(cd "$REPO_ROOT" && bash scripts/affected-tests.sh --with-always scripts/affected-tests-always.txt 2>&1)" -RC=$? -if [[ "$RC" -eq 0 ]] && contains "$out" "scripts/lib/gate-entry.test.sh"; then - ok "LIVE always-run: the shipped list resolves and joins the selection" +# --- --with-always is accepted and widens nothing ---------------------------- +# A caller that still passes it must neither fail nor get a wider selection: +# the live-tree suites it used to add declare a test-scope now. +mk_repo repo +run_sel "$repo" --with-always plugins/alpha/hooks/alpha-hook.sh +with_out="$OUT" with_rc="$RC" +run_sel "$repo" plugins/alpha/hooks/alpha-hook.sh +if [[ "$with_rc" -eq 0 && "$RC" -eq 0 && "$with_out" == "$OUT" ]]; then + ok "--with-always is accepted and changes nothing" else - fail "LIVE always-run: the shipped list did not resolve (rc=$RC): $out" + fail "--with-always changed the run (rc=$with_rc vs $RC): [$with_out] vs [$OUT]" fi +rm -rf "$repo" + +# --- LIVE repo: the two suite breaks only a full main run caught -------------- +# Both were a skill body edit breaking a suite that never spells the body's +# path the plain way: one scans its plugin's markdown (R8 declares it), the +# other spells the body relative to its plugin (AMBIGUOUS NAMES resolves it). +# The probe bodies are discovered, never spelled, so this suite does not name +# them and run on every edit to them. +for probe in 'plugins/github/skills/advise/S*.md|plugins/github/github.test.sh' \ + 'plugins/planning/skills/interview/S*.md|plugins/planning/tests/interview-defenses.test.sh'; do + body="$(cd "$REPO_ROOT" && git ls-files "${probe%%|*}")" + out="$(cd "$REPO_ROOT" && bash scripts/affected-tests.sh "$body" 2>/dev/null)" + RC=$? + if [[ -n "$body" && "$RC" -eq 0 ]] && has_line "$out" "${probe#*|}"; then + ok "LIVE: $body selects ${probe#*|}" + else + fail "LIVE: '$body' did not select ${probe#*|} (rc=$RC): $out" + fi +done -# --- skill ownership: a bare reference name means the skill's OWN file ------- -# Skills reuse reference names freely, so a suite naming `ownership-probe.md` from inside -# one skill is naming that skill's file. Changing another skill's `ownership-probe.md` -# must not select it, and a path-qualified mention through the owning skill must -# still select from anywhere. +# --- ambiguous names: a shared basename counts only where it resolves ------ +# Skills reuse reference names freely, so a bare `probe-doc.md` says nothing +# about which one. A mention names a file only when it resolves to it: a path +# suffix no other file of that name ends in, any mention from the file's own +# directory, or a path relative to a directory holding both files. The +# structural names (SKILL.md, plugin.json and the like) always resolve this way. mk_repo repo own_a=plugins/alpha/skills/sa own_b=plugins/alpha/skills/sb -mkdir -p "$repo/$own_a/reference" "$repo/$own_a/scripts" \ - "$repo/$own_b/reference" "$repo/$own_b/scripts" \ - "$repo/plugins/beta/skills/sc/scripts" -printf '# a probe\n' >"$repo/$own_a/reference/ownership-probe.md" -printf '# b probe\n' >"$repo/$own_b/reference/ownership-probe.md" -printf '#!/usr/bin/env bash\n# checks the wording in ownership-probe.md\n' >"$repo/$own_a/scripts/sa.test.sh" -printf '#!/usr/bin/env bash\n# checks the wording in ownership-probe.md\n' >"$repo/$own_b/scripts/sb.test.sh" -# Another skill's suite that spells the path through skill sb. -printf '#!/usr/bin/env bash\n# reads plugins/alpha/skills/sb/reference/ownership-probe.md\n' \ - >"$repo/plugins/beta/skills/sc/scripts/sc.test.sh" -# A bare mention from a skill that carries no file of that name itself. -printf '#!/usr/bin/env bash\n# checks the wording in ownership-probe.md\n' \ - >"$repo/plugins/beta/skills/sc/scripts/sc-bare.test.sh" -# A suite inside skill sa that names the file only through skill sb. -printf '#!/usr/bin/env bash\n# reads plugins/alpha/skills/sb/reference/ownership-probe.md\n' \ - >"$repo/$own_a/scripts/sa-cross.test.sh" -# Another plugin's skill of the same name as sa, carrying the same file, named by path. -mkdir -p "$repo/plugins/gamma/skills/sa/reference" -printf '# gamma probe\n' >"$repo/plugins/gamma/skills/sa/reference/ownership-probe.md" -printf '#!/usr/bin/env bash\n# reads plugins/gamma/skills/sa/reference/ownership-probe.md\n' \ - >"$repo/plugins/beta/skills/sc/scripts/sc-gamma.test.sh" +mkdir -p "$repo/$own_a/reference" "$repo/$own_a/scripts" "$repo/$own_b/reference" "$repo/$own_b/scripts" \ + "$repo/plugins/beta/skills/sc/scripts" "$repo/plugins/gamma/skills/sa/reference" \ + "$repo/plugins/alpha/skills/one" "$repo/plugins/beta/skills/one" "$repo/plugins/alpha/tests" \ + "$repo/plugins/alpha/.claude-plugin" +for d in "$own_a" "$own_b" plugins/gamma/skills/sa; do + printf '# probe\n' >"$repo/$d/reference/probe-doc.md" +done +# shellcheck disable=SC2016 # deliberate: the emitted fixtures must expand these +{ + printf 'grep -q wording "$SKILL_DIR/reference/probe-doc.md"\n' >"$repo/$own_a/scripts/sa.test.sh" + printf 'grep -q wording "$SKILL_DIR/reference/probe-doc.md"\n' >"$repo/$own_b/scripts/sb.test.sh" + printf 'grep -q wording "$ROOT/plugins/alpha/skills/sb/reference/probe-doc.md"\n' \ + >"$repo/plugins/beta/skills/sc/scripts/sc.test.sh" + printf 'grep -q wording probe-doc.md\n' >"$repo/plugins/beta/skills/sc/scripts/sc-bare.test.sh" + printf 'cat plugins/gamma/skills/sa/reference/probe-doc.md\n' >"$repo/plugins/beta/skills/sc/scripts/sc-gamma.test.sh" + printf '# reads plugins/alpha/skills/sa/reference/probe-doc.md\n' >"$repo/$own_a/scripts/sa-comment.test.sh" + # A bare mention from the file's own directory, in a script whose suite is its sibling. + printf 'grep -c . probe-doc.md\n' >"$repo/$own_a/reference/count.sh" + suite_body count >"$repo/$own_a/reference/count.test.sh" + # The SKILL.md shape: a plugin suite spelling the body relative to its plugin. + printf -- '---\nname: one\n---\n' >"$repo/plugins/alpha/skills/one/SKILL.md" + printf -- '---\nname: one\n---\n' >"$repo/plugins/beta/skills/one/SKILL.md" + printf 'body="$PLUGIN_DIR/skills/one/SKILL.md"\n' >"$repo/plugins/alpha/tests/skill-body.test.sh" + printf '{ "name": "alpha" }\n' >"$repo/plugins/alpha/.claude-plugin/plugin.json" + printf 'jq . "$PLUGIN_DIR/.claude-plugin/plugin.json"\n' >"$repo/plugins/alpha/tests/manifest.test.sh" +} git_test_config "$repo" add plugins >/dev/null -git_test_config "$repo" commit -qm skills >/dev/null +git_test_config "$repo" commit -qm ambiguous >/dev/null -# Every suite here sits in plugin alpha or beta, so R8 selects all of a touched -# plugin's suites; the ownership rule is read from the R1-R7 selection. -run_sel_rules "$repo" "$own_a/reference/ownership-probe.md" +run_sel "$repo" "$own_a/reference/probe-doc.md" if [[ "$RC" -eq 0 ]] && has_line "$OUT" "$own_a/scripts/sa.test.sh" && + has_line "$OUT" "$own_a/reference/count.test.sh" && ! has_line "$OUT" "$own_b/scripts/sb.test.sh" && - ! has_line "$OUT" plugins/beta/skills/sc/scripts/sc.test.sh; then - ok "ownership: skill A's ownership-probe.md selects A's suite and not B's" + ! has_line "$OUT" plugins/beta/skills/sc/scripts/sc.test.sh && + ! has_line "$OUT" plugins/beta/skills/sc/scripts/sc-bare.test.sh && + ! has_line "$OUT" plugins/beta/skills/sc/scripts/sc-gamma.test.sh && + ! has_line "$OUT" "$own_a/scripts/sa-comment.test.sh"; then + ok "ambiguous: skill-relative and same-directory mentions resolve; bare, other-skill and comment ones do not" else - fail "ownership: A's ownership-probe.md selection wrong (rc=$RC): $OUT" + fail "ambiguous: skill A's probe-doc.md selection wrong (rc=$RC): $OUT" fi -run_sel_rules "$repo" "$own_b/reference/ownership-probe.md" +run_sel "$repo" "$own_b/reference/probe-doc.md" if [[ "$RC" -eq 0 ]] && has_line "$OUT" "$own_b/scripts/sb.test.sh" && + has_line "$OUT" plugins/beta/skills/sc/scripts/sc.test.sh && ! has_line "$OUT" "$own_a/scripts/sa.test.sh"; then - ok "ownership: skill B's ownership-probe.md still selects B's suite and not A's" -else - fail "ownership: B's ownership-probe.md selection wrong (rc=$RC): $OUT" -fi - -if has_line "$OUT" plugins/beta/skills/sc/scripts/sc.test.sh; then - ok "ownership: a path-qualified mention from another skill still selects" + ok "ambiguous: a unique path suffix resolves from another plugin" else - fail "ownership: path-qualified mention from another skill lost (rc=$RC): $OUT" + fail "ambiguous: skill B's probe-doc.md selection wrong (rc=$RC): $OUT" fi -run_sel_rules "$repo" "$own_a/reference/ownership-probe.md" -if has_line "$OUT" plugins/beta/skills/sc/scripts/sc-bare.test.sh; then - ok "ownership: a bare mention from a skill without its own file still selects" +run_sel "$repo" plugins/gamma/skills/sa/reference/probe-doc.md +if [[ "$RC" -eq 0 ]] && has_line "$OUT" plugins/beta/skills/sc/scripts/sc-gamma.test.sh && + ! has_line "$OUT" "$own_a/scripts/sa.test.sh"; then + ok "ambiguous: a same-named skill in another plugin is told apart by its path" else - fail "ownership: bare mention from a file-less skill lost (rc=$RC): $OUT" + fail "ambiguous: gamma's probe-doc.md selection wrong (rc=$RC): $OUT" fi -if ! has_line "$OUT" "$own_a/scripts/sa-cross.test.sh"; then - ok "ownership: a suite in the owning skill that names another skill's file is not selected" +run_sel "$repo" plugins/alpha/skills/one/SKILL.md +alpha_out="$OUT" alpha_rc="$RC" +run_sel "$repo" plugins/beta/skills/one/SKILL.md +if [[ "$alpha_rc" -eq 0 ]] && has_line "$alpha_out" plugins/alpha/tests/skill-body.test.sh && + [[ "$RC" -eq 0 ]] && ! has_line "$OUT" plugins/alpha/tests/skill-body.test.sh; then + ok "ambiguous: a SKILL.md spelled relative to its plugin selects that plugin's suite only" else - fail "ownership: owner-local suite naming another skill's path was selected (rc=$RC): $OUT" + fail "ambiguous: SKILL.md resolution wrong (rc=$alpha_rc/$RC): [$alpha_out] [$OUT]" fi -if ! has_line "$OUT" plugins/beta/skills/sc/scripts/sc-gamma.test.sh; then - ok "ownership: a path through another plugin's same-named skill is not selected" +mkdir -p "$repo/plugins/beta/.claude-plugin" +printf '{ "name": "beta" }\n' >"$repo/plugins/beta/.claude-plugin/plugin.json" +run_sel "$repo" plugins/alpha/.claude-plugin/plugin.json +alpha_out="$OUT" alpha_rc="$RC" +run_sel "$repo" plugins/beta/.claude-plugin/plugin.json +if [[ "$alpha_rc" -eq 0 ]] && has_line "$alpha_out" plugins/alpha/tests/manifest.test.sh && + [[ "$RC" -eq 0 && -z "$OUT" ]]; then + ok "ambiguous: a manifest named relative to its plugin selects that suite; another plugin's does not" else - fail "ownership: same-named skill in another plugin was accepted (rc=$RC): $OUT" + fail "ambiguous: plugin.json resolution wrong (rc=$alpha_rc/$RC): [$alpha_out] [$OUT]" fi rm -rf "$repo" # --- R5 copies inside skill directories still reach every copy's suite ------- # A shared source outside any skill, copied into each skill's reference/, with -# each skill's suite naming its own copy bare. The source's fan-out must reach -# every suite: ownership only narrows basenames that no file outside a skill -# directory carries, and the source is one. +# each skill's suite naming its own copy bare. The basename is carried three +# times, but a shared library's copies keep the plain rule, so the source's +# fan-out reaches every suite. mk_repo repo mkdir -p "$repo/plugins/alpha/skills/sa/reference" "$repo/plugins/beta/skills/sb/reference" write_print_manifest "$repo/scripts/sync-guard.sh" "lib/guard-util.sh" \ @@ -1661,7 +1770,7 @@ printf 'guard_util() { echo guard; }\n' >"$repo/lib/guard-util.sh" for p in alpha beta; do s=s${p:0:1} printf 'guard_util() { echo guard; }\n' >"$repo/plugins/$p/skills/$s/reference/guard-util.sh" - printf '#!/usr/bin/env bash\n# sources guard-util.sh\n' >"$repo/plugins/$p/skills/$s/reference/$s.test.sh" + printf '#!/usr/bin/env bash\nsource guard-util.sh\n' >"$repo/plugins/$p/skills/$s/reference/$s.test.sh" done git_test_config "$repo" add lib scripts plugins >/dev/null git_test_config "$repo" commit -qm guard >/dev/null @@ -1686,6 +1795,54 @@ else fi rm -rf "$repo" +# --- --replay: each commit selected against its parent, with this tree's rules -- +# The replayed commits predate this tree's declarations, so a replay carries +# the tree's own test-scope table to every commit; --against runs the selector +# at with 's table and prints only the suites the two disagree on. +mk_repo repo +printf '#!/usr/bin/env bash\n# test-scope: plugins/alpha/*\necho old\n' >"$repo/scripts/zz-old-scan.test.sh" +printf '#!/usr/bin/env bash\necho new\n' >"$repo/scripts/zz-new-scan.test.sh" +git_test_config "$repo" add scripts >/dev/null +git_test_config "$repo" commit -qm scans >/dev/null +printf '# edited\n' >>"$repo/plugins/alpha/hooks/alpha-hook.sh" +git_test_config "$repo" commit -qam 'edit alpha hook' >/dev/null +alpha_commit="$(git -C "$repo" rev-parse HEAD)" +printf '#!/usr/bin/env bash\necho old\n' >"$repo/scripts/zz-old-scan.test.sh" +printf '#!/usr/bin/env bash\n# test-scope: plugins/alpha/*\necho new\n' >"$repo/scripts/zz-new-scan.test.sh" + +out="$(cd "$repo" && bash scripts/affected-tests.sh --replay HEAD~1..HEAD 2>/dev/null)" +RC=$? +if [[ "$RC" -eq 0 ]] && has_line "$out" "commit $alpha_commit 2 0 edit alpha hook" && + has_line "$out" " plugins/alpha/hooks/alpha-hook.test.sh (co-located with plugins/alpha/hooks/alpha-hook.sh)" && + has_line "$out" " scripts/zz-new-scan.test.sh (test-scope plugins/alpha/*)"; then + ok "--replay selects each commit against its parent with this tree's declarations" +else + fail "--replay output wrong (rc=$RC): $out" +fi + +out="$(cd "$repo" && bash scripts/affected-tests.sh --replay HEAD~1..HEAD --against HEAD 2>&1)" +RC=$? +if [[ "$RC" -eq 0 ]] && has_line "$out" "commit $alpha_commit 2 2 0 0 edit alpha hook" && + has_line "$out" " + scripts/zz-new-scan.test.sh (test-scope plugins/alpha/*)" && + has_line "$out" " - scripts/zz-old-scan.test.sh (test-scope plugins/alpha/*)" && + ! contains "$out" "alpha-hook.test.sh" && contains "$out" "replay: 1 commit(s)"; then + ok "--replay --against prints only the suites the two selectors disagree on, and a summary" +else + fail "--replay --against output wrong (rc=$RC): $out" +fi + +for args in "--replay HEAD~1..HEAD plugins/alpha/hooks/alpha-hook.sh" "--against HEAD" "--replay HEAD~1..HEAD --run"; do + # shellcheck disable=SC2086 # deliberate: each case is a word list + (cd "$repo" && bash scripts/affected-tests.sh $args >/dev/null 2>&1) + RC=$? + if [[ "$RC" -eq 2 ]]; then + ok "usage: '$args' exits 2" + else + fail "usage: '$args' should exit 2, got rc=$RC" + fi +done +rm -rf "$repo" + # --- --help reaches the actual end of the header ----------------------------- # usage() used to extract a hardcoded sed range that stopped mid-header as the # comment block grew. Pin a sentence that lives on the last header diff --git a/scripts/check-docs-only-gate.test.sh b/scripts/check-docs-only-gate.test.sh index 4adbd4a63e..847146fc94 100755 --- a/scripts/check-docs-only-gate.test.sh +++ b/scripts/check-docs-only-gate.test.sh @@ -731,12 +731,6 @@ else fail "scripts/ or lib/ suite(s) run as steps of their own:$(printf '%s' "$suite_steps" | tr '\n' ' ')" fi -if grep -qF -- '--with-always' "$live_workflow"; then - ok "the live workflow's selection carries the always-run live-tree suites" -else - fail "the live workflow's selection lost --with-always, so the live-tree suites run only on the schedule" -fi - # --- verdict ---------------------------------------------------------------- test_harness::report diff --git a/scripts/lib/gate-entry.test.sh b/scripts/lib/gate-entry.test.sh index d184727d78..3a59050fe2 100755 --- a/scripts/lib/gate-entry.test.sh +++ b/scripts/lib/gate-entry.test.sh @@ -3,6 +3,10 @@ # this process would take the suite down with it, which is the property under # test. # +# A live case scans every script under scripts/ for a hand-rolled base-ref +# predicate: +# test-scope: scripts/*.sh +# # shellcheck disable=SC2016 # child programs stay single-quoted so this shell does not expand $1 before bash -c set -uo pipefail diff --git a/scripts/lib/test-harness.test.sh b/scripts/lib/test-harness.test.sh index aca8f89f9f..41e08ee6a1 100755 --- a/scripts/lib/test-harness.test.sh +++ b/scripts/lib/test-harness.test.sh @@ -1,6 +1,9 @@ #!/usr/bin/env bash # Self-test for scripts/lib/test-harness.sh. # +# A live case reads every suite under scripts/ that sources the harness: +# test-scope: scripts/*.test.sh +# # The load-bearing property is the exit contract: a suite that recorded a # failed assertion and then called test_harness::report cannot exit 0. The # other cases pin the print format, the sourced-only guard, the last-line From abe54744ae49b74953a69224c2192fcd94e27977 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 02:17:35 -0400 Subject: [PATCH 02/18] perf(ci): resolve script-relative paths, follow Python imports, declare scanner scopes Ambiguous and structural basenames resolve through a path relative to a directory holding both files and through a bare name that is unique below its namer; a shell suite runs a same-directory sibling it names bare; Python import lines name the module they load. Suites the strace trace shows reading a directory they never name declare it in a test-scope header. Co-Authored-By: Claude Opus 5.5 (1M context) --- plugins/animation/scripts/animation.test.sh | 1 + .../check-security-binding.fixtures.test.sh | 1 + .../generate-identity-prerequisites.test.sh | 1 + .../resolve-prerequisites.fixtures.test.sh | 1 + plugins/code-metrics/scripts/dispatch.test.sh | 1 + .../scripts/tool-free-path.test.sh | 1 + .../scripts/audit-complexity.test.sh | 1 + .../scripts/audit-coverage.test.sh | 1 + .../scripts/audit-duplication.test.sh | 1 + .../audit-size/scripts/audit-size.test.sh | 1 + .../scripts/audit-type-debt.test.sh | 1 + .../skills/setup/scripts/setup-check.test.sh | 1 + .../scripts/allowed-tools-pairing.test.sh | 1 + .../hooks/zone-crossing-inject.test.sh | 1 + plugins/discovery/agents/tool-honesty.test.sh | 1 + plugins/discovery/scripts/contract.test.sh | 1 + .../skills/clean/scripts/hygiene.test.sh | 1 + .../clean/scripts/owner_registry.test.sh | 1 + .../setup/scripts/kill_switch_probe.test.sh | 1 + .../scripts/allowed-tools-pairing.test.sh | 1 + .../scripts/allowed-tools-pairing.test.sh | 1 + .../scripts/calibrate-judge.test.sh | 1 + .../plugin-eval/scripts/run-validity.test.sh | 1 + .../validate/scripts/validate-cases.test.sh | 1 + plugins/github/github.test.sh | 1 + .../guardrails/hooks/abort-boundary.test.sh | 1 + .../hooks/require-jq-notice-isolation.test.sh | 1 + .../hooks/require-jq-posture.test.sh | 1 + plugins/guardrails/hooks/run-guards.test.sh | 1 + .../scripts/inventory.test.sh | 1 + .../hooks/audit-session-id.test.sh | 1 + .../scripts/check-html-rows.test.sh | 1 + .../scripts/extract_blog_body.test.sh | 1 + .../scripts/allowed-tools-pairing.test.sh | 1 + plugins/pixel-art/scripts/backends.test.sh | 1 + plugins/planning/surface/surface.test.sh | 1 + plugins/planning/surface/test_exporters.py | 1 + plugins/planning/surface/test_round.py | 1 + plugins/planning/surface/test_schema.py | 1 + plugins/planning/surface/test_server.py | 1 + plugins/planning/surface/watch.test.sh | 1 + plugins/planning/tests/reattach-slice.test.sh | 1 + .../scripts/allowed-tools-pairing.test.sh | 1 + .../scripts/allowed-tools-pairing.test.sh | 1 + .../scripts/allowed-tools-pairing.test.sh | 1 + plugins/retro-audio/scripts/audio.test.sh | 1 + plugins/review/tests/change-set-block.test.sh | 1 + .../session-flow/scripts/save_point.test.sh | 1 + .../scripts/audit-sessions.test.sh | 1 + .../skills/babysit-prs/scripts/engine.test.sh | 1 + .../worktree/nesting-invariant-ssot.test.sh | 1 + plugins/speech/scripts/speech.test.sh | 1 + .../testing/scripts/gen-hook-filters.test.sh | 1 + .../skills/setup/scripts/setup.test.sh | 1 + .../no-hardcoded-priority-scheme.test.sh | 1 + .../adapters/gitea/list-items.test.sh | 1 + .../local-markdown/claim-integrity.test.sh | 1 + .../local-markdown/list-sub-items.test.sh | 1 + .../local-markdown/renew-lease.test.sh | 1 + .../conformance/bindings/jira.test.sh | 1 + .../bindings/local-markdown.test.sh | 1 + .../work-item-tracker.test.sh | 1 + scripts/affected-tests-no-suite.txt | 16 ---- scripts/affected-tests.sh | 75 +++++++++++-------- scripts/affected-tests.test.sh | 34 +++++++-- scripts/check-loop-lane-floor-drift.test.sh | 1 + scripts/validate-plugin-contracts.test.sh | 1 + 67 files changed, 135 insertions(+), 54 deletions(-) diff --git a/plugins/animation/scripts/animation.test.sh b/plugins/animation/scripts/animation.test.sh index 9d4c3a432a..c7d24397a6 100755 --- a/plugins/animation/scripts/animation.test.sh +++ b/plugins/animation/scripts/animation.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/animation/skills/*/SKILL.md plugins/animation/skills/*/scripts/* plugins/animation/hooks/*.sh # Contract tests for the animation scripts produce.py, pydeps.py, inkstats.py and woodcut_marks.py. # test_produce and test_pydeps need only the standard library and always run (test_pydeps skips without # pip). test_inkstats and test_woodcut_marks need numpy and opencv (../requirements.in pins them, diff --git a/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.sh b/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.sh index 7a0798ead7..7a360bdc25 100755 --- a/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.sh +++ b/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/autonomy/skills/setup/evals/fixtures/security-binding/* # Discovery wrapper: scripts/run-plugin-tests.sh finds plugins/**/*.test.sh, so # this hands off to the Node suite. SKIPs (exit 0) when Node is unavailable. set -uo pipefail diff --git a/plugins/autonomy/skills/setup/scripts/generate-identity-prerequisites.test.sh b/plugins/autonomy/skills/setup/scripts/generate-identity-prerequisites.test.sh index 2c833d05c5..d972be5eb6 100755 --- a/plugins/autonomy/skills/setup/scripts/generate-identity-prerequisites.test.sh +++ b/plugins/autonomy/skills/setup/scripts/generate-identity-prerequisites.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/autonomy/reference/routines/*.md # Unit tests for generate-identity-prerequisites.mjs. Cases are named in the # co-located manifest; this harness builds throwaway trees where needed and # drives generate / --check / drift / leaf↔emission parity. diff --git a/plugins/autonomy/skills/setup/scripts/resolve-prerequisites.fixtures.test.sh b/plugins/autonomy/skills/setup/scripts/resolve-prerequisites.fixtures.test.sh index 18c3e6561c..33ec7def83 100755 --- a/plugins/autonomy/skills/setup/scripts/resolve-prerequisites.fixtures.test.sh +++ b/plugins/autonomy/skills/setup/scripts/resolve-prerequisites.fixtures.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/autonomy/skills/setup/scripts/fixtures/prerequisite-resolution/* plugins/autonomy/generated/* # Discovery wrapper: scripts/run-plugin-tests.sh finds plugins/**/*.test.sh. set -uo pipefail diff --git a/plugins/code-metrics/scripts/dispatch.test.sh b/plugins/code-metrics/scripts/dispatch.test.sh index caa12023a1..32854c53e8 100755 --- a/plugins/code-metrics/scripts/dispatch.test.sh +++ b/plugins/code-metrics/scripts/dispatch.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* # Regression tests for dispatch.sh: scope, ladder walk, run rows, status, exit # codes. Collectors are stubbed at runtime: a temporary bin/ prepended to PATH # carries a fake `scc` that replays fixtures/tool-output/scc.json (design T13; diff --git a/plugins/code-metrics/scripts/tool-free-path.test.sh b/plugins/code-metrics/scripts/tool-free-path.test.sh index ecf7eeae68..081c660dc1 100755 --- a/plugins/code-metrics/scripts/tool-free-path.test.sh +++ b/plugins/code-metrics/scripts/tool-free-path.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/code-metrics/scripts/collectors/*.py # Regression tests for tool-free-path.sh: the excluded set is derived from # the collector ladder, the filled directory keeps those collectors off PATH, # and the resolvable-collector check fails when one is put back. diff --git a/plugins/code-metrics/skills/audit-complexity/scripts/audit-complexity.test.sh b/plugins/code-metrics/skills/audit-complexity/scripts/audit-complexity.test.sh index 2a70c36f1e..63ff47f136 100755 --- a/plugins/code-metrics/skills/audit-complexity/scripts/audit-complexity.test.sh +++ b/plugins/code-metrics/skills/audit-complexity/scripts/audit-complexity.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* # Regression tests for the audit-complexity entry point (audit-complexity.sh): # option parsing, the references it prints with their provenance, the lanes it # reports as unavailable, and exit-code passthrough from dispatch.sh. diff --git a/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh b/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh index 7132e29b4b..cd9a47ed38 100755 --- a/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh +++ b/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* # Regression tests for the audit-coverage entry point (audit-coverage.sh): # artifact discovery and the usage error for a named path that does not exist, # the join it prints for each committed artifact format diff --git a/plugins/code-metrics/skills/audit-duplication/scripts/audit-duplication.test.sh b/plugins/code-metrics/skills/audit-duplication/scripts/audit-duplication.test.sh index 293a10f201..531c9085e7 100755 --- a/plugins/code-metrics/skills/audit-duplication/scripts/audit-duplication.test.sh +++ b/plugins/code-metrics/skills/audit-duplication/scripts/audit-duplication.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* # Regression tests for the audit-duplication entry point # (audit-duplication.sh): the sanctioned-replication exclusion, the tunables it # exports for the collector adapters, option parsing, and exit codes. diff --git a/plugins/code-metrics/skills/audit-size/scripts/audit-size.test.sh b/plugins/code-metrics/skills/audit-size/scripts/audit-size.test.sh index 3824e6d78b..2c7c00171a 100755 --- a/plugins/code-metrics/skills/audit-size/scripts/audit-size.test.sh +++ b/plugins/code-metrics/skills/audit-size/scripts/audit-size.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* # Regression tests for the audit-size entry point (audit-size.sh): option parsing, # JSON versus markdown output, and exit-code passthrough from dispatch.sh. set -uo pipefail diff --git a/plugins/code-metrics/skills/audit-type-debt/scripts/audit-type-debt.test.sh b/plugins/code-metrics/skills/audit-type-debt/scripts/audit-type-debt.test.sh index 050d9b0c3b..85ffccaedf 100755 --- a/plugins/code-metrics/skills/audit-type-debt/scripts/audit-type-debt.test.sh +++ b/plugins/code-metrics/skills/audit-type-debt/scripts/audit-type-debt.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* # Regression tests for the audit-type-debt entry point (audit-type-debt.sh): # the file rows and the lane row both collectors produce, the lanes that are # not-applicable, the null reference, and what an absent tool looks like. diff --git a/plugins/code-metrics/skills/setup/scripts/setup-check.test.sh b/plugins/code-metrics/skills/setup/scripts/setup-check.test.sh index 389776e00d..f2ae072154 100755 --- a/plugins/code-metrics/skills/setup/scripts/setup-check.test.sh +++ b/plugins/code-metrics/skills/setup/scripts/setup-check.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/code-metrics/scripts/collectors/*.py # Regression tests for setup-check.sh: the layer rows, the tracked-file guard, # the resolved references, and one row per collector adapter. set -uo pipefail diff --git a/plugins/code-tidying/scripts/allowed-tools-pairing.test.sh b/plugins/code-tidying/scripts/allowed-tools-pairing.test.sh index a403cec32c..5dd21549a6 100755 --- a/plugins/code-tidying/scripts/allowed-tools-pairing.test.sh +++ b/plugins/code-tidying/scripts/allowed-tools-pairing.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/code-tidying/skills/*.md plugins/code-tidying/skills/*/scripts/* # Contract: every bundled-script `allowed-tools` grant in this plugin is PAIRED # with the invocation its skill body actually tells Claude to run. # diff --git a/plugins/context-guard/hooks/zone-crossing-inject.test.sh b/plugins/context-guard/hooks/zone-crossing-inject.test.sh index e5510267eb..4ee3da95f9 100755 --- a/plugins/context-guard/hooks/zone-crossing-inject.test.sh +++ b/plugins/context-guard/hooks/zone-crossing-inject.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/context-guard/hooks/* # Contract test for zone-crossing-inject.sh (PostToolBatch/UserPromptSubmit). # # Contract: emit ONCE per transition into a WORSE zone, splitting the report diff --git a/plugins/discovery/agents/tool-honesty.test.sh b/plugins/discovery/agents/tool-honesty.test.sh index f4e200f7a6..44c23f6d4b 100755 --- a/plugins/discovery/agents/tool-honesty.test.sh +++ b/plugins/discovery/agents/tool-honesty.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/discovery/agents/*.md # Contract test for the agent definitions in this directory. # # The defect this locks: `agents/researcher.md` carried a "Tool honesty" diff --git a/plugins/discovery/scripts/contract.test.sh b/plugins/discovery/scripts/contract.test.sh index 6f44a1ed13..3b7cfecd96 100755 --- a/plugins/discovery/scripts/contract.test.sh +++ b/plugins/discovery/scripts/contract.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/discovery/*.md plugins/discovery/*.json # Contract test for the discovery plugin's cross-file statements. # # The two sibling suites (`check-dispatch-artifact.test.sh`, diff --git a/plugins/disk-hygiene/skills/clean/scripts/hygiene.test.sh b/plugins/disk-hygiene/skills/clean/scripts/hygiene.test.sh index 1e387d70da..f79223f45f 100755 --- a/plugins/disk-hygiene/skills/clean/scripts/hygiene.test.sh +++ b/plugins/disk-hygiene/skills/clean/scripts/hygiene.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/disk-hygiene/skills/clean/SKILL.md plugins/disk-hygiene/skills/clean/reference/*.json plugins/disk-hygiene/hooks/* # Cross-platform contract wrapper for the stdlib Python test suite. set -euo pipefail diff --git a/plugins/disk-hygiene/skills/clean/scripts/owner_registry.test.sh b/plugins/disk-hygiene/skills/clean/scripts/owner_registry.test.sh index bfd30bb1e4..733c128cf7 100755 --- a/plugins/disk-hygiene/skills/clean/scripts/owner_registry.test.sh +++ b/plugins/disk-hygiene/skills/clean/scripts/owner_registry.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/disk-hygiene/*.py plugins/disk-hygiene/*.sh plugins/disk-hygiene/*.mjs plugins/disk-hygiene/*.json # Cross-platform contract wrapper for the owner-registry test suite. set -euo pipefail diff --git a/plugins/disk-hygiene/skills/setup/scripts/kill_switch_probe.test.sh b/plugins/disk-hygiene/skills/setup/scripts/kill_switch_probe.test.sh index f724cc2164..9bb6b21d71 100755 --- a/plugins/disk-hygiene/skills/setup/scripts/kill_switch_probe.test.sh +++ b/plugins/disk-hygiene/skills/setup/scripts/kill_switch_probe.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/disk-hygiene/*.py plugins/disk-hygiene/*.sh plugins/disk-hygiene/*.mjs plugins/disk-hygiene/*.json # Cross-platform contract wrapper for the kill-switch probe test suite. set -euo pipefail diff --git a/plugins/docs-hygiene/scripts/allowed-tools-pairing.test.sh b/plugins/docs-hygiene/scripts/allowed-tools-pairing.test.sh index 2c8223abc2..2be4e406c1 100755 --- a/plugins/docs-hygiene/scripts/allowed-tools-pairing.test.sh +++ b/plugins/docs-hygiene/scripts/allowed-tools-pairing.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/docs-hygiene/skills/*.md plugins/docs-hygiene/skills/*/scripts/* # Contract: every bundled-script `allowed-tools` grant in this plugin is PAIRED # with the invocation its skill body actually tells Claude to run. # diff --git a/plugins/docs-naming/scripts/allowed-tools-pairing.test.sh b/plugins/docs-naming/scripts/allowed-tools-pairing.test.sh index 952a5f992d..8d2b16184c 100755 --- a/plugins/docs-naming/scripts/allowed-tools-pairing.test.sh +++ b/plugins/docs-naming/scripts/allowed-tools-pairing.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/docs-naming/skills/*.md plugins/docs-naming/skills/*/scripts/* # Contract: every bundled-script `allowed-tools` grant in this plugin is PAIRED # with the invocation its skill body actually tells Claude to run. # diff --git a/plugins/evals/skills/plugin-eval/scripts/calibrate-judge.test.sh b/plugins/evals/skills/plugin-eval/scripts/calibrate-judge.test.sh index 231303fb36..e8d4feb62d 100755 --- a/plugins/evals/skills/plugin-eval/scripts/calibrate-judge.test.sh +++ b/plugins/evals/skills/plugin-eval/scripts/calibrate-judge.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/evals/evals/* plugins/evals/skills/plugin-eval/scripts/fixtures/calibrate-judge/* # Cross-platform wrapper for calibrate-judge.py's unittest suite, so the repo's # run-plugin-tests.sh discovery (plugins/**/*.test.sh) runs it. The interpreter # discovery follows plugins/evals/skills/validate/scripts/validate-cases.test.sh. diff --git a/plugins/evals/skills/plugin-eval/scripts/run-validity.test.sh b/plugins/evals/skills/plugin-eval/scripts/run-validity.test.sh index b6ed470ea3..06a8fcda03 100755 --- a/plugins/evals/skills/plugin-eval/scripts/run-validity.test.sh +++ b/plugins/evals/skills/plugin-eval/scripts/run-validity.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/evals/skills/plugin-eval/scripts/fixtures/run-validity/* # Cross-platform wrapper for run-validity.py's unittest suite, so the repo's # run-plugin-tests.sh discovery (plugins/**/*.test.sh) runs it. The interpreter # discovery follows plugins/evals/skills/validate/scripts/validate-cases.test.sh. diff --git a/plugins/evals/skills/validate/scripts/validate-cases.test.sh b/plugins/evals/skills/validate/scripts/validate-cases.test.sh index a5d8d622c6..3e0961ac72 100755 --- a/plugins/evals/skills/validate/scripts/validate-cases.test.sh +++ b/plugins/evals/skills/validate/scripts/validate-cases.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/evals/evals/* # Cross-platform wrapper for validate-cases.py's unittest suite, so the repo's # run-plugin-tests.sh discovery (plugins/**/*.test.sh) actually runs it. The # engine is Python and the runner step is bash-only, which is what this file diff --git a/plugins/github/github.test.sh b/plugins/github/github.test.sh index 0f958be0d3..f88e55dbf4 100755 --- a/plugins/github/github.test.sh +++ b/plugins/github/github.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/github/* # Contract test for the github plugin's durable invariants: # - D4 zero-vendored-knowledge: no baked endpoints, no shipped scope tables, no prices # - agnostic conformance: no publisher/org/tool assumptions in prose (plugin.json author is diff --git a/plugins/guardrails/hooks/abort-boundary.test.sh b/plugins/guardrails/hooks/abort-boundary.test.sh index 8f2897332a..693b4e1740 100755 --- a/plugins/guardrails/hooks/abort-boundary.test.sh +++ b/plugins/guardrails/hooks/abort-boundary.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/guardrails/hooks/* plugins/guardrails/lib/* # Contract test for hooks/abort-boundary.sh (guardrails plugin, #3528). # # Black-box where it matters: every registered hook is run as a subprocess on diff --git a/plugins/guardrails/hooks/require-jq-notice-isolation.test.sh b/plugins/guardrails/hooks/require-jq-notice-isolation.test.sh index e42df24a73..00c586de49 100755 --- a/plugins/guardrails/hooks/require-jq-notice-isolation.test.sh +++ b/plugins/guardrails/hooks/require-jq-notice-isolation.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/guardrails/hooks/*.sh # Cross-hook contract test: every guardrails hook's hook::require_jq call must # use a hook-specific notice_once key, not a key shared across the plugin. # diff --git a/plugins/guardrails/hooks/require-jq-posture.test.sh b/plugins/guardrails/hooks/require-jq-posture.test.sh index 7ad9a73696..b54010fd41 100755 --- a/plugins/guardrails/hooks/require-jq-posture.test.sh +++ b/plugins/guardrails/hooks/require-jq-posture.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/guardrails/hooks/*.sh # Contract + behavioral test for the jq-gate POSTURE split (#2146). # # Two things are proven here, and the second is the one that matters: diff --git a/plugins/guardrails/hooks/run-guards.test.sh b/plugins/guardrails/hooks/run-guards.test.sh index 442513237c..f3320f249c 100755 --- a/plugins/guardrails/hooks/run-guards.test.sh +++ b/plugins/guardrails/hooks/run-guards.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/guardrails/hooks/*.sh # Contract tests for hooks/run-guards.sh, the one-process dispatcher that runs # several guards for one hook event. The guards' own decisions are covered by # their own *.test.sh; this file covers what the dispatcher owns: stdin read diff --git a/plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh b/plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh index b64a1b52d6..933d3e0605 100755 --- a/plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh +++ b/plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/*/.mcp.json plugins/harness-config/skills/*/SKILL.md # Tests for inventory.sh (self-contained, ships with the plugin). # # The behavior under test: a location the script could not read must report diff --git a/plugins/harness-ops/hooks/audit-session-id.test.sh b/plugins/harness-ops/hooks/audit-session-id.test.sh index 99247a2885..814c851a88 100755 --- a/plugins/harness-ops/hooks/audit-session-id.test.sh +++ b/plugins/harness-ops/hooks/audit-session-id.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/harness-ops/hooks/*-audit.sh # The nine harness-ops audit rows put the payload's session_id into their # envelope `data` (additive, docs/conventions/hook-telemetry rule 1) so the # reference sink can route the line into the per-session log. One suite for diff --git a/plugins/knowledge/skills/docpage-digest/scripts/check-html-rows.test.sh b/plugins/knowledge/skills/docpage-digest/scripts/check-html-rows.test.sh index 58edcbd392..dda2c27a03 100755 --- a/plugins/knowledge/skills/docpage-digest/scripts/check-html-rows.test.sh +++ b/plugins/knowledge/skills/docpage-digest/scripts/check-html-rows.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/knowledge/skills/docpage-digest/scripts/fixtures/html-rows/* # Cross-platform wrapper so plugin-gate (plugins/**/*.test.sh) runs the # check-html-rows negative-control suite. set -euo pipefail diff --git a/plugins/knowledge/skills/docpage-digest/scripts/extract_blog_body.test.sh b/plugins/knowledge/skills/docpage-digest/scripts/extract_blog_body.test.sh index 3883d16f9a..ee6a82e3ae 100755 --- a/plugins/knowledge/skills/docpage-digest/scripts/extract_blog_body.test.sh +++ b/plugins/knowledge/skills/docpage-digest/scripts/extract_blog_body.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/knowledge/skills/docpage-digest/scripts/fixtures/blog-body.html # Cross-platform wrapper so plugin-gate (plugins/**/*.test.sh) runs the # extract_blog_body fixture suite. set -euo pipefail diff --git a/plugins/multi-agent/scripts/allowed-tools-pairing.test.sh b/plugins/multi-agent/scripts/allowed-tools-pairing.test.sh index 27f1910cc2..52bedd30c4 100755 --- a/plugins/multi-agent/scripts/allowed-tools-pairing.test.sh +++ b/plugins/multi-agent/scripts/allowed-tools-pairing.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/multi-agent/skills/*.md plugins/multi-agent/skills/*/scripts/* # Contract: every bundled-script `allowed-tools` grant in this plugin is PAIRED # with the invocation its skill body actually tells Claude to run. # diff --git a/plugins/pixel-art/scripts/backends.test.sh b/plugins/pixel-art/scripts/backends.test.sh index aed114640d..4c5c5c02f9 100755 --- a/plugins/pixel-art/scripts/backends.test.sh +++ b/plugins/pixel-art/scripts/backends.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/pixel-art/examples/* plugins/pixel-art/palettes/* # Runs every pixel-art test suite (test_*.py). set -uo pipefail diff --git a/plugins/planning/surface/surface.test.sh b/plugins/planning/surface/surface.test.sh index eaf24b6c3a..2522d547bd 100755 --- a/plugins/planning/surface/surface.test.sh +++ b/plugins/planning/surface/surface.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/planning/surface/schema/* # Hygiene checks, then the browser suites, for the interview surface. # bash surface.test.sh # Suites and files it grades: index.html, tests/ui_a.js, tests/ui_b.js, tests/ui_c.js, tests/ui_journey.js, tests/ui_live.js diff --git a/plugins/planning/surface/test_exporters.py b/plugins/planning/surface/test_exporters.py index dc51e3d01f..e07cae1e83 100644 --- a/plugins/planning/surface/test_exporters.py +++ b/plugins/planning/surface/test_exporters.py @@ -1,3 +1,4 @@ +# test-scope: plugins/planning/surface/tests/fixtures/ledger-legacy/* """Tests for the exporters and import-ledger (AC27 to AC29), driven through round.py's CLI. Sessions are built in temporary data dirs: questions.json written directly, responses.json diff --git a/plugins/planning/surface/test_round.py b/plugins/planning/surface/test_round.py index 2201505e7a..37246550f6 100644 --- a/plugins/planning/surface/test_round.py +++ b/plugins/planning/surface/test_round.py @@ -1,3 +1,4 @@ +# test-scope: plugins/planning/surface/tests/fixtures/*.json """Tests for round.py V1: schema validation, refusals and warnings, --affects, apply, archive, status --latency, the sidecar lock and the rebuild check. diff --git a/plugins/planning/surface/test_schema.py b/plugins/planning/surface/test_schema.py index 3b9bcc9c74..4233502a4c 100644 --- a/plugins/planning/surface/test_schema.py +++ b/plugins/planning/surface/test_schema.py @@ -1,3 +1,4 @@ +# test-scope: plugins/planning/surface/tests/fixtures/*.json """Tests for schema.py, the stdlib JSON Schema subset round.py validates with.""" from __future__ import annotations diff --git a/plugins/planning/surface/test_server.py b/plugins/planning/surface/test_server.py index ebbe72f8b3..dce77257e7 100644 --- a/plugins/planning/surface/test_server.py +++ b/plugins/planning/surface/test_server.py @@ -1,3 +1,4 @@ +# test-scope: plugins/planning/surface/tests/fixtures/*.json """Tests for the interview surface server and its lifecycle commands. Every class starts its own server through `round.py ensure-running --port 0` in a diff --git a/plugins/planning/surface/watch.test.sh b/plugins/planning/surface/watch.test.sh index 853bffb9a0..5aec972521 100755 --- a/plugins/planning/surface/watch.test.sh +++ b/plugins/planning/surface/watch.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/planning/surface/schema/* # Tests for watch.sh against a live server started through round.sh ensure-running. # bash watch.test.sh # Cases: curl missing (WATCH_CURL override), wrong token (exit 2 at once), a PORT that is not all diff --git a/plugins/planning/tests/reattach-slice.test.sh b/plugins/planning/tests/reattach-slice.test.sh index 39ea76aa65..662b44872b 100755 --- a/plugins/planning/tests/reattach-slice.test.sh +++ b/plugins/planning/tests/reattach-slice.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/planning/skills/*/SKILL.md # The plan, prd, and design hubs' mandatory gates stay inside the compaction # re-attach slice (#4255). The stand-in for 5,000 tokens is the first 20,000 # bytes. A phrase that also appears after that cut is a gate the re-attach can diff --git a/plugins/prototype/scripts/allowed-tools-pairing.test.sh b/plugins/prototype/scripts/allowed-tools-pairing.test.sh index f79f34a302..33b221a4fb 100755 --- a/plugins/prototype/scripts/allowed-tools-pairing.test.sh +++ b/plugins/prototype/scripts/allowed-tools-pairing.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/prototype/skills/*.md plugins/prototype/skills/*/scripts/* # Contract: every bundled-script `allowed-tools` grant in this plugin is PAIRED # with the invocation its skill body actually tells Claude to run. # diff --git a/plugins/repo-fleet-hygiene/scripts/allowed-tools-pairing.test.sh b/plugins/repo-fleet-hygiene/scripts/allowed-tools-pairing.test.sh index caf400262e..3bc7c47d7c 100755 --- a/plugins/repo-fleet-hygiene/scripts/allowed-tools-pairing.test.sh +++ b/plugins/repo-fleet-hygiene/scripts/allowed-tools-pairing.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/repo-fleet-hygiene/skills/*.md plugins/repo-fleet-hygiene/skills/*/scripts/* # Contract: every bundled-script `allowed-tools` grant in this plugin is PAIRED # with the invocation its skill body actually tells Claude to run. # diff --git a/plugins/repo-hygiene/scripts/allowed-tools-pairing.test.sh b/plugins/repo-hygiene/scripts/allowed-tools-pairing.test.sh index 8e12a28cd2..4e603d2355 100755 --- a/plugins/repo-hygiene/scripts/allowed-tools-pairing.test.sh +++ b/plugins/repo-hygiene/scripts/allowed-tools-pairing.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/repo-hygiene/skills/*.md plugins/repo-hygiene/skills/*/scripts/* # Contract: every bundled-script `allowed-tools` grant in this plugin is PAIRED # with the invocation its skill body actually tells Claude to run. # diff --git a/plugins/retro-audio/scripts/audio.test.sh b/plugins/retro-audio/scripts/audio.test.sh index 3e860248a4..b3ae53b7f3 100755 --- a/plugins/retro-audio/scripts/audio.test.sh +++ b/plugins/retro-audio/scripts/audio.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/retro-audio/examples/* # Contract tests for retro-audio WAV rendering. set -uo pipefail diff --git a/plugins/review/tests/change-set-block.test.sh b/plugins/review/tests/change-set-block.test.sh index 31db933001..b0510edc83 100755 --- a/plugins/review/tests/change-set-block.test.sh +++ b/plugins/review/tests/change-set-block.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/review/agents/*.md # Offline test of the agents' change-set block: extracts the fenced block from # agents/code-reviewer.md step 2, asserts the other reviewer agents carry the # same text, and runs it in scratch repos. It proves the block's output only, diff --git a/plugins/session-flow/scripts/save_point.test.sh b/plugins/session-flow/scripts/save_point.test.sh index 559565c082..0f4b877304 100755 --- a/plugins/session-flow/scripts/save_point.test.sh +++ b/plugins/session-flow/scripts/save_point.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/session-flow/scripts/tests/fixtures/* # Contract tests for save_point.py — delegates to the pytest suite under tests/. # # SKIPs (exit 0) when Python 3.10+ or pytest is unavailable, matching the diff --git a/plugins/session-flow/skills/audit-sessions/scripts/audit-sessions.test.sh b/plugins/session-flow/skills/audit-sessions/scripts/audit-sessions.test.sh index 77fc0087f4..9c9f141146 100755 --- a/plugins/session-flow/skills/audit-sessions/scripts/audit-sessions.test.sh +++ b/plugins/session-flow/skills/audit-sessions/scripts/audit-sessions.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/session-flow/skills/audit-sessions/scripts/tests/fixtures/* # Contract tests for the audit-sessions scripts — delegates to the pytest suite. # # SKIPs (exit 0) when Python 3.10+ or pytest is unavailable, matching the diff --git a/plugins/source-control/skills/babysit-prs/scripts/engine.test.sh b/plugins/source-control/skills/babysit-prs/scripts/engine.test.sh index db6ec4f120..689384ee07 100755 --- a/plugins/source-control/skills/babysit-prs/scripts/engine.test.sh +++ b/plugins/source-control/skills/babysit-prs/scripts/engine.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/source-control/*.md plugins/source-control/scripts/babysit-*.sh # Test entry for the babysit-prs engine: runs the stdlib-unittest suite under # tests/ (test_babysit_delta.py and siblings, covering babysit_delta.py and the # other engine modules), an optional ruff lint pass, and a bash-level check of diff --git a/plugins/source-control/skills/worktree/nesting-invariant-ssot.test.sh b/plugins/source-control/skills/worktree/nesting-invariant-ssot.test.sh index 6fe4dcd476..b34b3cea18 100755 --- a/plugins/source-control/skills/worktree/nesting-invariant-ssot.test.sh +++ b/plugins/source-control/skills/worktree/nesting-invariant-ssot.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/source-control/* # nesting-invariant-ssot.test.sh — the mechanism claim has ONE owner. # # The nesting invariant justifies a machine-wide worktree-placement rule enforced diff --git a/plugins/speech/scripts/speech.test.sh b/plugins/speech/scripts/speech.test.sh index 3f66bbf01e..df51fa071d 100755 --- a/plugins/speech/scripts/speech.test.sh +++ b/plugins/speech/scripts/speech.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/speech/skills/*/SKILL.md plugins/speech/skills/*/scripts/* plugins/speech/hooks/*.sh plugins/speech/scripts/*.json # Contract tests for the speech scripts narrate.py, assets.py, check.py and pydeps.py. # test_assets, test_speech_pydeps and most of test_narrate need only the standard library and always run (test_speech_pydeps # skips without pip). test_narrate's timing tests need numpy (../requirements.in pins it, ../requirements.txt diff --git a/plugins/testing/scripts/gen-hook-filters.test.sh b/plugins/testing/scripts/gen-hook-filters.test.sh index 48c5ceca8e..5aa7fd3afa 100755 --- a/plugins/testing/scripts/gen-hook-filters.test.sh +++ b/plugins/testing/scripts/gen-hook-filters.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/testing/skills/audit/adapters/*.yaml # Test for gen-hook-filters.sh: the shipped hooks.json is in sync with the # adapters, every row is gated by an `if`, no row matches a non-test path, no # glob repeats, and --check catches drift. diff --git a/plugins/testing/skills/setup/scripts/setup.test.sh b/plugins/testing/skills/setup/scripts/setup.test.sh index 6abfa8aa6b..768f61cc62 100755 --- a/plugins/testing/skills/setup/scripts/setup.test.sh +++ b/plugins/testing/skills/setup/scripts/setup.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/testing/skills/audit/adapters/*.yaml # Tests for setup.sh: check's four sections, lint findings, the consumer hook # entry, and an apply that writes only the docs convention file or .claude/testing.yaml. # shellcheck disable=SC2016 # fence lines in fixtures are literal text diff --git a/plugins/work-items/tests/no-hardcoded-priority-scheme.test.sh b/plugins/work-items/tests/no-hardcoded-priority-scheme.test.sh index 84c5a21647..f5ed97a383 100755 --- a/plugins/work-items/tests/no-hardcoded-priority-scheme.test.sh +++ b/plugins/work-items/tests/no-hardcoded-priority-scheme.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/work-items/*.md # Regression guard for #1253: plugin prose must never name a `pN-*` priority value, # qualified or bare. CHANGELOG.md is exempt as a historical record. # shellcheck disable=SC2016 # fixture bodies are literal prose in single quotes; expansion is never wanted diff --git a/plugins/work-items/tools/work-item-tracker/adapters/gitea/list-items.test.sh b/plugins/work-items/tools/work-item-tracker/adapters/gitea/list-items.test.sh index 564cd5b494..3d6e96dfdd 100755 --- a/plugins/work-items/tools/work-item-tracker/adapters/gitea/list-items.test.sh +++ b/plugins/work-items/tools/work-item-tracker/adapters/gitea/list-items.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/work-items/tools/work-item-tracker/adapters/gitea/* # shellcheck disable=SC2154 # FAILED/CASE_NUM initialized by the sourced helper # list-items: offline contract tests. Pagination, PR exclusion, and the per-item # blocker count are all driven through a mocked curl — no live Gitea call. diff --git a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh index 154854a999..2e877c4451 100755 --- a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh +++ b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* # Local-markdown lease/assignee-integrity behaviors that the abstract conformance # suite cannot assert (it runs against every adapter, and these two behaviors are # local-markdown-specific — the GitHub adapter has reclaim and a real assignee diff --git a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh index 0dc3658918..ee23a5a1f8 100755 --- a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh +++ b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* # End-to-end list-sub-items behavior through the core CLI against the offline # local-markdown store: direct-child enumeration with state # filtering and parent stamping, the container-scoped frontier (list-frontier diff --git a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh index d2364cc708..78f5eaf931 100755 --- a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh +++ b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* # shellcheck disable=SC2154 # FAILED/CASE_NUM initialized by the sourced helper set -uo pipefail diff --git a/plugins/work-items/tools/work-item-tracker/conformance/bindings/jira.test.sh b/plugins/work-items/tools/work-item-tracker/conformance/bindings/jira.test.sh index 70436cf335..0bf7415a89 100755 --- a/plugins/work-items/tools/work-item-tracker/conformance/bindings/jira.test.sh +++ b/plugins/work-items/tools/work-item-tracker/conformance/bindings/jira.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/work-items/tools/work-item-tracker/adapters/jira/* # shellcheck disable=SC2154 # FAILED/CASE_NUM initialized by the sourced lib # RUNS the full abstract suite against the consume-only jira adapter, once normally and # once under a PATH shim that makes gh/curl fail: every exercised path is pre-network. diff --git a/plugins/work-items/tools/work-item-tracker/conformance/bindings/local-markdown.test.sh b/plugins/work-items/tools/work-item-tracker/conformance/bindings/local-markdown.test.sh index a731ba54d1..b9276bd042 100755 --- a/plugins/work-items/tools/work-item-tracker/conformance/bindings/local-markdown.test.sh +++ b/plugins/work-items/tools/work-item-tracker/conformance/bindings/local-markdown.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* # shellcheck disable=SC2154 # FAILED/CASE_NUM initialized by the sourced lib # RUNS the full abstract conformance suite offline against the local-markdown adapter, # once normally and once under a PATH shim that makes gh/curl fail. diff --git a/plugins/work-items/tools/work-item-tracker/work-item-tracker.test.sh b/plugins/work-items/tools/work-item-tracker/work-item-tracker.test.sh index b8cf884b8f..8af3fb141a 100755 --- a/plugins/work-items/tools/work-item-tracker/work-item-tracker.test.sh +++ b/plugins/work-items/tools/work-item-tracker/work-item-tracker.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* # Tests for the core dispatcher: usage, binding resolution, capability gating, and # list-frontier derivation — all against a fake adapter (no network, no gh). set -uo pipefail diff --git a/scripts/affected-tests-no-suite.txt b/scripts/affected-tests-no-suite.txt index 4f71f15075..10559c5c09 100644 --- a/scripts/affected-tests-no-suite.txt +++ b/scripts/affected-tests-no-suite.txt @@ -156,19 +156,3 @@ plugins/testing/skills/audit/evals/judge-calibration/cases/* # sweep table. The lane covering the code that reads them is the testing # plugin's shell lane: metrics.test.sh tests --table and --rerun on stub sweeps. plugins/testing/skills/audit/evals/judge-calibration/sweep/* - -# Python modules imported by Python suites that share no stem with them, so no -# selection rule maps them. The covering lane is the affected-tests step's -# pytest run, which executes test_install_state.py and test_deep_inventory.py -# (plugin_cache_versions.py), test_overlap.py and test_native_drift.py -# (discover.py), and test_inventory.py (docs_crosscheck.py). -plugins/harness-ops/lib/plugin_cache_versions.py -plugins/harness-ops/skills/audit-native-overlap/scripts/discover.py -plugins/harness-ops/skills/inventory/scripts/docs_crosscheck.py - -# The generated session-bridge copy (scripts/shared-copies.txt). It changes only -# with its canonical, lib/session-bridge/session_bridge.py, whose shared-lib -# fan-out selects the carrying plugin's suites; the copy itself is covered by -# the shared-copies --check drift step and by the affected-tests step's pytest -# run of the planning surface's test_server.py and test_round.py, which import it. -plugins/*/surface/session_bridge.py diff --git a/scripts/affected-tests.sh b/scripts/affected-tests.sh index 0ed16a7d7d..d067f19341 100755 --- a/scripts/affected-tests.sh +++ b/scripts/affected-tests.sh @@ -83,9 +83,11 @@ # line runs or loads the file: an interpreter or process API on # the line (bash, sh, python3, node, pwsh, source, subprocess, # spawn*, exec*, ...), a path to the file rather than its bare -# name, or a shell script as the named file (another language -# has no other use for one). A chain takes at most one such -# transition and then keeps walking its new language freely. +# name, a shell script as the named file (another language +# has no other use for one), or a file in the naming file's +# own directory (a wrapper suite hands its sibling Python +# suite to a runner by bare name). A chain takes at most one +# such transition and then keeps walking its new language. # A data file (any extension that is not code) reaches code of # every language that names it, and that first step spends no # transition: data has no language of its own to stay inside. @@ -127,14 +129,16 @@ # plugin.json, marketplace.json, settings.json, hooks.json, package.json, # package-lock.json), name a specific file only when the mention RESOLVES to # it, because a bare `SKILL.md` or `config.json` says nothing about which one. -# Any mention from the file's own directory resolves. Elsewhere a bare name -# never does, and a path does when it ends in the shortest suffix of the file's -# path that no other file of that name ends in, or in the file's path relative -# to a directory that holds both files: `$SCRIPT_DIR/lib/x.sh` from a script -# beside lib/, `$PLUGIN_DIR/skills/interview/SKILL.md` or -# `$PLUGIN_ROOT/hooks/hooks.json` from inside the plugin. Shared-library -# basenames are the exception and keep the plain rule: R5's copies share a -# basename on purpose, change together with their source, and a suite naming +# Any mention from the file's own directory resolves. A bare name resolves from +# a directory above the file when no other file of that name sits below that +# directory (`FIXTURES / "questions.json"`), and never otherwise. A path +# resolves when it ends in the shortest suffix of the file's path that no other +# file of that name ends in, or in the file's path relative to a directory that +# holds both files: `$SCRIPT_DIR/lib/x.sh` from a script beside lib/, +# `$PLUGIN_DIR/skills/interview/SKILL.md` or `$PLUGIN_ROOT/hooks/hooks.json` +# from inside the plugin. A shared library's +# source and copies are the exception and keep the plain rule: R5's copies share +# a basename on purpose, change together with their source, and a suite naming # its own plugin's copy is naming the shared source. # # COMMENTS. A line that is only a comment (`#` in shell, Python and @@ -489,7 +493,7 @@ scope_table() { } declare -A AMBIGUOUS=() # basename -> 1 when two or more files carry it -declare -A SYNC_BASE=() # basename -> 1 when a shared library or its copy carries it +declare -A SYNC_MEMBER=() # path -> 1 for a shared library's source and each copy declare -a SCOPE_SUITES=() SCOPE_GLOBS=() # build_tree_index: every tracked or untracked-unignored file, listed once for # the ambiguous-name set, the reverse lookup's resolution, the declared scopes @@ -505,9 +509,9 @@ build_tree_index() { AMBIGUOUS["$b"]=1 done <"$WORK_DIR/ambiguous" for src in "${!SYNC_SRC_COPIES[@]}"; do - SYNC_BASE["${src##*/}"]=1 + SYNC_MEMBER["$src"]=1 while IFS= read -r copy; do - [[ -n "$copy" ]] && SYNC_BASE["${copy##*/}"]=1 + [[ -n "$copy" ]] && SYNC_MEMBER["$copy"]=1 done <<<"${SYNC_SRC_COPIES[$src]}" done @@ -590,7 +594,7 @@ lang_family() { # token_hits # Reduce `git grep`'s SUBSTRING hits to the mentions the rules mean. Inputs: -# the plain basenames this level asked about, the frontier paths whose names +# the frontier paths this level looks up by plain basename, those whose names # must RESOLVE (AMBIGUOUS NAMES), and the `:` grep output. Output, # one line per pair: # p<1 when a kept line runs or loads it, else 0> @@ -600,12 +604,15 @@ token_hits() { function dir_of(p) { sub(/[^\/]*$/, "", p); return p } function base_of(p) { sub(/.*\//, "", p); return p } function ends(s, t) { return length(s) >= length(t) && substr(s, length(s) - length(t) + 1) == t } - # Plain basenames: a name that is itself a path token gets the exact test; - # anything else keeps the substring test rather than losing coverage. + # Plain names: a basename that is itself a path token gets the exact test; + # anything else keeps the substring test rather than losing coverage. The + # directories that carry each name feed the same-directory arm of R4. FILENAME == plainf { if ($0 == "") next - if ($0 ~ /^[A-Za-z0-9_.-]+$/) want[$0] = 1 - else loose[$0] = 1 + b = base_of($0) + pdir[b, dir_of($0)] = 1 + if (b ~ /^[A-Za-z0-9_.-]+$/) want[b] = 1 + else loose[b] = 1 next } FILENAME == resf { @@ -637,14 +644,21 @@ token_hits() { return "" } # resolves: does path token pt, written in file namer, mean target t? Any - # mention from the directory of t does; elsewhere a bare name never does, - # and a path does when it ends in the shortest unique suffix of t, or in the - # path of t relative to a directory holding both files ($SCRIPT_DIR/lib/x.sh, + # mention from the directory of t does. A bare name does from a directory + # above t when no other file of that name sits below that directory. A path + # does when it ends in the shortest unique suffix of t, or in the path of t + # relative to a directory holding both files ($SCRIPT_DIR/lib/x.sh, # $PLUGIN_DIR/skills//SKILL.md). - function resolves(namer, pt, t, u, a) { + function resolves(namer, pt, t, u, a, i, b) { a = dir_of(namer) if (a == dir_of(t)) return 1 - if (!index(pt, "/")) return 0 + if (!index(pt, "/")) { + if (a != "" && index(t, a) != 1) return 0 + b = base_of(t) + for (i = 1; i <= nsame[b]; i++) + if (same[b, i] != t && (a == "" || index(same[b, i], a) == 1)) return 0 + return 1 + } if (!(t in usuf)) usuf[t] = uniq_suffix(t) u = usuf[t] if (u != "" && (pt == u || ends(pt, "/" u))) return 1 @@ -655,16 +669,17 @@ token_hits() { } } # runs_or_loads: R4. An interpreter or process API on the line, a path to - # the file, or a shell script as the named file. - function runs_or_loads(name, n, j) { - if (exec_line || name ~ /\.(sh|bash)$/) return 1 + # the file, a shell script as the named file, or a file of the same + # directory: a wrapper suite hands its sibling to a runner by bare name. + function runs_or_loads(path, name, n, j) { + if (exec_line || name ~ /\.(sh|bash)$/ || ((name SUBSEP dir_of(path)) in pdir)) return 1 for (j = 1; j <= n; j++) if (ends(ptok[j], "/" name)) return 1 return 0 } function keep(path, name, n) { key = path SUBSEP name if (!(key in kept)) { kept[key] = 0; order[++nkept] = key } - if (!kept[key] && runs_or_loads(name, n)) kept[key] = 1 + if (!kept[key] && runs_or_loads(path, name, n)) kept[key] = 1 } # comment_only: a whole-line comment names nothing (COMMENTS in the header), # except a shellcheck source directive and a JSDoc type import. @@ -896,7 +911,7 @@ select_for() { b="${p##*/}" printf '%s\n' "$b" >>"$WORK_DIR/patterns" if [[ "$STRUCTURAL_BASENAMES" == *" $b "* ]] || - [[ -n "${AMBIGUOUS[$b]:-}" && -z "${SYNC_BASE[$b]:-}" ]]; then + [[ -n "${AMBIGUOUS[$b]:-}" && -z "${SYNC_MEMBER[$p]:-}" ]]; then printf '%s\n' "$p" >>"$WORK_DIR/resolve" continue fi @@ -910,7 +925,7 @@ select_for() { # Any contributor that has NOT yet crossed wins: over-select. [[ "${CROSSED[$p]:-0}" == "0" ]] && PATTERN_CROSSED["$b"]=0 fi - printf '%s\n' "$b" >>"$WORK_DIR/plain" + printf '%s\n' "$p" >>"$WORK_DIR/plain" done [[ -s "$WORK_DIR/patterns" ]] || break diff --git a/scripts/affected-tests.test.sh b/scripts/affected-tests.test.sh index 2b089d08b1..ea0f494db2 100755 --- a/scripts/affected-tests.test.sh +++ b/scripts/affected-tests.test.sh @@ -1049,14 +1049,16 @@ fi # A file in another language that merely contains the name (a string, a log # message) is not a dependent and its suite is not selected: across languages # the text says nothing about a dependency unless the line runs or loads the -# file. An interpreter on the line, or a path to the file, does. -mkdir -p "$repo/eco/hop" +# file. An interpreter on the line, a path to the file, or a bare name from the +# file's own directory (a wrapper handing its sibling to a runner) does. +mkdir -p "$repo/eco/hop" "$repo/eco/elsewhere" printf 'export const c = 3;\n' >"$repo/eco/hop/origin.js" -printf 'echo "origin.js is the entry point"\n' >"$repo/eco/hop/mention.test.sh" +printf 'echo "origin.js is the entry point"\n' >"$repo/eco/elsewhere/mention.test.sh" # shellcheck disable=SC2016 # deliberate: the emitted fixture must expand these -printf 'node "$(dirname "$0")/origin.js"\n' >"$repo/eco/hop/node-runs.test.sh" +printf 'node "$ROOT/eco/hop/origin.js"\n' >"$repo/eco/elsewhere/node-runs.test.sh" # shellcheck disable=SC2016 # deliberate: the emitted fixture must expand these -printf 'cp "$SRC/eco/hop/origin.js" "$DEST"\n' >"$repo/eco/hop/path-loads.test.sh" +printf 'cp "$SRC/eco/hop/origin.js" "$DEST"\n' >"$repo/eco/elsewhere/path-loads.test.sh" +printf 'run_suite origin.js\n' >"$repo/eco/hop/wrapper.test.sh" # A .ps1 that runs the js, so the walk crosses into it once, and a shell file # that runs the .ps1: reaching ITS suite takes a second transition. printf "node origin.js\nfunction Get-Far { 2 }\n" >"$repo/eco/hop/Far.ps1" @@ -1064,9 +1066,10 @@ suite_body far >"$repo/eco/hop/Far.Tests.ps1" printf 'pwsh -File Far.ps1\n' >"$repo/eco/hop/far-runner.sh" suite_body far-runner >"$repo/eco/hop/far-runner.test.sh" run_sel "$repo" eco/hop/origin.js -if ! has_line "$OUT" eco/hop/mention.test.sh && - has_line "$OUT" eco/hop/node-runs.test.sh && - has_line "$OUT" eco/hop/path-loads.test.sh; then +if ! has_line "$OUT" eco/elsewhere/mention.test.sh && + has_line "$OUT" eco/elsewhere/node-runs.test.sh && + has_line "$OUT" eco/elsewhere/path-loads.test.sh && + has_line "$OUT" eco/hop/wrapper.test.sh; then ok "R4: another language's suite runs only where its line runs or loads the file" else fail "R4: cross-language selection wrong (rc=$RC): $OUT" @@ -1690,6 +1693,10 @@ done >"$repo/plugins/beta/skills/sc/scripts/sc.test.sh" printf 'grep -q wording probe-doc.md\n' >"$repo/plugins/beta/skills/sc/scripts/sc-bare.test.sh" printf 'cat plugins/gamma/skills/sa/reference/probe-doc.md\n' >"$repo/plugins/beta/skills/sc/scripts/sc-gamma.test.sh" + # A bare name from a directory above the file: one probe-doc.md below gamma's + # skill, two below alpha's skills/ directory. + printf 'grep -c . probe-doc.md\n' >"$repo/plugins/gamma/skills/sa/above.test.sh" + printf 'grep -c . probe-doc.md\n' >"$repo/plugins/alpha/skills/above-both.test.sh" printf '# reads plugins/alpha/skills/sa/reference/probe-doc.md\n' >"$repo/$own_a/scripts/sa-comment.test.sh" # A bare mention from the file's own directory, in a script whose suite is its sibling. printf 'grep -c . probe-doc.md\n' >"$repo/$own_a/reference/count.sh" @@ -1733,6 +1740,17 @@ if [[ "$RC" -eq 0 ]] && has_line "$OUT" plugins/beta/skills/sc/scripts/sc-gamma. else fail "ambiguous: gamma's probe-doc.md selection wrong (rc=$RC): $OUT" fi +if has_line "$OUT" plugins/gamma/skills/sa/above.test.sh; then + ok "ambiguous: a bare name from above resolves when only one such file sits below" +else + fail "ambiguous: a bare name unique below its namer did not resolve (rc=$RC): $OUT" +fi +run_sel "$repo" "$own_a/reference/probe-doc.md" +if ! has_line "$OUT" plugins/alpha/skills/above-both.test.sh; then + ok "ambiguous: a bare name from above does not resolve when two such files sit below" +else + fail "ambiguous: a bare name with two files below its namer resolved (rc=$RC): $OUT" +fi run_sel "$repo" plugins/alpha/skills/one/SKILL.md alpha_out="$OUT" alpha_rc="$RC" diff --git a/scripts/check-loop-lane-floor-drift.test.sh b/scripts/check-loop-lane-floor-drift.test.sh index c7220d1fa8..b7db12db10 100755 --- a/scripts/check-loop-lane-floor-drift.test.sh +++ b/scripts/check-loop-lane-floor-drift.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/*/reference/reader-contract.md # Black-box contract test for check-loop-lane-floor-drift.sh. # # Self-contained and cwd-independent: builds a throwaway root holding a fake diff --git a/scripts/validate-plugin-contracts.test.sh b/scripts/validate-plugin-contracts.test.sh index 80afcda3da..6fd625e271 100755 --- a/scripts/validate-plugin-contracts.test.sh +++ b/scripts/validate-plugin-contracts.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/*/retirements.yaml plugins/*/skills/*/evals/evals.json # Black-box contract test for the check-only carve-out assertions in # validate-plugin-contracts.mjs. # From 7c4ad64877e6785ffbf052072b12b49c79772281 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 02:20:08 -0400 Subject: [PATCH 03/18] style(ci): align the selector's declaration comments Co-Authored-By: Claude Opus 5.5 (1M context) --- scripts/affected-tests.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/affected-tests.sh b/scripts/affected-tests.sh index d067f19341..717f1233e5 100755 --- a/scripts/affected-tests.sh +++ b/scripts/affected-tests.sh @@ -492,7 +492,7 @@ scope_table() { }' "$2" } -declare -A AMBIGUOUS=() # basename -> 1 when two or more files carry it +declare -A AMBIGUOUS=() # basename -> 1 when two or more files carry it declare -A SYNC_MEMBER=() # path -> 1 for a shared library's source and each copy declare -a SCOPE_SUITES=() SCOPE_GLOBS=() # build_tree_index: every tracked or untracked-unignored file, listed once for From 9157b762dd700e55b53dfd06854db8c460f2470a Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 02:38:40 -0400 Subject: [PATCH 04/18] perf(ci): declare the remaining directory reads the trace found Co-Authored-By: Claude Opus 5.5 (1M context) --- plugins/actionlint/hooks/actionlint-check.test.sh | 1 + .../setup/scripts/check-prerequisite-resolution-slice.test.sh | 1 + .../skills/audit-coverage/scripts/audit-coverage.test.sh | 2 +- plugins/code-tidying/scripts/evals-fixtures.test.sh | 1 + plugins/multi-agent/tests/drift-audit.test.sh | 1 + plugins/planning/surface/surface.test.sh | 2 +- plugins/source-control/scripts/babysit-wrapper-help.test.sh | 1 + scripts/validate-plugin-contracts.test.sh | 2 +- 8 files changed, 8 insertions(+), 3 deletions(-) diff --git a/plugins/actionlint/hooks/actionlint-check.test.sh b/plugins/actionlint/hooks/actionlint-check.test.sh index b0a8694879..f164ba2bdd 100755 --- a/plugins/actionlint/hooks/actionlint-check.test.sh +++ b/plugins/actionlint/hooks/actionlint-check.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/actionlint/hooks/* plugins/actionlint/.claude-plugin/plugin.json # Black-box contract test for actionlint-check.sh (the actionlint plugin hook). # # Proves WIRING: the hook fires on .github/workflows/*.yml and *.yaml, skips diff --git a/plugins/autonomy/skills/setup/scripts/check-prerequisite-resolution-slice.test.sh b/plugins/autonomy/skills/setup/scripts/check-prerequisite-resolution-slice.test.sh index d7fa439d0b..6047ae5d70 100755 --- a/plugins/autonomy/skills/setup/scripts/check-prerequisite-resolution-slice.test.sh +++ b/plugins/autonomy/skills/setup/scripts/check-prerequisite-resolution-slice.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/autonomy/skills/setup/scripts/fixtures/prerequisite-resolution/* # Tests for the prerequisite-resolution setup slice wrappers. set -uo pipefail diff --git a/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh b/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh index cd9a47ed38..79b7f458dc 100755 --- a/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh +++ b/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* +# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/parsers/*.py plugins/code-metrics/scripts/fixtures/* # Regression tests for the audit-coverage entry point (audit-coverage.sh): # artifact discovery and the usage error for a named path that does not exist, # the join it prints for each committed artifact format diff --git a/plugins/code-tidying/scripts/evals-fixtures.test.sh b/plugins/code-tidying/scripts/evals-fixtures.test.sh index 8316246c10..55c3cfa842 100755 --- a/plugins/code-tidying/scripts/evals-fixtures.test.sh +++ b/plugins/code-tidying/scripts/evals-fixtures.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/code-tidying/evals/* # Contract: every seeded eval fixture parses and self-certifies through # change-shape.py, so a 0-score case is a skill regression. The UNPROVABLE # excerpt's exit 21 is the point of the case it feeds, not a defect. diff --git a/plugins/multi-agent/tests/drift-audit.test.sh b/plugins/multi-agent/tests/drift-audit.test.sh index 757ba51988..b6067b287c 100755 --- a/plugins/multi-agent/tests/drift-audit.test.sh +++ b/plugins/multi-agent/tests/drift-audit.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/multi-agent/agents/*.md # Discovery wrapper: scripts/run-plugin-tests.sh finds plugins/**/*.test.sh, so # this hands off to the Node suite. SKIPs (exit 0) when Node is unavailable. set -uo pipefail diff --git a/plugins/planning/surface/surface.test.sh b/plugins/planning/surface/surface.test.sh index 2522d547bd..39ba428052 100755 --- a/plugins/planning/surface/surface.test.sh +++ b/plugins/planning/surface/surface.test.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# test-scope: plugins/planning/surface/schema/* +# test-scope: plugins/planning/surface/schema/* plugins/planning/surface/tests/fixtures/* # Hygiene checks, then the browser suites, for the interview surface. # bash surface.test.sh # Suites and files it grades: index.html, tests/ui_a.js, tests/ui_b.js, tests/ui_c.js, tests/ui_journey.js, tests/ui_live.js diff --git a/plugins/source-control/scripts/babysit-wrapper-help.test.sh b/plugins/source-control/scripts/babysit-wrapper-help.test.sh index dc5a34b288..be54805e12 100755 --- a/plugins/source-control/scripts/babysit-wrapper-help.test.sh +++ b/plugins/source-control/scripts/babysit-wrapper-help.test.sh @@ -1,4 +1,5 @@ #!/usr/bin/env bash +# test-scope: plugins/source-control/scripts/* plugins/source-control/skills/babysit-prs/scripts/babysit_*.py # Regression tests for the scripts/ wrappers' --help path. # # /source-control:setup's lane-script reachability probe (#787) invokes diff --git a/scripts/validate-plugin-contracts.test.sh b/scripts/validate-plugin-contracts.test.sh index 6fd625e271..3546f82d6c 100755 --- a/scripts/validate-plugin-contracts.test.sh +++ b/scripts/validate-plugin-contracts.test.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# test-scope: plugins/*/retirements.yaml plugins/*/skills/*/evals/evals.json +# test-scope: plugins/*/retirements.yaml plugins/*/skills/*/evals/evals.json plugins/*/reference/artifact-protocol.md # Black-box contract test for the check-only carve-out assertions in # validate-plugin-contracts.mjs. # From 29fb6785fbaf19a4f7e79fd08255093c94d2c97a Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 03:01:07 -0400 Subject: [PATCH 05/18] fix(speech): drop a test-scope glob that matches no file Co-Authored-By: Claude Opus 5.5 (1M context) --- plugins/speech/scripts/speech.test.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/plugins/speech/scripts/speech.test.sh b/plugins/speech/scripts/speech.test.sh index df51fa071d..4972c987de 100755 --- a/plugins/speech/scripts/speech.test.sh +++ b/plugins/speech/scripts/speech.test.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# test-scope: plugins/speech/skills/*/SKILL.md plugins/speech/skills/*/scripts/* plugins/speech/hooks/*.sh plugins/speech/scripts/*.json +# test-scope: plugins/speech/skills/*/SKILL.md plugins/speech/hooks/*.sh plugins/speech/scripts/*.json # Contract tests for the speech scripts narrate.py, assets.py, check.py and pydeps.py. # test_assets, test_speech_pydeps and most of test_narrate need only the standard library and always run (test_speech_pydeps # skips without pip). test_narrate's timing tests need numpy (../requirements.in pins it, ../requirements.txt From 52fba8a7cca242957675853a6c04c928800cb5d3 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 03:19:33 -0400 Subject: [PATCH 06/18] perf(ci): declare test scopes in one list instead of suite headers Every change under plugins/

/ needs a version bump and a release entry, and the 29 plugins whose suites declare a scope took about 240 bumps on main in a day, so headers would conflict on nearly every merge. scripts/affected-tests- scopes.txt holds the same declarations: one ` ...` line each, an entry naming no suite failing every run, and a glob matching no file failing the run that changes the list. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/ci-runner-routing.md | 4 +- .../actionlint/hooks/actionlint-check.test.sh | 1 - plugins/animation/scripts/animation.test.sh | 1 - ...heck-prerequisite-resolution-slice.test.sh | 1 - .../check-security-binding.fixtures.test.sh | 1 - .../generate-identity-prerequisites.test.sh | 1 - .../resolve-prerequisites.fixtures.test.sh | 1 - plugins/code-metrics/scripts/dispatch.test.sh | 1 - .../scripts/tool-free-path.test.sh | 1 - .../scripts/audit-complexity.test.sh | 1 - .../scripts/audit-coverage.test.sh | 1 - .../scripts/audit-duplication.test.sh | 1 - .../audit-size/scripts/audit-size.test.sh | 1 - .../scripts/audit-type-debt.test.sh | 1 - .../skills/setup/scripts/setup-check.test.sh | 1 - .../scripts/allowed-tools-pairing.test.sh | 1 - .../scripts/evals-fixtures.test.sh | 1 - .../hooks/zone-crossing-inject.test.sh | 1 - plugins/discovery/agents/tool-honesty.test.sh | 1 - plugins/discovery/scripts/contract.test.sh | 1 - .../skills/clean/scripts/hygiene.test.sh | 1 - .../clean/scripts/owner_registry.test.sh | 1 - .../setup/scripts/kill_switch_probe.test.sh | 1 - .../scripts/allowed-tools-pairing.test.sh | 1 - .../scripts/allowed-tools-pairing.test.sh | 1 - .../scripts/calibrate-judge.test.sh | 1 - .../plugin-eval/scripts/run-validity.test.sh | 1 - .../validate/scripts/validate-cases.test.sh | 1 - plugins/github/github.test.sh | 1 - .../guardrails/hooks/abort-boundary.test.sh | 1 - .../hooks/require-jq-notice-isolation.test.sh | 1 - .../hooks/require-jq-posture.test.sh | 1 - plugins/guardrails/hooks/run-guards.test.sh | 1 - .../scripts/inventory.test.sh | 1 - .../hooks/audit-session-id.test.sh | 1 - .../scripts/check-html-rows.test.sh | 1 - .../scripts/extract_blog_body.test.sh | 1 - .../scripts/allowed-tools-pairing.test.sh | 1 - plugins/multi-agent/tests/drift-audit.test.sh | 1 - plugins/pixel-art/scripts/backends.test.sh | 1 - plugins/planning/surface/surface.test.sh | 1 - plugins/planning/surface/test_exporters.py | 1 - plugins/planning/surface/test_round.py | 1 - plugins/planning/surface/test_schema.py | 1 - plugins/planning/surface/test_server.py | 1 - plugins/planning/surface/watch.test.sh | 1 - plugins/planning/tests/reattach-slice.test.sh | 1 - .../scripts/allowed-tools-pairing.test.sh | 1 - .../scripts/allowed-tools-pairing.test.sh | 1 - .../scripts/allowed-tools-pairing.test.sh | 1 - plugins/retro-audio/scripts/audio.test.sh | 1 - plugins/review/tests/change-set-block.test.sh | 1 - .../session-flow/scripts/save_point.test.sh | 1 - .../scripts/audit-sessions.test.sh | 1 - .../scripts/babysit-wrapper-help.test.sh | 1 - .../skills/babysit-prs/scripts/engine.test.sh | 1 - .../worktree/nesting-invariant-ssot.test.sh | 1 - plugins/speech/scripts/speech.test.sh | 1 - .../testing/scripts/gen-hook-filters.test.sh | 1 - .../skills/setup/scripts/setup.test.sh | 1 - .../no-hardcoded-priority-scheme.test.sh | 1 - .../adapters/gitea/list-items.test.sh | 1 - .../local-markdown/claim-integrity.test.sh | 1 - .../local-markdown/list-sub-items.test.sh | 1 - .../local-markdown/renew-lease.test.sh | 1 - .../conformance/bindings/jira.test.sh | 1 - .../bindings/local-markdown.test.sh | 1 - .../work-item-tracker.test.sh | 1 - scripts/affected-tests-scopes.txt | 100 ++++++++++ scripts/affected-tests.sh | 176 ++++++++---------- scripts/affected-tests.test.sh | 94 +++++----- scripts/check-loop-lane-floor-drift.test.sh | 1 - scripts/lib/gate-entry.test.sh | 4 - scripts/lib/test-harness.test.sh | 3 - scripts/validate-plugin-contracts.test.sh | 1 - 75 files changed, 228 insertions(+), 222 deletions(-) create mode 100644 scripts/affected-tests-scopes.txt diff --git a/docs/ci-runner-routing.md b/docs/ci-runner-routing.md index f9c8d7d4f2..87630ef82f 100644 --- a/docs/ci-runner-routing.md +++ b/docs/ci-runner-routing.md @@ -104,8 +104,8 @@ four (four on the whole tree or an UNMAPPED file), and, per leg, whether its slice needs the animation wheels, the inventory's parser packages or the DuckDB CLI. A leg installs only those; the shfmt and DuckDB downloads are cached. -A suite that scans a directory never names the file that changed, so it -declares what it reads in a `# test-scope: ` header, and the selector's +A suite that scans a directory never names the file that changed, so +`scripts/affected-tests-scopes.txt` declares what it reads, and the selector's rule R8 selects it for any changed file matching the glob. The rules, and the `--replay` mode that shows a selector change's effect on recent main commits, are in the header of `scripts/affected-tests.sh`. diff --git a/plugins/actionlint/hooks/actionlint-check.test.sh b/plugins/actionlint/hooks/actionlint-check.test.sh index f164ba2bdd..b0a8694879 100755 --- a/plugins/actionlint/hooks/actionlint-check.test.sh +++ b/plugins/actionlint/hooks/actionlint-check.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/actionlint/hooks/* plugins/actionlint/.claude-plugin/plugin.json # Black-box contract test for actionlint-check.sh (the actionlint plugin hook). # # Proves WIRING: the hook fires on .github/workflows/*.yml and *.yaml, skips diff --git a/plugins/animation/scripts/animation.test.sh b/plugins/animation/scripts/animation.test.sh index c7d24397a6..9d4c3a432a 100755 --- a/plugins/animation/scripts/animation.test.sh +++ b/plugins/animation/scripts/animation.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/animation/skills/*/SKILL.md plugins/animation/skills/*/scripts/* plugins/animation/hooks/*.sh # Contract tests for the animation scripts produce.py, pydeps.py, inkstats.py and woodcut_marks.py. # test_produce and test_pydeps need only the standard library and always run (test_pydeps skips without # pip). test_inkstats and test_woodcut_marks need numpy and opencv (../requirements.in pins them, diff --git a/plugins/autonomy/skills/setup/scripts/check-prerequisite-resolution-slice.test.sh b/plugins/autonomy/skills/setup/scripts/check-prerequisite-resolution-slice.test.sh index 6047ae5d70..d7fa439d0b 100755 --- a/plugins/autonomy/skills/setup/scripts/check-prerequisite-resolution-slice.test.sh +++ b/plugins/autonomy/skills/setup/scripts/check-prerequisite-resolution-slice.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/autonomy/skills/setup/scripts/fixtures/prerequisite-resolution/* # Tests for the prerequisite-resolution setup slice wrappers. set -uo pipefail diff --git a/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.sh b/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.sh index 7a360bdc25..7a0798ead7 100755 --- a/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.sh +++ b/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/autonomy/skills/setup/evals/fixtures/security-binding/* # Discovery wrapper: scripts/run-plugin-tests.sh finds plugins/**/*.test.sh, so # this hands off to the Node suite. SKIPs (exit 0) when Node is unavailable. set -uo pipefail diff --git a/plugins/autonomy/skills/setup/scripts/generate-identity-prerequisites.test.sh b/plugins/autonomy/skills/setup/scripts/generate-identity-prerequisites.test.sh index d972be5eb6..2c833d05c5 100755 --- a/plugins/autonomy/skills/setup/scripts/generate-identity-prerequisites.test.sh +++ b/plugins/autonomy/skills/setup/scripts/generate-identity-prerequisites.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/autonomy/reference/routines/*.md # Unit tests for generate-identity-prerequisites.mjs. Cases are named in the # co-located manifest; this harness builds throwaway trees where needed and # drives generate / --check / drift / leaf↔emission parity. diff --git a/plugins/autonomy/skills/setup/scripts/resolve-prerequisites.fixtures.test.sh b/plugins/autonomy/skills/setup/scripts/resolve-prerequisites.fixtures.test.sh index 33ec7def83..18c3e6561c 100755 --- a/plugins/autonomy/skills/setup/scripts/resolve-prerequisites.fixtures.test.sh +++ b/plugins/autonomy/skills/setup/scripts/resolve-prerequisites.fixtures.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/autonomy/skills/setup/scripts/fixtures/prerequisite-resolution/* plugins/autonomy/generated/* # Discovery wrapper: scripts/run-plugin-tests.sh finds plugins/**/*.test.sh. set -uo pipefail diff --git a/plugins/code-metrics/scripts/dispatch.test.sh b/plugins/code-metrics/scripts/dispatch.test.sh index 32854c53e8..caa12023a1 100755 --- a/plugins/code-metrics/scripts/dispatch.test.sh +++ b/plugins/code-metrics/scripts/dispatch.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* # Regression tests for dispatch.sh: scope, ladder walk, run rows, status, exit # codes. Collectors are stubbed at runtime: a temporary bin/ prepended to PATH # carries a fake `scc` that replays fixtures/tool-output/scc.json (design T13; diff --git a/plugins/code-metrics/scripts/tool-free-path.test.sh b/plugins/code-metrics/scripts/tool-free-path.test.sh index 081c660dc1..ecf7eeae68 100755 --- a/plugins/code-metrics/scripts/tool-free-path.test.sh +++ b/plugins/code-metrics/scripts/tool-free-path.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/code-metrics/scripts/collectors/*.py # Regression tests for tool-free-path.sh: the excluded set is derived from # the collector ladder, the filled directory keeps those collectors off PATH, # and the resolvable-collector check fails when one is put back. diff --git a/plugins/code-metrics/skills/audit-complexity/scripts/audit-complexity.test.sh b/plugins/code-metrics/skills/audit-complexity/scripts/audit-complexity.test.sh index 63ff47f136..2a70c36f1e 100755 --- a/plugins/code-metrics/skills/audit-complexity/scripts/audit-complexity.test.sh +++ b/plugins/code-metrics/skills/audit-complexity/scripts/audit-complexity.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* # Regression tests for the audit-complexity entry point (audit-complexity.sh): # option parsing, the references it prints with their provenance, the lanes it # reports as unavailable, and exit-code passthrough from dispatch.sh. diff --git a/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh b/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh index 79b7f458dc..7132e29b4b 100755 --- a/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh +++ b/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/parsers/*.py plugins/code-metrics/scripts/fixtures/* # Regression tests for the audit-coverage entry point (audit-coverage.sh): # artifact discovery and the usage error for a named path that does not exist, # the join it prints for each committed artifact format diff --git a/plugins/code-metrics/skills/audit-duplication/scripts/audit-duplication.test.sh b/plugins/code-metrics/skills/audit-duplication/scripts/audit-duplication.test.sh index 531c9085e7..293a10f201 100755 --- a/plugins/code-metrics/skills/audit-duplication/scripts/audit-duplication.test.sh +++ b/plugins/code-metrics/skills/audit-duplication/scripts/audit-duplication.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* # Regression tests for the audit-duplication entry point # (audit-duplication.sh): the sanctioned-replication exclusion, the tunables it # exports for the collector adapters, option parsing, and exit codes. diff --git a/plugins/code-metrics/skills/audit-size/scripts/audit-size.test.sh b/plugins/code-metrics/skills/audit-size/scripts/audit-size.test.sh index 2c7c00171a..3824e6d78b 100755 --- a/plugins/code-metrics/skills/audit-size/scripts/audit-size.test.sh +++ b/plugins/code-metrics/skills/audit-size/scripts/audit-size.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* # Regression tests for the audit-size entry point (audit-size.sh): option parsing, # JSON versus markdown output, and exit-code passthrough from dispatch.sh. set -uo pipefail diff --git a/plugins/code-metrics/skills/audit-type-debt/scripts/audit-type-debt.test.sh b/plugins/code-metrics/skills/audit-type-debt/scripts/audit-type-debt.test.sh index 85ffccaedf..050d9b0c3b 100755 --- a/plugins/code-metrics/skills/audit-type-debt/scripts/audit-type-debt.test.sh +++ b/plugins/code-metrics/skills/audit-type-debt/scripts/audit-type-debt.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* # Regression tests for the audit-type-debt entry point (audit-type-debt.sh): # the file rows and the lane row both collectors produce, the lanes that are # not-applicable, the null reference, and what an absent tool looks like. diff --git a/plugins/code-metrics/skills/setup/scripts/setup-check.test.sh b/plugins/code-metrics/skills/setup/scripts/setup-check.test.sh index f2ae072154..389776e00d 100755 --- a/plugins/code-metrics/skills/setup/scripts/setup-check.test.sh +++ b/plugins/code-metrics/skills/setup/scripts/setup-check.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/code-metrics/scripts/collectors/*.py # Regression tests for setup-check.sh: the layer rows, the tracked-file guard, # the resolved references, and one row per collector adapter. set -uo pipefail diff --git a/plugins/code-tidying/scripts/allowed-tools-pairing.test.sh b/plugins/code-tidying/scripts/allowed-tools-pairing.test.sh index 5dd21549a6..a403cec32c 100755 --- a/plugins/code-tidying/scripts/allowed-tools-pairing.test.sh +++ b/plugins/code-tidying/scripts/allowed-tools-pairing.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/code-tidying/skills/*.md plugins/code-tidying/skills/*/scripts/* # Contract: every bundled-script `allowed-tools` grant in this plugin is PAIRED # with the invocation its skill body actually tells Claude to run. # diff --git a/plugins/code-tidying/scripts/evals-fixtures.test.sh b/plugins/code-tidying/scripts/evals-fixtures.test.sh index 8778a9d937..e88ee8c82e 100755 --- a/plugins/code-tidying/scripts/evals-fixtures.test.sh +++ b/plugins/code-tidying/scripts/evals-fixtures.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/code-tidying/evals/* # Contract: every seeded eval fixture parses and self-certifies through # change-shape.py, so a 0-score case is a skill regression. The UNPROVABLE # excerpt's exit 21 is the point of the case it feeds, not a defect. diff --git a/plugins/context-guard/hooks/zone-crossing-inject.test.sh b/plugins/context-guard/hooks/zone-crossing-inject.test.sh index 4ee3da95f9..e5510267eb 100755 --- a/plugins/context-guard/hooks/zone-crossing-inject.test.sh +++ b/plugins/context-guard/hooks/zone-crossing-inject.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/context-guard/hooks/* # Contract test for zone-crossing-inject.sh (PostToolBatch/UserPromptSubmit). # # Contract: emit ONCE per transition into a WORSE zone, splitting the report diff --git a/plugins/discovery/agents/tool-honesty.test.sh b/plugins/discovery/agents/tool-honesty.test.sh index 44c23f6d4b..f4e200f7a6 100755 --- a/plugins/discovery/agents/tool-honesty.test.sh +++ b/plugins/discovery/agents/tool-honesty.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/discovery/agents/*.md # Contract test for the agent definitions in this directory. # # The defect this locks: `agents/researcher.md` carried a "Tool honesty" diff --git a/plugins/discovery/scripts/contract.test.sh b/plugins/discovery/scripts/contract.test.sh index 3b7cfecd96..6f44a1ed13 100755 --- a/plugins/discovery/scripts/contract.test.sh +++ b/plugins/discovery/scripts/contract.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/discovery/*.md plugins/discovery/*.json # Contract test for the discovery plugin's cross-file statements. # # The two sibling suites (`check-dispatch-artifact.test.sh`, diff --git a/plugins/disk-hygiene/skills/clean/scripts/hygiene.test.sh b/plugins/disk-hygiene/skills/clean/scripts/hygiene.test.sh index f79223f45f..1e387d70da 100755 --- a/plugins/disk-hygiene/skills/clean/scripts/hygiene.test.sh +++ b/plugins/disk-hygiene/skills/clean/scripts/hygiene.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/disk-hygiene/skills/clean/SKILL.md plugins/disk-hygiene/skills/clean/reference/*.json plugins/disk-hygiene/hooks/* # Cross-platform contract wrapper for the stdlib Python test suite. set -euo pipefail diff --git a/plugins/disk-hygiene/skills/clean/scripts/owner_registry.test.sh b/plugins/disk-hygiene/skills/clean/scripts/owner_registry.test.sh index 733c128cf7..bfd30bb1e4 100755 --- a/plugins/disk-hygiene/skills/clean/scripts/owner_registry.test.sh +++ b/plugins/disk-hygiene/skills/clean/scripts/owner_registry.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/disk-hygiene/*.py plugins/disk-hygiene/*.sh plugins/disk-hygiene/*.mjs plugins/disk-hygiene/*.json # Cross-platform contract wrapper for the owner-registry test suite. set -euo pipefail diff --git a/plugins/disk-hygiene/skills/setup/scripts/kill_switch_probe.test.sh b/plugins/disk-hygiene/skills/setup/scripts/kill_switch_probe.test.sh index 9bb6b21d71..f724cc2164 100755 --- a/plugins/disk-hygiene/skills/setup/scripts/kill_switch_probe.test.sh +++ b/plugins/disk-hygiene/skills/setup/scripts/kill_switch_probe.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/disk-hygiene/*.py plugins/disk-hygiene/*.sh plugins/disk-hygiene/*.mjs plugins/disk-hygiene/*.json # Cross-platform contract wrapper for the kill-switch probe test suite. set -euo pipefail diff --git a/plugins/docs-hygiene/scripts/allowed-tools-pairing.test.sh b/plugins/docs-hygiene/scripts/allowed-tools-pairing.test.sh index 2be4e406c1..2c8223abc2 100755 --- a/plugins/docs-hygiene/scripts/allowed-tools-pairing.test.sh +++ b/plugins/docs-hygiene/scripts/allowed-tools-pairing.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/docs-hygiene/skills/*.md plugins/docs-hygiene/skills/*/scripts/* # Contract: every bundled-script `allowed-tools` grant in this plugin is PAIRED # with the invocation its skill body actually tells Claude to run. # diff --git a/plugins/docs-naming/scripts/allowed-tools-pairing.test.sh b/plugins/docs-naming/scripts/allowed-tools-pairing.test.sh index 8d2b16184c..952a5f992d 100755 --- a/plugins/docs-naming/scripts/allowed-tools-pairing.test.sh +++ b/plugins/docs-naming/scripts/allowed-tools-pairing.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/docs-naming/skills/*.md plugins/docs-naming/skills/*/scripts/* # Contract: every bundled-script `allowed-tools` grant in this plugin is PAIRED # with the invocation its skill body actually tells Claude to run. # diff --git a/plugins/evals/skills/plugin-eval/scripts/calibrate-judge.test.sh b/plugins/evals/skills/plugin-eval/scripts/calibrate-judge.test.sh index e8d4feb62d..231303fb36 100755 --- a/plugins/evals/skills/plugin-eval/scripts/calibrate-judge.test.sh +++ b/plugins/evals/skills/plugin-eval/scripts/calibrate-judge.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/evals/evals/* plugins/evals/skills/plugin-eval/scripts/fixtures/calibrate-judge/* # Cross-platform wrapper for calibrate-judge.py's unittest suite, so the repo's # run-plugin-tests.sh discovery (plugins/**/*.test.sh) runs it. The interpreter # discovery follows plugins/evals/skills/validate/scripts/validate-cases.test.sh. diff --git a/plugins/evals/skills/plugin-eval/scripts/run-validity.test.sh b/plugins/evals/skills/plugin-eval/scripts/run-validity.test.sh index 06a8fcda03..b6ed470ea3 100755 --- a/plugins/evals/skills/plugin-eval/scripts/run-validity.test.sh +++ b/plugins/evals/skills/plugin-eval/scripts/run-validity.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/evals/skills/plugin-eval/scripts/fixtures/run-validity/* # Cross-platform wrapper for run-validity.py's unittest suite, so the repo's # run-plugin-tests.sh discovery (plugins/**/*.test.sh) runs it. The interpreter # discovery follows plugins/evals/skills/validate/scripts/validate-cases.test.sh. diff --git a/plugins/evals/skills/validate/scripts/validate-cases.test.sh b/plugins/evals/skills/validate/scripts/validate-cases.test.sh index 3e0961ac72..a5d8d622c6 100755 --- a/plugins/evals/skills/validate/scripts/validate-cases.test.sh +++ b/plugins/evals/skills/validate/scripts/validate-cases.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/evals/evals/* # Cross-platform wrapper for validate-cases.py's unittest suite, so the repo's # run-plugin-tests.sh discovery (plugins/**/*.test.sh) actually runs it. The # engine is Python and the runner step is bash-only, which is what this file diff --git a/plugins/github/github.test.sh b/plugins/github/github.test.sh index f88e55dbf4..0f958be0d3 100755 --- a/plugins/github/github.test.sh +++ b/plugins/github/github.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/github/* # Contract test for the github plugin's durable invariants: # - D4 zero-vendored-knowledge: no baked endpoints, no shipped scope tables, no prices # - agnostic conformance: no publisher/org/tool assumptions in prose (plugin.json author is diff --git a/plugins/guardrails/hooks/abort-boundary.test.sh b/plugins/guardrails/hooks/abort-boundary.test.sh index 693b4e1740..8f2897332a 100755 --- a/plugins/guardrails/hooks/abort-boundary.test.sh +++ b/plugins/guardrails/hooks/abort-boundary.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/guardrails/hooks/* plugins/guardrails/lib/* # Contract test for hooks/abort-boundary.sh (guardrails plugin, #3528). # # Black-box where it matters: every registered hook is run as a subprocess on diff --git a/plugins/guardrails/hooks/require-jq-notice-isolation.test.sh b/plugins/guardrails/hooks/require-jq-notice-isolation.test.sh index 00c586de49..e42df24a73 100755 --- a/plugins/guardrails/hooks/require-jq-notice-isolation.test.sh +++ b/plugins/guardrails/hooks/require-jq-notice-isolation.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/guardrails/hooks/*.sh # Cross-hook contract test: every guardrails hook's hook::require_jq call must # use a hook-specific notice_once key, not a key shared across the plugin. # diff --git a/plugins/guardrails/hooks/require-jq-posture.test.sh b/plugins/guardrails/hooks/require-jq-posture.test.sh index b54010fd41..7ad9a73696 100755 --- a/plugins/guardrails/hooks/require-jq-posture.test.sh +++ b/plugins/guardrails/hooks/require-jq-posture.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/guardrails/hooks/*.sh # Contract + behavioral test for the jq-gate POSTURE split (#2146). # # Two things are proven here, and the second is the one that matters: diff --git a/plugins/guardrails/hooks/run-guards.test.sh b/plugins/guardrails/hooks/run-guards.test.sh index f3320f249c..442513237c 100755 --- a/plugins/guardrails/hooks/run-guards.test.sh +++ b/plugins/guardrails/hooks/run-guards.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/guardrails/hooks/*.sh # Contract tests for hooks/run-guards.sh, the one-process dispatcher that runs # several guards for one hook event. The guards' own decisions are covered by # their own *.test.sh; this file covers what the dispatcher owns: stdin read diff --git a/plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh b/plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh index 933d3e0605..b64a1b52d6 100755 --- a/plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh +++ b/plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/*/.mcp.json plugins/harness-config/skills/*/SKILL.md # Tests for inventory.sh (self-contained, ships with the plugin). # # The behavior under test: a location the script could not read must report diff --git a/plugins/harness-ops/hooks/audit-session-id.test.sh b/plugins/harness-ops/hooks/audit-session-id.test.sh index 814c851a88..99247a2885 100755 --- a/plugins/harness-ops/hooks/audit-session-id.test.sh +++ b/plugins/harness-ops/hooks/audit-session-id.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/harness-ops/hooks/*-audit.sh # The nine harness-ops audit rows put the payload's session_id into their # envelope `data` (additive, docs/conventions/hook-telemetry rule 1) so the # reference sink can route the line into the per-session log. One suite for diff --git a/plugins/knowledge/skills/docpage-digest/scripts/check-html-rows.test.sh b/plugins/knowledge/skills/docpage-digest/scripts/check-html-rows.test.sh index dda2c27a03..58edcbd392 100755 --- a/plugins/knowledge/skills/docpage-digest/scripts/check-html-rows.test.sh +++ b/plugins/knowledge/skills/docpage-digest/scripts/check-html-rows.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/knowledge/skills/docpage-digest/scripts/fixtures/html-rows/* # Cross-platform wrapper so plugin-gate (plugins/**/*.test.sh) runs the # check-html-rows negative-control suite. set -euo pipefail diff --git a/plugins/knowledge/skills/docpage-digest/scripts/extract_blog_body.test.sh b/plugins/knowledge/skills/docpage-digest/scripts/extract_blog_body.test.sh index ee6a82e3ae..3883d16f9a 100755 --- a/plugins/knowledge/skills/docpage-digest/scripts/extract_blog_body.test.sh +++ b/plugins/knowledge/skills/docpage-digest/scripts/extract_blog_body.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/knowledge/skills/docpage-digest/scripts/fixtures/blog-body.html # Cross-platform wrapper so plugin-gate (plugins/**/*.test.sh) runs the # extract_blog_body fixture suite. set -euo pipefail diff --git a/plugins/multi-agent/scripts/allowed-tools-pairing.test.sh b/plugins/multi-agent/scripts/allowed-tools-pairing.test.sh index 52bedd30c4..27f1910cc2 100755 --- a/plugins/multi-agent/scripts/allowed-tools-pairing.test.sh +++ b/plugins/multi-agent/scripts/allowed-tools-pairing.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/multi-agent/skills/*.md plugins/multi-agent/skills/*/scripts/* # Contract: every bundled-script `allowed-tools` grant in this plugin is PAIRED # with the invocation its skill body actually tells Claude to run. # diff --git a/plugins/multi-agent/tests/drift-audit.test.sh b/plugins/multi-agent/tests/drift-audit.test.sh index b6067b287c..757ba51988 100755 --- a/plugins/multi-agent/tests/drift-audit.test.sh +++ b/plugins/multi-agent/tests/drift-audit.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/multi-agent/agents/*.md # Discovery wrapper: scripts/run-plugin-tests.sh finds plugins/**/*.test.sh, so # this hands off to the Node suite. SKIPs (exit 0) when Node is unavailable. set -uo pipefail diff --git a/plugins/pixel-art/scripts/backends.test.sh b/plugins/pixel-art/scripts/backends.test.sh index 4c5c5c02f9..aed114640d 100755 --- a/plugins/pixel-art/scripts/backends.test.sh +++ b/plugins/pixel-art/scripts/backends.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/pixel-art/examples/* plugins/pixel-art/palettes/* # Runs every pixel-art test suite (test_*.py). set -uo pipefail diff --git a/plugins/planning/surface/surface.test.sh b/plugins/planning/surface/surface.test.sh index 39ba428052..eaf24b6c3a 100755 --- a/plugins/planning/surface/surface.test.sh +++ b/plugins/planning/surface/surface.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/planning/surface/schema/* plugins/planning/surface/tests/fixtures/* # Hygiene checks, then the browser suites, for the interview surface. # bash surface.test.sh # Suites and files it grades: index.html, tests/ui_a.js, tests/ui_b.js, tests/ui_c.js, tests/ui_journey.js, tests/ui_live.js diff --git a/plugins/planning/surface/test_exporters.py b/plugins/planning/surface/test_exporters.py index e07cae1e83..dc51e3d01f 100644 --- a/plugins/planning/surface/test_exporters.py +++ b/plugins/planning/surface/test_exporters.py @@ -1,4 +1,3 @@ -# test-scope: plugins/planning/surface/tests/fixtures/ledger-legacy/* """Tests for the exporters and import-ledger (AC27 to AC29), driven through round.py's CLI. Sessions are built in temporary data dirs: questions.json written directly, responses.json diff --git a/plugins/planning/surface/test_round.py b/plugins/planning/surface/test_round.py index 37246550f6..2201505e7a 100644 --- a/plugins/planning/surface/test_round.py +++ b/plugins/planning/surface/test_round.py @@ -1,4 +1,3 @@ -# test-scope: plugins/planning/surface/tests/fixtures/*.json """Tests for round.py V1: schema validation, refusals and warnings, --affects, apply, archive, status --latency, the sidecar lock and the rebuild check. diff --git a/plugins/planning/surface/test_schema.py b/plugins/planning/surface/test_schema.py index 4233502a4c..3b9bcc9c74 100644 --- a/plugins/planning/surface/test_schema.py +++ b/plugins/planning/surface/test_schema.py @@ -1,4 +1,3 @@ -# test-scope: plugins/planning/surface/tests/fixtures/*.json """Tests for schema.py, the stdlib JSON Schema subset round.py validates with.""" from __future__ import annotations diff --git a/plugins/planning/surface/test_server.py b/plugins/planning/surface/test_server.py index dce77257e7..ebbe72f8b3 100644 --- a/plugins/planning/surface/test_server.py +++ b/plugins/planning/surface/test_server.py @@ -1,4 +1,3 @@ -# test-scope: plugins/planning/surface/tests/fixtures/*.json """Tests for the interview surface server and its lifecycle commands. Every class starts its own server through `round.py ensure-running --port 0` in a diff --git a/plugins/planning/surface/watch.test.sh b/plugins/planning/surface/watch.test.sh index 5aec972521..853bffb9a0 100755 --- a/plugins/planning/surface/watch.test.sh +++ b/plugins/planning/surface/watch.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/planning/surface/schema/* # Tests for watch.sh against a live server started through round.sh ensure-running. # bash watch.test.sh # Cases: curl missing (WATCH_CURL override), wrong token (exit 2 at once), a PORT that is not all diff --git a/plugins/planning/tests/reattach-slice.test.sh b/plugins/planning/tests/reattach-slice.test.sh index 662b44872b..39ea76aa65 100755 --- a/plugins/planning/tests/reattach-slice.test.sh +++ b/plugins/planning/tests/reattach-slice.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/planning/skills/*/SKILL.md # The plan, prd, and design hubs' mandatory gates stay inside the compaction # re-attach slice (#4255). The stand-in for 5,000 tokens is the first 20,000 # bytes. A phrase that also appears after that cut is a gate the re-attach can diff --git a/plugins/prototype/scripts/allowed-tools-pairing.test.sh b/plugins/prototype/scripts/allowed-tools-pairing.test.sh index 33b221a4fb..f79f34a302 100755 --- a/plugins/prototype/scripts/allowed-tools-pairing.test.sh +++ b/plugins/prototype/scripts/allowed-tools-pairing.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/prototype/skills/*.md plugins/prototype/skills/*/scripts/* # Contract: every bundled-script `allowed-tools` grant in this plugin is PAIRED # with the invocation its skill body actually tells Claude to run. # diff --git a/plugins/repo-fleet-hygiene/scripts/allowed-tools-pairing.test.sh b/plugins/repo-fleet-hygiene/scripts/allowed-tools-pairing.test.sh index 3bc7c47d7c..caf400262e 100755 --- a/plugins/repo-fleet-hygiene/scripts/allowed-tools-pairing.test.sh +++ b/plugins/repo-fleet-hygiene/scripts/allowed-tools-pairing.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/repo-fleet-hygiene/skills/*.md plugins/repo-fleet-hygiene/skills/*/scripts/* # Contract: every bundled-script `allowed-tools` grant in this plugin is PAIRED # with the invocation its skill body actually tells Claude to run. # diff --git a/plugins/repo-hygiene/scripts/allowed-tools-pairing.test.sh b/plugins/repo-hygiene/scripts/allowed-tools-pairing.test.sh index 4e603d2355..8e12a28cd2 100755 --- a/plugins/repo-hygiene/scripts/allowed-tools-pairing.test.sh +++ b/plugins/repo-hygiene/scripts/allowed-tools-pairing.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/repo-hygiene/skills/*.md plugins/repo-hygiene/skills/*/scripts/* # Contract: every bundled-script `allowed-tools` grant in this plugin is PAIRED # with the invocation its skill body actually tells Claude to run. # diff --git a/plugins/retro-audio/scripts/audio.test.sh b/plugins/retro-audio/scripts/audio.test.sh index b3ae53b7f3..3e860248a4 100755 --- a/plugins/retro-audio/scripts/audio.test.sh +++ b/plugins/retro-audio/scripts/audio.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/retro-audio/examples/* # Contract tests for retro-audio WAV rendering. set -uo pipefail diff --git a/plugins/review/tests/change-set-block.test.sh b/plugins/review/tests/change-set-block.test.sh index b0510edc83..31db933001 100755 --- a/plugins/review/tests/change-set-block.test.sh +++ b/plugins/review/tests/change-set-block.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/review/agents/*.md # Offline test of the agents' change-set block: extracts the fenced block from # agents/code-reviewer.md step 2, asserts the other reviewer agents carry the # same text, and runs it in scratch repos. It proves the block's output only, diff --git a/plugins/session-flow/scripts/save_point.test.sh b/plugins/session-flow/scripts/save_point.test.sh index 0f4b877304..559565c082 100755 --- a/plugins/session-flow/scripts/save_point.test.sh +++ b/plugins/session-flow/scripts/save_point.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/session-flow/scripts/tests/fixtures/* # Contract tests for save_point.py — delegates to the pytest suite under tests/. # # SKIPs (exit 0) when Python 3.10+ or pytest is unavailable, matching the diff --git a/plugins/session-flow/skills/audit-sessions/scripts/audit-sessions.test.sh b/plugins/session-flow/skills/audit-sessions/scripts/audit-sessions.test.sh index 9c9f141146..77fc0087f4 100755 --- a/plugins/session-flow/skills/audit-sessions/scripts/audit-sessions.test.sh +++ b/plugins/session-flow/skills/audit-sessions/scripts/audit-sessions.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/session-flow/skills/audit-sessions/scripts/tests/fixtures/* # Contract tests for the audit-sessions scripts — delegates to the pytest suite. # # SKIPs (exit 0) when Python 3.10+ or pytest is unavailable, matching the diff --git a/plugins/source-control/scripts/babysit-wrapper-help.test.sh b/plugins/source-control/scripts/babysit-wrapper-help.test.sh index be54805e12..dc5a34b288 100755 --- a/plugins/source-control/scripts/babysit-wrapper-help.test.sh +++ b/plugins/source-control/scripts/babysit-wrapper-help.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/source-control/scripts/* plugins/source-control/skills/babysit-prs/scripts/babysit_*.py # Regression tests for the scripts/ wrappers' --help path. # # /source-control:setup's lane-script reachability probe (#787) invokes diff --git a/plugins/source-control/skills/babysit-prs/scripts/engine.test.sh b/plugins/source-control/skills/babysit-prs/scripts/engine.test.sh index 689384ee07..db6ec4f120 100755 --- a/plugins/source-control/skills/babysit-prs/scripts/engine.test.sh +++ b/plugins/source-control/skills/babysit-prs/scripts/engine.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/source-control/*.md plugins/source-control/scripts/babysit-*.sh # Test entry for the babysit-prs engine: runs the stdlib-unittest suite under # tests/ (test_babysit_delta.py and siblings, covering babysit_delta.py and the # other engine modules), an optional ruff lint pass, and a bash-level check of diff --git a/plugins/source-control/skills/worktree/nesting-invariant-ssot.test.sh b/plugins/source-control/skills/worktree/nesting-invariant-ssot.test.sh index b34b3cea18..6fe4dcd476 100755 --- a/plugins/source-control/skills/worktree/nesting-invariant-ssot.test.sh +++ b/plugins/source-control/skills/worktree/nesting-invariant-ssot.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/source-control/* # nesting-invariant-ssot.test.sh — the mechanism claim has ONE owner. # # The nesting invariant justifies a machine-wide worktree-placement rule enforced diff --git a/plugins/speech/scripts/speech.test.sh b/plugins/speech/scripts/speech.test.sh index 4972c987de..3f66bbf01e 100755 --- a/plugins/speech/scripts/speech.test.sh +++ b/plugins/speech/scripts/speech.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/speech/skills/*/SKILL.md plugins/speech/hooks/*.sh plugins/speech/scripts/*.json # Contract tests for the speech scripts narrate.py, assets.py, check.py and pydeps.py. # test_assets, test_speech_pydeps and most of test_narrate need only the standard library and always run (test_speech_pydeps # skips without pip). test_narrate's timing tests need numpy (../requirements.in pins it, ../requirements.txt diff --git a/plugins/testing/scripts/gen-hook-filters.test.sh b/plugins/testing/scripts/gen-hook-filters.test.sh index 5aa7fd3afa..48c5ceca8e 100755 --- a/plugins/testing/scripts/gen-hook-filters.test.sh +++ b/plugins/testing/scripts/gen-hook-filters.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/testing/skills/audit/adapters/*.yaml # Test for gen-hook-filters.sh: the shipped hooks.json is in sync with the # adapters, every row is gated by an `if`, no row matches a non-test path, no # glob repeats, and --check catches drift. diff --git a/plugins/testing/skills/setup/scripts/setup.test.sh b/plugins/testing/skills/setup/scripts/setup.test.sh index 768f61cc62..6abfa8aa6b 100755 --- a/plugins/testing/skills/setup/scripts/setup.test.sh +++ b/plugins/testing/skills/setup/scripts/setup.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/testing/skills/audit/adapters/*.yaml # Tests for setup.sh: check's four sections, lint findings, the consumer hook # entry, and an apply that writes only the docs convention file or .claude/testing.yaml. # shellcheck disable=SC2016 # fence lines in fixtures are literal text diff --git a/plugins/work-items/tests/no-hardcoded-priority-scheme.test.sh b/plugins/work-items/tests/no-hardcoded-priority-scheme.test.sh index f5ed97a383..84c5a21647 100755 --- a/plugins/work-items/tests/no-hardcoded-priority-scheme.test.sh +++ b/plugins/work-items/tests/no-hardcoded-priority-scheme.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/work-items/*.md # Regression guard for #1253: plugin prose must never name a `pN-*` priority value, # qualified or bare. CHANGELOG.md is exempt as a historical record. # shellcheck disable=SC2016 # fixture bodies are literal prose in single quotes; expansion is never wanted diff --git a/plugins/work-items/tools/work-item-tracker/adapters/gitea/list-items.test.sh b/plugins/work-items/tools/work-item-tracker/adapters/gitea/list-items.test.sh index 3d6e96dfdd..564cd5b494 100755 --- a/plugins/work-items/tools/work-item-tracker/adapters/gitea/list-items.test.sh +++ b/plugins/work-items/tools/work-item-tracker/adapters/gitea/list-items.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/work-items/tools/work-item-tracker/adapters/gitea/* # shellcheck disable=SC2154 # FAILED/CASE_NUM initialized by the sourced helper # list-items: offline contract tests. Pagination, PR exclusion, and the per-item # blocker count are all driven through a mocked curl — no live Gitea call. diff --git a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh index 2e877c4451..154854a999 100755 --- a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh +++ b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* # Local-markdown lease/assignee-integrity behaviors that the abstract conformance # suite cannot assert (it runs against every adapter, and these two behaviors are # local-markdown-specific — the GitHub adapter has reclaim and a real assignee diff --git a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh index ee23a5a1f8..0dc3658918 100755 --- a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh +++ b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* # End-to-end list-sub-items behavior through the core CLI against the offline # local-markdown store: direct-child enumeration with state # filtering and parent stamping, the container-scoped frontier (list-frontier diff --git a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh index 78f5eaf931..d2364cc708 100755 --- a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh +++ b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* # shellcheck disable=SC2154 # FAILED/CASE_NUM initialized by the sourced helper set -uo pipefail diff --git a/plugins/work-items/tools/work-item-tracker/conformance/bindings/jira.test.sh b/plugins/work-items/tools/work-item-tracker/conformance/bindings/jira.test.sh index 0bf7415a89..70436cf335 100755 --- a/plugins/work-items/tools/work-item-tracker/conformance/bindings/jira.test.sh +++ b/plugins/work-items/tools/work-item-tracker/conformance/bindings/jira.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/work-items/tools/work-item-tracker/adapters/jira/* # shellcheck disable=SC2154 # FAILED/CASE_NUM initialized by the sourced lib # RUNS the full abstract suite against the consume-only jira adapter, once normally and # once under a PATH shim that makes gh/curl fail: every exercised path is pre-network. diff --git a/plugins/work-items/tools/work-item-tracker/conformance/bindings/local-markdown.test.sh b/plugins/work-items/tools/work-item-tracker/conformance/bindings/local-markdown.test.sh index b9276bd042..a731ba54d1 100755 --- a/plugins/work-items/tools/work-item-tracker/conformance/bindings/local-markdown.test.sh +++ b/plugins/work-items/tools/work-item-tracker/conformance/bindings/local-markdown.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* # shellcheck disable=SC2154 # FAILED/CASE_NUM initialized by the sourced lib # RUNS the full abstract conformance suite offline against the local-markdown adapter, # once normally and once under a PATH shim that makes gh/curl fail. diff --git a/plugins/work-items/tools/work-item-tracker/work-item-tracker.test.sh b/plugins/work-items/tools/work-item-tracker/work-item-tracker.test.sh index 8af3fb141a..b8cf884b8f 100755 --- a/plugins/work-items/tools/work-item-tracker/work-item-tracker.test.sh +++ b/plugins/work-items/tools/work-item-tracker/work-item-tracker.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* # Tests for the core dispatcher: usage, binding resolution, capability gating, and # list-frontier derivation — all against a fake adapter (no network, no gh). set -uo pipefail diff --git a/scripts/affected-tests-scopes.txt b/scripts/affected-tests-scopes.txt new file mode 100644 index 0000000000..dc7f9bb8bd --- /dev/null +++ b/scripts/affected-tests-scopes.txt @@ -0,0 +1,100 @@ +# Declared test scopes, read by scripts/affected-tests.sh (rule R8). A suite +# listed here reads files it never names: it greps or globs a directory, copies +# a tree, or builds a path from parts. A changed file matching one of its globs +# selects the suite and counts as mapped. +# +# Format: one ` [...]` line per suite. Each glob is matched +# against the repo-relative path with bash pattern matching, so `*` crosses +# `/`, as in scripts/affected-tests-no-suite.txt. Blank lines and `#` comments +# are ignored. An entry naming no suite fails every selector run, and a glob +# matching no file fails the run that changes this list. +# +# Each entry was found by tracing the suite under strace: the files it opened +# or listed that no other selection rule connects to it. Declare what the suite +# reads, not the whole plugin, unless it reads the whole plugin. + +# Live-tree suites under scripts/: they scan the repository rather than fixtures. +scripts/affected-tests.test.sh scripts/affected-tests* scripts/sync-*.sh scripts/lib/sync-*.sh # runs every sync manifest and reads the selector's lists +scripts/lib/gate-entry.test.sh scripts/*.sh # scans every script for a hand-rolled base-ref predicate +scripts/lib/test-harness.test.sh scripts/*.test.sh # every harness suite must end on test_harness::report +scripts/validate-plugin-contracts.test.sh plugins/*/retirements.yaml plugins/*/skills/*/evals/evals.json plugins/*/reference/artifact-protocol.md # the validator walks every plugin +scripts/check-loop-lane-floor-drift.test.sh plugins/*/reference/reader-contract.md # compares every copy of the reader contract + +# Plugin prose and manifest scanners: grep -r or find over the plugin. +plugins/github/github.test.sh plugins/github/* +plugins/source-control/skills/worktree/nesting-invariant-ssot.test.sh plugins/source-control/* +plugins/work-items/tests/no-hardcoded-priority-scheme.test.sh plugins/work-items/*.md +plugins/discovery/scripts/contract.test.sh plugins/discovery/*.md plugins/discovery/*.json +plugins/discovery/agents/tool-honesty.test.sh plugins/discovery/agents/*.md +plugins/review/tests/change-set-block.test.sh plugins/review/agents/*.md +plugins/multi-agent/tests/drift-audit.test.sh plugins/multi-agent/agents/*.md +plugins/planning/tests/reattach-slice.test.sh plugins/planning/skills/*/SKILL.md +plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh plugins/*/.mcp.json plugins/harness-config/skills/*/SKILL.md +plugins/source-control/skills/babysit-prs/scripts/engine.test.sh plugins/source-control/*.md plugins/source-control/scripts/babysit-*.sh # test_guards.py walks the plugin + +# allowed-tools pairing: every listed skill's body and bundled scripts. +plugins/code-tidying/scripts/allowed-tools-pairing.test.sh plugins/code-tidying/skills/*.md plugins/code-tidying/skills/*/scripts/* +plugins/docs-hygiene/scripts/allowed-tools-pairing.test.sh plugins/docs-hygiene/skills/*.md plugins/docs-hygiene/skills/*/scripts/* +plugins/docs-naming/scripts/allowed-tools-pairing.test.sh plugins/docs-naming/skills/*.md plugins/docs-naming/skills/*/scripts/* +plugins/multi-agent/scripts/allowed-tools-pairing.test.sh plugins/multi-agent/skills/*.md plugins/multi-agent/skills/*/scripts/* +plugins/prototype/scripts/allowed-tools-pairing.test.sh plugins/prototype/skills/*.md plugins/prototype/skills/*/scripts/* +plugins/repo-fleet-hygiene/scripts/allowed-tools-pairing.test.sh plugins/repo-fleet-hygiene/skills/*.md plugins/repo-fleet-hygiene/skills/*/scripts/* +plugins/repo-hygiene/scripts/allowed-tools-pairing.test.sh plugins/repo-hygiene/skills/*.md plugins/repo-hygiene/skills/*/scripts/* + +# Python dependency probes: glob the skills, scripts and hooks they ship. +plugins/animation/scripts/animation.test.sh plugins/animation/skills/*/SKILL.md plugins/animation/skills/*/scripts/* plugins/animation/hooks/*.sh +plugins/speech/scripts/speech.test.sh plugins/speech/skills/*/SKILL.md plugins/speech/hooks/*.sh plugins/speech/scripts/*.json + +# Registries and dispatchers that enumerate a directory of modules or adapters. +plugins/code-metrics/scripts/dispatch.test.sh plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* +plugins/code-metrics/scripts/tool-free-path.test.sh plugins/code-metrics/scripts/collectors/*.py +plugins/code-metrics/skills/audit-complexity/scripts/audit-complexity.test.sh plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* +plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/parsers/*.py plugins/code-metrics/scripts/fixtures/* +plugins/code-metrics/skills/audit-duplication/scripts/audit-duplication.test.sh plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* +plugins/code-metrics/skills/audit-size/scripts/audit-size.test.sh plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* +plugins/code-metrics/skills/audit-type-debt/scripts/audit-type-debt.test.sh plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* +plugins/code-metrics/skills/setup/scripts/setup-check.test.sh plugins/code-metrics/scripts/collectors/*.py +plugins/testing/scripts/gen-hook-filters.test.sh plugins/testing/skills/audit/adapters/*.yaml +plugins/testing/skills/setup/scripts/setup.test.sh plugins/testing/skills/audit/adapters/*.yaml +plugins/work-items/tools/work-item-tracker/work-item-tracker.test.sh plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* +plugins/work-items/tools/work-item-tracker/conformance/bindings/local-markdown.test.sh plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* +plugins/work-items/tools/work-item-tracker/conformance/bindings/jira.test.sh plugins/work-items/tools/work-item-tracker/adapters/jira/* +plugins/work-items/tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* +plugins/work-items/tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* +plugins/work-items/tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* +plugins/work-items/tools/work-item-tracker/adapters/gitea/list-items.test.sh plugins/work-items/tools/work-item-tracker/adapters/gitea/* +plugins/harness-ops/hooks/audit-session-id.test.sh plugins/harness-ops/hooks/*-audit.sh +plugins/source-control/scripts/babysit-wrapper-help.test.sh plugins/source-control/scripts/* plugins/source-control/skills/babysit-prs/scripts/babysit_*.py + +# Suites that copy their plugin and run its hooks from the copy. +plugins/guardrails/hooks/abort-boundary.test.sh plugins/guardrails/hooks/* plugins/guardrails/lib/* +plugins/guardrails/hooks/run-guards.test.sh plugins/guardrails/hooks/*.sh +plugins/guardrails/hooks/require-jq-posture.test.sh plugins/guardrails/hooks/*.sh +plugins/guardrails/hooks/require-jq-notice-isolation.test.sh plugins/guardrails/hooks/*.sh +plugins/context-guard/hooks/zone-crossing-inject.test.sh plugins/context-guard/hooks/* +plugins/actionlint/hooks/actionlint-check.test.sh plugins/actionlint/hooks/* plugins/actionlint/.claude-plugin/plugin.json +plugins/disk-hygiene/skills/setup/scripts/kill_switch_probe.test.sh plugins/disk-hygiene/*.py plugins/disk-hygiene/*.sh plugins/disk-hygiene/*.mjs plugins/disk-hygiene/*.json +plugins/disk-hygiene/skills/clean/scripts/owner_registry.test.sh plugins/disk-hygiene/*.py plugins/disk-hygiene/*.sh plugins/disk-hygiene/*.mjs plugins/disk-hygiene/*.json +plugins/disk-hygiene/skills/clean/scripts/hygiene.test.sh plugins/disk-hygiene/skills/clean/SKILL.md plugins/disk-hygiene/skills/clean/reference/*.json plugins/disk-hygiene/hooks/* + +# Fixture directories a suite enumerates or reads by a built path. +plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.sh plugins/autonomy/skills/setup/evals/fixtures/security-binding/* +plugins/autonomy/skills/setup/scripts/resolve-prerequisites.fixtures.test.sh plugins/autonomy/skills/setup/scripts/fixtures/prerequisite-resolution/* plugins/autonomy/generated/* +plugins/autonomy/skills/setup/scripts/check-prerequisite-resolution-slice.test.sh plugins/autonomy/skills/setup/scripts/fixtures/prerequisite-resolution/* +plugins/autonomy/skills/setup/scripts/generate-identity-prerequisites.test.sh plugins/autonomy/reference/routines/*.md +plugins/code-tidying/scripts/evals-fixtures.test.sh plugins/code-tidying/evals/* +plugins/evals/skills/validate/scripts/validate-cases.test.sh plugins/evals/evals/* +plugins/evals/skills/plugin-eval/scripts/calibrate-judge.test.sh plugins/evals/evals/* plugins/evals/skills/plugin-eval/scripts/fixtures/calibrate-judge/* +plugins/evals/skills/plugin-eval/scripts/run-validity.test.sh plugins/evals/skills/plugin-eval/scripts/fixtures/run-validity/* +plugins/knowledge/skills/docpage-digest/scripts/check-html-rows.test.sh plugins/knowledge/skills/docpage-digest/scripts/fixtures/html-rows/* +plugins/knowledge/skills/docpage-digest/scripts/extract_blog_body.test.sh plugins/knowledge/skills/docpage-digest/scripts/fixtures/blog-body.html +plugins/pixel-art/scripts/backends.test.sh plugins/pixel-art/examples/* plugins/pixel-art/palettes/* +plugins/retro-audio/scripts/audio.test.sh plugins/retro-audio/examples/* +plugins/planning/surface/surface.test.sh plugins/planning/surface/schema/* plugins/planning/surface/tests/fixtures/* +plugins/planning/surface/watch.test.sh plugins/planning/surface/schema/* +plugins/planning/surface/test_exporters.py plugins/planning/surface/tests/fixtures/ledger-legacy/* +plugins/planning/surface/test_round.py plugins/planning/surface/tests/fixtures/*.json +plugins/planning/surface/test_schema.py plugins/planning/surface/tests/fixtures/*.json +plugins/planning/surface/test_server.py plugins/planning/surface/tests/fixtures/*.json +plugins/session-flow/scripts/save_point.test.sh plugins/session-flow/scripts/tests/fixtures/* +plugins/session-flow/skills/audit-sessions/scripts/audit-sessions.test.sh plugins/session-flow/skills/audit-sessions/scripts/tests/fixtures/* diff --git a/scripts/affected-tests.sh b/scripts/affected-tests.sh index 717f1233e5..6e7cf6bdc4 100755 --- a/scripts/affected-tests.sh +++ b/scripts/affected-tests.sh @@ -24,7 +24,7 @@ # the selector at (see REPLAY) # # --with-always is accepted and changes nothing: the suites that assert against -# the live tree declare what they read with a test-scope header (R8) instead. +# the live tree declare what they read in scripts/affected-tests-scopes.txt (R8). # # Exit: 0 selected (or nothing to do); 1 an unmapped changed file, or a failing # suite under --run; 2 usage or a broken derivation; 3 --run ran every shell @@ -102,16 +102,16 @@ # plugin-contract validator's suite, which bans vendor names # across that directory without naming any file in it. # R8 declared scope a suite that enumerates a directory of the live tree -# (a grep -r, a find, a glob over a plugin or scripts/) never -# names the files it reads, so it declares them in its header: -# # test-scope: plugins/github/*.md -# (`//` for Node). A changed file matching a glob selects the -# suite and counts as mapped. The globs use the dialect of the -# no-suite list: matched against the repo-relative path, `*` -# crosses `/`. Only the leading comment block is read, so a -# fixture line further down can never declare one; a -# changed suite whose declaration sits below that block, or -# names a glob matching no file, fails the run (exit 2). +# (a grep -r, a find, a glob over a plugin or scripts/), or +# builds a path from parts, never names the files it reads, +# so scripts/affected-tests-scopes.txt declares them, one +# ` ...` line per suite: +# plugins/github/github.test.sh plugins/github/* +# A changed file matching a glob selects the suite and counts +# as mapped. The globs use the dialect of the no-suite list: +# matched against the repo-relative path, `*` crosses `/`. +# An entry naming no suite fails every run (exit 2), and a +# glob matching no file fails the run that changes the list. # # MATCHING. One file NAMES another when the basename stands in a line as a WHOLE # PATH TOKEN: bounded on both sides by a character outside [A-Za-z0-9_.-]. `/` @@ -158,21 +158,19 @@ # REPLAY. `--replay ` selects every first-parent commit of # (`git rev-list --first-parent `) against its parent, the squash-merged # pull request's net diff, in a scratch clone checked out at that commit, with -# THIS script's rules, no-suite list and test-scope declarations, so it answers -# "what would this selector have run for those pull requests". It prints one +# THIS script's rules, no-suite list and scopes list, so it answers "what would +# this selector have run for those pull requests". It prints one # `commit ` line per commit, an indented # `unmapped ` line per unmapped file and one indented # ` ()` line per suite. With `--against ` it also runs the # selector at , with 's own lists, on the same tree, and prints only # the suites that differ (`+` this script only, `-` only, each with its # reason) after a `commit ` line -# and its `unmapped` lines, -# so a selector change shows its blast radius. Both sides run with -# --allow-unmapped; a summary on stderr counts suites per commit (p50, p95, -# max, total) and the commits with an unmapped file on each side. The replay -# points each run at the scratch clone with AFFECTED_TESTS_ROOT and hands it a -# test-scope table with AFFECTED_TESTS_SCOPES; AFFECTED_TESTS_NO_SUITE names -# the no-suite list. +# and its `unmapped` lines, so a selector change shows its blast radius. Both +# sides run with --allow-unmapped; a summary on stderr counts suites per commit +# (p50, p95, max, total) and the commits with an unmapped file on each side. +# Each run is pointed at the scratch clone with AFFECTED_TESTS_ROOT, and at the +# lists with AFFECTED_TESTS_NO_SUITE and AFFECTED_TESTS_SCOPES. # # MECHANICALLY the reverse lookup is two stages. `git grep -F` finds the # candidate LINES with the substring test, which keeps git's fixed-string fast @@ -192,6 +190,7 @@ cd "${AFFECTED_TESTS_ROOT:-$SCRIPT_DIR/..}" || exit 2 . "$SCRIPT_DIR/lib/read-list.sh" || exit 2 NO_SUITE_LIST="${AFFECTED_TESTS_NO_SUITE:-scripts/affected-tests-no-suite.txt}" +SCOPES_LIST="${AFFECTED_TESTS_SCOPES:-scripts/affected-tests-scopes.txt}" # Basenames that name a repository-wide role, reached only through a resolved # mention (AMBIGUOUS NAMES in the header), however few files carry them today. @@ -470,28 +469,6 @@ build_sync_map() { # Tree index: every file, the ambiguous basenames, the declared scopes # --------------------------------------------------------------------------- -# scope_table -> `\t` for every R8 declaration -# in the leading comment block of each suite the list names (paths relative to -# ). Reading stops at the first line that is neither blank nor a comment. -scope_table() { - awk -v root="$1" ' - { b = $0; sub(/.*\//, "", b) } - $0 ~ /\.(test\.(sh|js|mjs)|Tests\.ps1)$/ || b ~ /^test_.*\.py$/ { - f = root "/" $0 - while ((getline line < f) > 0) { - sub(/\r$/, "", line) - if (line ~ /^[ \t]*$/) continue - if (line !~ /^[ \t]*(#|\/\/|\/\*|\*)/) break - if (line ~ /^[ \t]*(#|\/\/)[ \t]*test-scope:/) { - sub(/^[^:]*:/, "", line) - n = split(line, g, /[ \t]+/) - for (i = 1; i <= n; i++) if (g[i] != "") print $0 "\t" g[i] - } - } - close(f) - }' "$2" -} - declare -A AMBIGUOUS=() # basename -> 1 when two or more files carry it declare -A SYNC_MEMBER=() # path -> 1 for a shared library's source and each copy declare -a SCOPE_SUITES=() SCOPE_GLOBS=() @@ -499,7 +476,7 @@ declare -a SCOPE_SUITES=() SCOPE_GLOBS=() # the ambiguous-name set, the reverse lookup's resolution, the declared scopes # and the unmapped corpora. Fatal on a failed listing: a short list under-selects. build_tree_index() { - local b src copy suite glob + local b src copy suite if ! git ls-files --cached --others --exclude-standard >"$WORK_DIR/all-files" || ! awk '{ sub(/.*\//, ""); if (++count[$0] == 2) print }' "$WORK_DIR/all-files" >"$WORK_DIR/ambiguous"; then echo "error: listing the tree failed." >&2 @@ -525,19 +502,28 @@ build_tree_index() { exit 2 fi - # R8. A replay supplies the table of the tree it was started from, since the - # commits it checks out predate the declarations. - if [[ -n "${AFFECTED_TESTS_SCOPES:-}" ]]; then - cp "$AFFECTED_TESTS_SCOPES" "$WORK_DIR/scopes" || exit 2 - elif ! scope_table . "$WORK_DIR/all-files" >"$WORK_DIR/scopes"; then - echo "error: reading the test-scope declarations failed." >&2 + # R8, one ` ...` line per suite. An entry naming no suite fails + # the run: a declaration must not outlive what it declares. A replay hands in + # the list of the tree it started from, whose suites an older commit may lack. + local -a entries=() words=() + local entry i + if [[ ! -f "$SCOPES_LIST" ]]; then + echo "error: missing $SCOPES_LIST, the declared test scopes (R8)." >&2 exit 2 fi - while IFS=$'\t' read -r suite glob; do - [[ -n "$glob" ]] || continue - SCOPE_SUITES+=("$suite") - SCOPE_GLOBS+=("$glob") - done <"$WORK_DIR/scopes" + read_list::into entries "$SCOPES_LIST" --comments inline || exit 2 + for entry in ${entries[@]+"${entries[@]}"}; do + read -r -a words <<<"$entry" + suite="${words[0]}" + if [[ -z "${AFFECTED_TESTS_SCOPES:-}" ]] && { ! is_suite_path "$suite" || [[ ! -f "$suite" ]]; }; then + echo "error: $SCOPES_LIST names '$suite', which is not a suite; update or remove the entry." >&2 + exit 2 + fi + for ((i = 1; i < ${#words[@]}; i++)); do + SCOPE_SUITES+=("$suite") + SCOPE_GLOBS+=("${words[i]}") + done + done } # --------------------------------------------------------------------------- @@ -1005,32 +991,31 @@ select_for() { done } -# check_declarations -> exit 2 when a test-scope line of the -# suite sits below its leading comment block, where R8 never reads it, or when -# one of its globs matches no file of the tree. Either mistake silently drops -# the suite from the changes it reads, so the pull request that makes it fails. -check_declarations() { - local suite="$1" i f hit - if ! awk 'BEGIN { hdr = 1 } { sub(/\r$/, "") } - hdr && !/^[ \t]*$/ && !/^[ \t]*(#|\/\/|\/\*|\*)/ { hdr = 0 } - !hdr && /^[ \t]*(#|\/\/)[ \t]*test-scope:/ { printf "%s:%d\n", FILENAME, FNR; bad = 1 } - END { exit bad }' "$suite" >"$WORK_DIR/misplaced"; then - echo "error: a test-scope declaration below the leading comment block is never read:" >&2 - sed 's/^/ - /' "$WORK_DIR/misplaced" >&2 - exit 2 - fi - for i in "${!SCOPE_GLOBS[@]}"; do - [[ "${SCOPE_SUITES[i]}" == "$suite" ]] || continue - hit=0 - while IFS= read -r f; do - # shellcheck disable=SC2053 # the right-hand side is a glob pattern by design. - [[ "$f" == ${SCOPE_GLOBS[i]} ]] && hit=1 && break - done <"$WORK_DIR/all-files" - if [[ "$hit" -eq 0 ]]; then - echo "error: $suite declares test-scope ${SCOPE_GLOBS[i]}, which matches no file of the tree." >&2 - exit 2 - fi - done +# check_scope_globs -> exit 2 when a declared glob matches no file of the tree: +# it declares nothing, so the suite misses the changes it reads. Run when the +# scopes list itself changes, which is when a glob is written or goes stale. +check_scope_globs() { + [[ ${#SCOPE_GLOBS[@]} -gt 0 ]] || return 0 + printf '%s\n' "${SCOPE_GLOBS[@]}" | awk ' + # The glob dialect of the lists: `*` any run of characters, `/` included. + function to_regex(g, r, i, c) { + r = "^" + for (i = 1; i <= length(g); i++) { + c = substr(g, i, 1) + if (c == "*") r = r ".*" + else if (c == "?") r = r "." + else if (index(".+(){}|^$\\", c)) r = r "\\" c + else r = r c + } + return r "$" + } + FNR == NR { if (!($0 in want)) { want[$0] = to_regex($0); order[++n] = $0 } next } + { for (g in want) if (!(g in hit) && $0 ~ want[g]) hit[g] = 1 } + END { for (i = 1; i <= n; i++) if (!(order[i] in hit)) { print order[i]; bad = 1 } exit bad } + ' - "$WORK_DIR/all-files" >"$WORK_DIR/stale-globs" && return 0 + echo "error: $SCOPES_LIST declares globs that match no file of the tree:" >&2 + sed 's/^/ - /' "$WORK_DIR/stale-globs" >&2 + exit 2 } # select_scoped -> R8: add every suite whose declared test-scope @@ -1137,13 +1122,12 @@ fi # Replay # --------------------------------------------------------------------------- -# replay_select +# replay_select # [...] -- ... # One selection in the replay tree, through a fresh process. Writes # .sel (`\t`) and .unmapped, and fails # loud on any exit but 0, because a broken selection counted as an empty one -# would understate the side it ran for. An empty lets the -# selector read the tree's own declarations. +# would understate the side it ran for. replay_select() { local out="$1" tree="$2" sel="$3" list="$4" scopes="$5" rc=0 shift 5 @@ -1172,12 +1156,9 @@ run_replay() { echo "error: '$replay_range' holds no commits to replay." >&2 exit 2 fi - # This tree's rules travel with the replay: its no-suite list and its - # test-scope table, read once here. - build_sync_map - build_tree_index + # This tree's rules travel with the replay: its no-suite and scopes lists. cp "$NO_SUITE_LIST" "$WORK_DIR/replay-no-suite" || exit 2 - cp "$WORK_DIR/scopes" "$WORK_DIR/replay-scopes" || exit 2 + cp "$SCOPES_LIST" "$WORK_DIR/replay-scopes" || exit 2 if ! git clone -q --shared --no-checkout . "$tree"; then echo "error: could not make the scratch clone for the replay." >&2 exit 2 @@ -1186,7 +1167,7 @@ run_replay() { # 's selector, its scripts/lib/ and its lists, pointed at the replay # tree through AFFECTED_TESTS_ROOT. A selector without that override gets # its one `cd` line rewritten; one with neither form cannot be pointed. - mkdir -p "$against/scripts" "$against/tree" || exit 2 + mkdir -p "$against/scripts" || exit 2 if ! git show "$against_ref:scripts/affected-tests.sh" >"$against/selector.orig" || ! git archive "$against_ref" scripts/lib | tar -x -C "$against" || ! git show "$against_ref:scripts/affected-tests-no-suite.txt" >"$against/no-suite.txt"; then @@ -1202,15 +1183,9 @@ run_replay() { grep -q -- '--with-always)' "$against/scripts/affected-tests.sh" && against_flags+=(--with-always) git show "$against_ref:scripts/affected-tests-always.txt" >"$against/always.txt" 2>/dev/null || rm -f "$against/always.txt" - # A selector that reads test-scope declarations gets 's own table. - if grep -q 'AFFECTED_TESTS_SCOPES' "$against/scripts/affected-tests.sh"; then - if ! git archive "$against_ref" | tar -x -C "$against/tree" || - ! git ls-tree -r --name-only "$against_ref" >"$against/files" || - ! scope_table "$against/tree" "$against/files" >"$against/scopes"; then - echo "error: could not read the test-scope declarations at '$against_ref'." >&2 - exit 2 - fi - against_scopes="$against/scopes" + # A selector that reads a scopes list gets 's own. + if git show "$against_ref:scripts/affected-tests-scopes.txt" >"$against/scopes.txt" 2>/dev/null; then + against_scopes="$against/scopes.txt" fi fi @@ -1321,10 +1296,7 @@ build_tree_index declare -a NO_SUITE_FILES=() for f in "${changed[@]}"; do [[ -n "$f" ]] || continue - # A replay's table comes from another tree, so only a tree's own is checked. - if [[ -z "${AFFECTED_TESTS_SCOPES:-}" && -f "$f" ]] && is_suite_path "$f"; then - check_declarations "$f" - fi + [[ "$f" == "$SCOPES_LIST" ]] && check_scope_globs select_for "$f" select_scoped "$f" if [[ "$SEED_HITS" -eq 0 ]]; then diff --git a/scripts/affected-tests.test.sh b/scripts/affected-tests.test.sh index 3aab98a89b..6fcd872113 100755 --- a/scripts/affected-tests.test.sh +++ b/scripts/affected-tests.test.sh @@ -8,9 +8,6 @@ # against the LIVE repo — the derived shared-lib copy set and the real no-suite # list — because a synthetic fixture cannot show that the derivation still # tracks reality, which is the whole failure mode this tool exists to avoid. -# -# The live cases run every sync manifest and read the selector's own lists: -# test-scope: scripts/affected-tests* scripts/sync-*.sh scripts/lib/sync-*.sh set -uo pipefail TMP_ROOT="$(mktemp -d)" @@ -76,6 +73,8 @@ mk_repo() { # mkdir -p "$dir/lib" "$dir/plugins/alpha/hooks" "$dir/plugins/beta/hooks" cp "$NO_SUITE" "$dir/scripts/affected-tests-no-suite.txt" + # The live scopes list names suites this fixture does not have. + printf '# fixture scopes\n' >"$dir/scripts/affected-tests-scopes.txt" # --jobs N delegates to the SIBLING run-plugin-tests.sh rather than spawning # anything itself, so the fixture carries that sibling too. Its serial @@ -1525,26 +1524,26 @@ else fi fi -# --- R8: a declared test-scope selects the suite that scans a directory ------ +# --- R8: a declared scope selects the suite that scans a directory ----------- # A suite that greps or globs a directory never spells the files it reads, so -# it declares them in its leading comment block, and a matching change selects +# scripts/affected-tests-scopes.txt declares them, and a matching change selects # it and counts as mapped. Pinned: the glob crosses `/`, the plugin's other -# suites stay out, a declaration below the first code line is not read, the -# Node `//` form works, and a plugin file nothing names or declares is still -# UNMAPPED. +# suites stay out, an inline comment ends the entry, and a plugin file nothing +# names or declares is still UNMAPPED. mk_repo repo mkdir -p "$repo/plugins/alpha/skills/one" "$repo/plugins/alpha/tests" printf -- '---\nname: one\n---\n' >"$repo/plugins/alpha/skills/one/SKILL.md" printf 'kind: probe\n' >"$repo/plugins/alpha/skills/one/probe.yaml" -{ - printf '#!/usr/bin/env bash\n# Scans every skill body.\n# test-scope: plugins/alpha/skills/*.md\n' - printf '# test-scope: plugins/alpha/*.yaml\n\nset -u\n# test-scope: plugins/beta/*\n' -} >"$repo/plugins/alpha/tests/scan.test.sh" -printf '// test-scope: plugins/alpha/skills/*/SKILL.md\nimport test from "node:test";\n' \ - >"$repo/plugins/alpha/tests/scan.test.mjs" +suite_body alpha-scan >"$repo/plugins/alpha/tests/scan.test.sh" +printf 'import test from "node:test";\n' >"$repo/plugins/alpha/tests/scan.test.mjs" printf 'import unittest\n' >"$repo/plugins/alpha/tests/test_scan.py" printf 'echo orphan\n' >"$repo/plugins/alpha/zzorphan-plugin.sh" -git_test_config "$repo" add plugins >/dev/null +{ + printf '# fixture scopes\n' + printf 'plugins/alpha/tests/scan.test.sh plugins/alpha/skills/*.md plugins/alpha/*.yaml # scans skill bodies\n' + printf 'plugins/alpha/tests/scan.test.mjs plugins/alpha/skills/*/SKILL.md\n' +} >"$repo/scripts/affected-tests-scopes.txt" +git_test_config "$repo" add plugins scripts >/dev/null git_test_config "$repo" commit -qm r8 >/dev/null run_sel "$repo" plugins/alpha/skills/one/SKILL.md @@ -1558,7 +1557,8 @@ else fi out="$(cd "$repo" && bash scripts/affected-tests.sh --explain plugins/alpha/skills/one/SKILL.md 2>&1)" -if contains "$out" "select: plugins/alpha/tests/scan.test.sh (test-scope plugins/alpha/skills/*.md)"; then +if contains "$out" "select: plugins/alpha/tests/scan.test.sh (test-scope plugins/alpha/skills/*.md)" && + ! contains "$out" "scans skill bodies"; then ok "R8: --explain reports the declared glob" else fail "R8: --explain lacks the declared glob: $out" @@ -1571,13 +1571,6 @@ else fail "R8: a declared file was not mapped by its scope (rc=$RC): $OUT" fi -run_sel "$repo" plugins/beta/hooks/beta-hook.sh -if [[ "$RC" -eq 0 ]] && ! has_line "$OUT" plugins/alpha/tests/scan.test.sh; then - ok "R8: a declaration below the leading comment block is not read" -else - fail "R8: a declaration after the first code line was honored (rc=$RC): $OUT" -fi - run_sel "$repo" plugins/alpha/zzorphan-plugin.sh if [[ "$RC" -eq 1 ]]; then ok "R8: a plugin file no suite names or declares is still UNMAPPED" @@ -1585,24 +1578,41 @@ else fail "R8: a plugin file nothing names or declares was mapped (rc=$RC): $OUT" fi -# A changed suite's declarations are checked: one below the leading comment -# block is never read, and a glob matching no file has outlived what it read. -out="$(cd "$repo" && bash scripts/affected-tests.sh plugins/alpha/tests/scan.test.sh 2>&1)" +# The list is checked: an entry naming no suite fails every run, and a glob +# matching no file fails the run that changes the list. +cp "$repo/scripts/affected-tests-scopes.txt" "$TMP_ROOT/scopes.keep" +printf 'plugins/alpha/tests/gone.test.sh plugins/alpha/*\n' >>"$repo/scripts/affected-tests-scopes.txt" +out="$(cd "$repo" && bash scripts/affected-tests.sh plugins/beta/hooks/beta-hook.sh 2>&1)" RC=$? -if [[ "$RC" -eq 2 ]] && contains "$out" 'plugins/alpha/tests/scan.test.sh:7'; then - ok "R8: a changed suite with a declaration below its header fails loud" +if [[ "$RC" -eq 2 ]] && contains "$out" "names 'plugins/alpha/tests/gone.test.sh', which is not a suite"; then + ok "R8: an entry naming no suite fails the run" else - fail "R8: a misplaced declaration was not refused (rc=$RC): $out" + fail "R8: a stale suite entry was not refused (rc=$RC): $out" fi -printf '#!/usr/bin/env bash\n# test-scope: plugins/nowhere/*\necho stale\n' >"$repo/plugins/alpha/tests/stale.test.sh" -out="$(cd "$repo" && bash scripts/affected-tests.sh plugins/alpha/tests/stale.test.sh 2>&1)" +cp "$TMP_ROOT/scopes.keep" "$repo/scripts/affected-tests-scopes.txt" +printf 'plugins/alpha/tests/scan.test.sh plugins/nowhere/*\n' >>"$repo/scripts/affected-tests-scopes.txt" +out="$(cd "$repo" && bash scripts/affected-tests.sh plugins/beta/hooks/beta-hook.sh 2>&1)" RC=$? -if [[ "$RC" -eq 2 ]] && contains "$out" 'plugins/nowhere/*, which matches no file'; then - ok "R8: a changed suite declaring a glob that matches nothing fails loud" +if [[ "$RC" -eq 0 ]]; then + ok "R8: a glob matching nothing is not checked while the list is unchanged" else - fail "R8: a stale declaration was not refused (rc=$RC): $out" + fail "R8: an unchanged list failed the run (rc=$RC): $out" +fi +out="$(cd "$repo" && bash scripts/affected-tests.sh --allow-unmapped scripts/affected-tests-scopes.txt 2>&1)" +RC=$? +if [[ "$RC" -eq 2 ]] && contains "$out" ' - plugins/nowhere/*'; then + ok "R8: a changed list declaring a glob that matches nothing fails loud" +else + fail "R8: a stale glob in a changed list was not refused (rc=$RC): $out" +fi +cp "$TMP_ROOT/scopes.keep" "$repo/scripts/affected-tests-scopes.txt" +out="$(cd "$repo" && bash scripts/affected-tests.sh --allow-unmapped scripts/affected-tests-scopes.txt 2>&1)" +RC=$? +if [[ "$RC" -eq 0 ]]; then + ok "R8: a changed list whose globs all match passes the check" +else + fail "R8: a valid list was refused (rc=$RC): $out" fi -rm -f "$repo/plugins/alpha/tests/stale.test.sh" # --- --unmapped-corpus: an unmapped file selects its own language's corpus --- # The report stays, the exit says so (4), and only the file's language runs: a @@ -1639,7 +1649,7 @@ rm -rf "$repo" # --- --with-always is accepted and widens nothing ---------------------------- # A caller that still passes it must neither fail nor get a wider selection: -# the live-tree suites it used to add declare a test-scope now. +# the live-tree suites it used to add are declared in the scopes list now. mk_repo repo run_sel "$repo" --with-always plugins/alpha/hooks/alpha-hook.sh with_out="$OUT" with_rc="$RC" @@ -1814,19 +1824,19 @@ fi rm -rf "$repo" # --- --replay: each commit selected against its parent, with this tree's rules -- -# The replayed commits predate this tree's declarations, so a replay carries -# the tree's own test-scope table to every commit; --against runs the selector -# at with 's table and prints only the suites the two disagree on. +# A replay carries this tree's lists to every commit, since the commits may +# predate them; --against runs the selector at with 's lists and +# prints only the suites the two disagree on. mk_repo repo -printf '#!/usr/bin/env bash\n# test-scope: plugins/alpha/*\necho old\n' >"$repo/scripts/zz-old-scan.test.sh" +printf '#!/usr/bin/env bash\necho old\n' >"$repo/scripts/zz-old-scan.test.sh" printf '#!/usr/bin/env bash\necho new\n' >"$repo/scripts/zz-new-scan.test.sh" +printf 'scripts/zz-old-scan.test.sh plugins/alpha/*\n' >"$repo/scripts/affected-tests-scopes.txt" git_test_config "$repo" add scripts >/dev/null git_test_config "$repo" commit -qm scans >/dev/null printf '# edited\n' >>"$repo/plugins/alpha/hooks/alpha-hook.sh" git_test_config "$repo" commit -qam 'edit alpha hook' >/dev/null alpha_commit="$(git -C "$repo" rev-parse HEAD)" -printf '#!/usr/bin/env bash\necho old\n' >"$repo/scripts/zz-old-scan.test.sh" -printf '#!/usr/bin/env bash\n# test-scope: plugins/alpha/*\necho new\n' >"$repo/scripts/zz-new-scan.test.sh" +printf 'scripts/zz-new-scan.test.sh plugins/alpha/*\n' >"$repo/scripts/affected-tests-scopes.txt" out="$(cd "$repo" && bash scripts/affected-tests.sh --replay HEAD~1..HEAD 2>/dev/null)" RC=$? diff --git a/scripts/check-loop-lane-floor-drift.test.sh b/scripts/check-loop-lane-floor-drift.test.sh index b7db12db10..c7220d1fa8 100755 --- a/scripts/check-loop-lane-floor-drift.test.sh +++ b/scripts/check-loop-lane-floor-drift.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/*/reference/reader-contract.md # Black-box contract test for check-loop-lane-floor-drift.sh. # # Self-contained and cwd-independent: builds a throwaway root holding a fake diff --git a/scripts/lib/gate-entry.test.sh b/scripts/lib/gate-entry.test.sh index 3a59050fe2..d184727d78 100755 --- a/scripts/lib/gate-entry.test.sh +++ b/scripts/lib/gate-entry.test.sh @@ -3,10 +3,6 @@ # this process would take the suite down with it, which is the property under # test. # -# A live case scans every script under scripts/ for a hand-rolled base-ref -# predicate: -# test-scope: scripts/*.sh -# # shellcheck disable=SC2016 # child programs stay single-quoted so this shell does not expand $1 before bash -c set -uo pipefail diff --git a/scripts/lib/test-harness.test.sh b/scripts/lib/test-harness.test.sh index 41e08ee6a1..aca8f89f9f 100755 --- a/scripts/lib/test-harness.test.sh +++ b/scripts/lib/test-harness.test.sh @@ -1,9 +1,6 @@ #!/usr/bin/env bash # Self-test for scripts/lib/test-harness.sh. # -# A live case reads every suite under scripts/ that sources the harness: -# test-scope: scripts/*.test.sh -# # The load-bearing property is the exit contract: a suite that recorded a # failed assertion and then called test_harness::report cannot exit 0. The # other cases pin the print format, the sourced-only guard, the last-line diff --git a/scripts/validate-plugin-contracts.test.sh b/scripts/validate-plugin-contracts.test.sh index 3546f82d6c..80afcda3da 100755 --- a/scripts/validate-plugin-contracts.test.sh +++ b/scripts/validate-plugin-contracts.test.sh @@ -1,5 +1,4 @@ #!/usr/bin/env bash -# test-scope: plugins/*/retirements.yaml plugins/*/skills/*/evals/evals.json plugins/*/reference/artifact-protocol.md # Black-box contract test for the check-only carve-out assertions in # validate-plugin-contracts.mjs. # From 66bb5abb5cf9a02342baad057d33609d1de5d3ce Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 03:39:47 -0400 Subject: [PATCH 07/18] fix(scripts): resolve a name-only path to the nearest file of that name `$PLUGIN_DIR/README.md` spells only the basename after a variable, so the selector matched it against every README.md whose path relative to a common ancestor is `README.md`, including the repository root's. A change to the root README.md then selected every suite that reads its own plugin's README. A file of that name between the naming file and the candidate now claims the mention. Co-Authored-By: Claude Opus 5.5 (1M context) --- scripts/affected-tests.sh | 18 +++++++++++++----- scripts/affected-tests.test.sh | 19 +++++++++++++++++++ 2 files changed, 32 insertions(+), 5 deletions(-) diff --git a/scripts/affected-tests.sh b/scripts/affected-tests.sh index 6e7cf6bdc4..097f541c3f 100755 --- a/scripts/affected-tests.sh +++ b/scripts/affected-tests.sh @@ -136,7 +136,8 @@ # file of that name ends in, or in the file's path relative to a directory that # holds both files: `$SCRIPT_DIR/lib/x.sh` from a script beside lib/, # `$PLUGIN_DIR/skills/interview/SKILL.md` or `$PLUGIN_ROOT/hooks/hooks.json` -# from inside the plugin. A shared library's +# from inside the plugin. A path that spells only the name, `$DIR/README.md`, +# means the nearest file of that name above the naming file. A shared library's # source and copies are the exception and keep the plain rule: R5's copies share # a basename on purpose, change together with their source, and a suite naming # its own plugin's copy is naming the shared source. @@ -609,7 +610,7 @@ token_hits() { } FILENAME == allf { b = base_of($0) - if (b in nrt) same[b, ++nsame[b]] = $0 + if (b in nrt) { same[b, ++nsame[b]] = $0; here[$0] = 1 } next } # uniq_suffix: the shortest path suffix, two components or more, that no @@ -635,12 +636,14 @@ token_hits() { # does when it ends in the shortest unique suffix of t, or in the path of t # relative to a directory holding both files ($SCRIPT_DIR/lib/x.sh, # $PLUGIN_DIR/skills//SKILL.md). - function resolves(namer, pt, t, u, a, i, b) { + # A path spelling only the name, `$DIR/README.md`, means the nearest file of + # that name above the namer: a closer one claims it from any farther one. + function resolves(namer, pt, t, u, a, i, b, rel, claimed) { a = dir_of(namer) if (a == dir_of(t)) return 1 + b = base_of(t) if (!index(pt, "/")) { if (a != "" && index(t, a) != 1) return 0 - b = base_of(t) for (i = 1; i <= nsame[b]; i++) if (same[b, i] != t && (a == "" || index(same[b, i], a) == 1)) return 0 return 1 @@ -648,8 +651,13 @@ token_hits() { if (!(t in usuf)) usuf[t] = uniq_suffix(t) u = usuf[t] if (u != "" && (pt == u || ends(pt, "/" u))) return 1 + claimed = 0 while (1) { - if ((a == "" || index(t, a) == 1) && (pt == substr(t, length(a) + 1) || ends(pt, "/" substr(t, length(a) + 1)))) return 1 + if (a == "" || index(t, a) == 1) { + rel = substr(t, length(a) + 1) + if ((pt == rel || ends(pt, "/" rel)) && (index(rel, "/") || !claimed)) return 1 + } + if (((a b) in here) && (a b) != t) claimed = 1 if (a == "") return 0 sub(/[^\/]*\/$/, "", a) } diff --git a/scripts/affected-tests.test.sh b/scripts/affected-tests.test.sh index 6fcd872113..5cafccebde 100755 --- a/scripts/affected-tests.test.sh +++ b/scripts/affected-tests.test.sh @@ -1783,6 +1783,25 @@ if [[ "$alpha_rc" -eq 0 ]] && has_line "$alpha_out" plugins/alpha/tests/manifest else fail "ambiguous: plugin.json resolution wrong (rc=$alpha_rc/$RC): [$alpha_out] [$OUT]" fi + +# `$DIR/README.md` spells only the name, so it means the nearest README.md above +# the naming file: the plugin's own from inside the plugin, the root one from +# scripts/. +printf '# root\n' >"$repo/README.md" +# shellcheck disable=SC2016 # deliberate: the emitted fixtures must expand these +{ + printf 'grep -q x "$PLUGIN_DIR/README.md"\n' >"$repo/plugins/alpha/tests/readme.test.sh" + printf 'grep -q x "$REPO_ROOT/README.md"\n' >"$repo/scripts/zz-root-readme.test.sh" +} +run_sel "$repo" README.md +root_out="$OUT" +run_sel "$repo" plugins/alpha/README.md +if has_line "$root_out" scripts/zz-root-readme.test.sh && ! has_line "$root_out" plugins/alpha/tests/readme.test.sh && + has_line "$OUT" plugins/alpha/tests/readme.test.sh && ! has_line "$OUT" scripts/zz-root-readme.test.sh; then + ok "ambiguous: a path spelling only the name means the nearest file of that name" +else + fail "ambiguous: nearest-file resolution wrong: [$root_out] [$OUT]" +fi rm -rf "$repo" # --- R5 copies inside skill directories still reach every copy's suite ------- From fc49d42abc3846465b3979cc4b085d93d671f764 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 04:04:44 -0400 Subject: [PATCH 08/18] fix(scripts): report a replayed commit's suites as dropped when the new selector picks none `--replay --against` split its two selections with `FNR == 1 { side++ }`. An empty new selection has no first line, so the old selection was read as the new one and every suite ran was printed as added. The split now keys on the file name. Co-Authored-By: Claude Opus 5.5 (1M context) --- scripts/affected-tests.sh | 2 +- scripts/affected-tests.test.sh | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/scripts/affected-tests.sh b/scripts/affected-tests.sh index 097f541c3f..14a21a9676 100755 --- a/scripts/affected-tests.sh +++ b/scripts/affected-tests.sh @@ -1230,7 +1230,7 @@ run_replay() { u_old=$(grep -c . "$WORK_DIR/old.unmapped") printf 'commit %s %s %s %s %s %s\n' "$c" "$n_new" "$n_old" "$u_new" "$u_old" "$subject" sed 's/^/ unmapped /' "$WORK_DIR/new.unmapped" - awk -F '\t' 'FNR == 1 { side++ } side == 1 { n[$1] = $2; next } { o[$1] = $2 } + awk -F '\t' 'FILENAME == ARGV[1] { n[$1] = $2; next } { o[$1] = $2 } END { for (s in n) if (!(s in o)) print " + " s " (" n[s] ")" for (s in o) if (!(s in n)) print " - " s " (" o[s] ")" diff --git a/scripts/affected-tests.test.sh b/scripts/affected-tests.test.sh index 5cafccebde..ad1f477eeb 100755 --- a/scripts/affected-tests.test.sh +++ b/scripts/affected-tests.test.sh @@ -1878,6 +1878,24 @@ else fail "--replay --against output wrong (rc=$RC): $out" fi +# A commit this selector maps to no suite still reports what ran as +# dropped, not as added. +printf 'scripts/zz-old-scan.test.sh plugins/beta/*\n' >"$repo/scripts/affected-tests-scopes.txt" +git_test_config "$repo" commit -qam 'scan beta' >/dev/null +printf 'notes\n' >"$repo/plugins/beta/zz-notes.yaml" +git_test_config "$repo" add plugins >/dev/null +git_test_config "$repo" commit -qm 'add beta notes' >/dev/null +beta_commit="$(git -C "$repo" rev-parse HEAD)" +printf 'scripts/zz-new-scan.test.sh plugins/alpha/*\n' >"$repo/scripts/affected-tests-scopes.txt" +out="$(cd "$repo" && bash scripts/affected-tests.sh --replay HEAD~1..HEAD --against HEAD 2>/dev/null)" +RC=$? +if [[ "$RC" -eq 0 ]] && contains "$out" "commit $beta_commit 0 1 " && + has_line "$out" " - scripts/zz-old-scan.test.sh (test-scope plugins/beta/*)" && ! contains "$out" " + "; then + ok "--replay --against lists a commit's suites as dropped when this selector picks none" +else + fail "--replay --against with an empty selection wrong (rc=$RC): $out" +fi + for args in "--replay HEAD~1..HEAD plugins/alpha/hooks/alpha-hook.sh" "--against HEAD" "--replay HEAD~1..HEAD --run"; do # shellcheck disable=SC2086 # deliberate: each case is a word list (cd "$repo" && bash scripts/affected-tests.sh $args >/dev/null 2>&1) From cd900f3e06ad7813515fbeac2149d88ee4f1ad07 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 04:12:54 -0400 Subject: [PATCH 09/18] fix(scripts): record the performance plugin's spawn_noise copy as covered by the drift step The copy of lib/spawn_noise.py that sync-shared-copies.sh writes into the performance plugin has no importer in code, so once the selector stopped treating a plugin directory as one scope it mapped to no suite. It changes only with its canonical, whose fan-out selects harness-ops's spawn_noise suites, and the shared-copies --check step fails a copy that drifts. Co-Authored-By: Claude Opus 5.5 (1M context) --- scripts/affected-tests-no-suite.txt | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/scripts/affected-tests-no-suite.txt b/scripts/affected-tests-no-suite.txt index 10559c5c09..8aac31f564 100644 --- a/scripts/affected-tests-no-suite.txt +++ b/scripts/affected-tests-no-suite.txt @@ -145,6 +145,13 @@ plugins/*/evals/* # which run those tests whenever any script in that directory changes. plugins/evals/skills/plugin-eval/scripts/fixtures/*.jsonl +# The performance plugin's generated copy of lib/spawn_noise.py +# (scripts/shared-copies.txt). No code in the plugin imports it; its skills +# show the import in prose. It changes only with its canonical, whose shared-lib +# fan-out selects harness-ops's spawn_noise suites, and the shared-copies +# --check drift step fails a copy that differs from the canonical. +plugins/performance/lib/spawn_noise.py + # The task-end judge's calibration cases: test and production code copied # verbatim from other repositories' histories, read as data by the judge, never # run. The lane covering them is the testing plugin's shell lane: metrics.test.sh From 023d668ee829fa03aa26745f6bb1599ca02b5b8d Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 05:07:52 -0400 Subject: [PATCH 10/18] fix(scripts): apply the design's manifest and cross-language rules to test selection Manifests and changelogs (plugin.json, marketplace.json, hooks.json, settings.json, package*.json, CHANGELOG.md, LICENSE) no longer select a suite through a mention; the manifest and changelog gates own them. A file in another language counts only where its line runs or loads the changed file: the interpreter test now matches a word, not the `.sh` of a file name, and the shell-script and same-directory exceptions are gone. An ambiguous name resolves only through a path below the repository root that has a directory in it, so `$T/README.md` and `$ROOT/.github/workflows/ci.yml` no longer reach the real files. The suites the strace showed reading such files declare them in affected-tests-scopes.txt. Co-Authored-By: Claude Opus 5.5 (1M context) --- scripts/affected-tests-scopes.txt | 25 ++++++- scripts/affected-tests.sh | 104 ++++++++++++++++-------------- scripts/affected-tests.test.sh | 63 ++++++++++++------ 3 files changed, 120 insertions(+), 72 deletions(-) diff --git a/scripts/affected-tests-scopes.txt b/scripts/affected-tests-scopes.txt index dc7f9bb8bd..a5ea00b9f9 100644 --- a/scripts/affected-tests-scopes.txt +++ b/scripts/affected-tests-scopes.txt @@ -14,7 +14,28 @@ # reads, not the whole plugin, unless it reads the whole plugin. # Live-tree suites under scripts/: they scan the repository rather than fixtures. -scripts/affected-tests.test.sh scripts/affected-tests* scripts/sync-*.sh scripts/lib/sync-*.sh # runs every sync manifest and reads the selector's lists +scripts/affected-tests.test.sh scripts/affected-tests* scripts/sync-*.sh scripts/lib/sync-*.sh .github/workflows/ci.yml # runs every sync manifest, reads the selector's lists and the live workflow +scripts/check-docs-only-gate.test.sh .github/workflows/ci.yml # checks the live workflow against the docs-only contract +scripts/ci-fail-a-draft.test.sh .github/workflows/ci.yml # reads the live workflow +scripts/check-lane-coverage.test.sh .github/workflows/ci.yml # checks the live workflow's lane coverage +scripts/check-hook-wiring-liveness.test.sh .claude/settings.json # reads the live project settings + +# Suites that read a skill body or README through a path that spells only its +# name ($SKILL_DIR/SKILL.md), which does not resolve. +plugins/prototype/skills/explore-directions/scripts/detect-ecosystems.test.sh plugins/prototype/skills/explore-directions/SKILL.md +plugins/prototype/skills/pressure-test/scripts/detect-ecosystems.test.sh plugins/prototype/skills/pressure-test/SKILL.md +plugins/repo-hygiene/skills/clean/scripts/destructive-guard.test.sh plugins/repo-hygiene/skills/clean/SKILL.md +plugins/source-control/skills/babysit-prs/scripts/tests/test_guards.py plugins/source-control/skills/*/SKILL.md +plugins/disk-hygiene/skills/clean/scripts/test_hygiene.py plugins/disk-hygiene/skills/clean/SKILL.md +plugins/playwright/skills/playwright/scripts/update.test.sh plugins/playwright/skills/playwright/SKILL.md +plugins/guardrails/hooks/coverage-manifest.test.sh plugins/guardrails/README.md + +# Suites that read the repository's own .claude/settings.json, which no mention +# reaches (MANIFESTS, and a root path does not resolve). +plugins/docs-hygiene/skills/compress/scripts/detect-caveman.test.sh .claude/settings.json +plugins/harness-config/skills/audit-permission-state/scripts/audit.test.sh .claude/settings.json +plugins/harness-ops/skills/plugins/scripts/fleet-state.test.sh .claude/settings.json +plugins/harness-ops/skills/plugins/scripts/sync-run.test.sh .claude/settings.json scripts/lib/gate-entry.test.sh scripts/*.sh # scans every script for a hand-rolled base-ref predicate scripts/lib/test-harness.test.sh scripts/*.test.sh # every harness suite must end on test_harness::report scripts/validate-plugin-contracts.test.sh plugins/*/retirements.yaml plugins/*/skills/*/evals/evals.json plugins/*/reference/artifact-protocol.md # the validator walks every plugin @@ -29,7 +50,7 @@ plugins/discovery/agents/tool-honesty.test.sh plugins/discovery/agents/*.md plugins/review/tests/change-set-block.test.sh plugins/review/agents/*.md plugins/multi-agent/tests/drift-audit.test.sh plugins/multi-agent/agents/*.md plugins/planning/tests/reattach-slice.test.sh plugins/planning/skills/*/SKILL.md -plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh plugins/*/.mcp.json plugins/harness-config/skills/*/SKILL.md +plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh plugins/*/.mcp.json plugins/harness-config/skills/*/SKILL.md .claude/settings.json plugins/source-control/skills/babysit-prs/scripts/engine.test.sh plugins/source-control/*.md plugins/source-control/scripts/babysit-*.sh # test_guards.py walks the plugin # allowed-tools pairing: every listed skill's body and bundled scripts. diff --git a/scripts/affected-tests.sh b/scripts/affected-tests.sh index 14a21a9676..6589a3872e 100755 --- a/scripts/affected-tests.sh +++ b/scripts/affected-tests.sh @@ -82,12 +82,11 @@ # R4 other language a file in another language counts only where the naming # line runs or loads the file: an interpreter or process API on # the line (bash, sh, python3, node, pwsh, source, subprocess, -# spawn*, exec*, ...), a path to the file rather than its bare -# name, a shell script as the named file (another language -# has no other use for one), or a file in the naming file's -# own directory (a wrapper suite hands its sibling Python -# suite to a runner by bare name). A chain takes at most one -# such transition and then keeps walking its new language. +# spawn*, exec*, ...), or a path to the file rather than its +# bare name. Nothing else on the line counts: not the named +# file being a shell script, not the naming file sitting in +# the same directory. A chain takes at most one such +# transition and then keeps walking its new language. # A data file (any extension that is not code) reaches code of # every language that names it, and that first step spends no # transition: data has no language of its own to stay inside. @@ -124,23 +123,33 @@ # cannot spell, one with a character outside the class, keeps the substring # test rather than losing its coverage. # -# AMBIGUOUS NAMES. A basename two or more files carry, and the structural names -# (README.md, SKILL.md, AGENTS.md, CLAUDE.md, index.md, CHANGELOG.md, LICENSE, -# plugin.json, marketplace.json, settings.json, hooks.json, package.json, -# package-lock.json), name a specific file only when the mention RESOLVES to -# it, because a bare `SKILL.md` or `config.json` says nothing about which one. -# Any mention from the file's own directory resolves. A bare name resolves from -# a directory above the file when no other file of that name sits below that -# directory (`FIXTURES / "questions.json"`), and never otherwise. A path -# resolves when it ends in the shortest suffix of the file's path that no other -# file of that name ends in, or in the file's path relative to a directory that -# holds both files: `$SCRIPT_DIR/lib/x.sh` from a script beside lib/, -# `$PLUGIN_DIR/skills/interview/SKILL.md` or `$PLUGIN_ROOT/hooks/hooks.json` -# from inside the plugin. A path that spells only the name, `$DIR/README.md`, -# means the nearest file of that name above the naming file. A shared library's -# source and copies are the exception and keep the plain rule: R5's copies share -# a basename on purpose, change together with their source, and a suite naming -# its own plugin's copy is naming the shared source. +# MANIFESTS. plugin.json, marketplace.json, hooks.json, settings.json, +# package.json, package-lock.json, CHANGELOG.md and LICENSE select no suite +# through a mention: a suite that reads one reads its name or version, and the +# manifest, changelog and catalog gates own those files. Only R1, R2 and a +# declared scope (R8) reach a suite from them; anything else is on the no-suite +# list. +# +# AMBIGUOUS NAMES. A basename two or more files carry, and the structural docs +# (README.md, SKILL.md, AGENTS.md, CLAUDE.md, index.md), name a specific file +# only when the mention RESOLVES to it, because a bare `SKILL.md` or +# `config.json` says nothing about which one. Any mention from the file's own +# directory resolves. A bare name resolves from a directory above the file when +# no other file of that name sits below that directory +# (`FIXTURES / "questions.json"`), and never otherwise. A path resolves when it +# ends in the shortest suffix of the file's path, two components or more, that +# no other file of that name ends in, or in the file's path relative to a +# directory below the repository root that holds both files, when that relative +# path has a directory in it: `$SCRIPT_DIR/lib/x.sh` from a script beside lib/, +# `$PLUGIN_DIR/skills/interview/SKILL.md` from inside the plugin. A path that +# spells only the name (`$SKILL_DIR/SKILL.md`, `$T/README.md`) or that is +# relative to the root alone (`$ROOT/.github/workflows/ci.yml`) does not +# resolve: tests build the same path under a temporary directory as often as +# they read the real file, so a suite that reads such a file declares it (R8), +# as the strace of every suite showed where one does. A shared +# library's source and copies are the exception and keep the plain rule: R5's +# copies share a basename on purpose, change together with their source, and a +# suite naming its own plugin's copy is naming the shared source. # # COMMENTS. A line that is only a comment (`#` in shell, Python and # PowerShell; `//`, `/*` or a `*` continuation in Node) names nothing, in suites @@ -195,8 +204,10 @@ SCOPES_LIST="${AFFECTED_TESTS_SCOPES:-scripts/affected-tests-scopes.txt}" # Basenames that name a repository-wide role, reached only through a resolved # mention (AMBIGUOUS NAMES in the header), however few files carry them today. -STRUCTURAL_BASENAMES=" README.md SKILL.md AGENTS.md CLAUDE.md index.md CHANGELOG.md LICENSE " -STRUCTURAL_BASENAMES+="plugin.json marketplace.json settings.json hooks.json package.json package-lock.json " +STRUCTURAL_BASENAMES=" README.md SKILL.md AGENTS.md CLAUDE.md index.md " +# Manifests and changelogs, which no mention reaches (MANIFESTS in the header). +MANIFEST_BASENAMES=" plugin.json marketplace.json hooks.json settings.json package.json package-lock.json " +MANIFEST_BASENAMES+="CHANGELOG.md LICENSE " # Print the header block (everything after the shebang up to the first # non-comment line) with its comment markers stripped. @@ -592,12 +603,10 @@ token_hits() { function base_of(p) { sub(/.*\//, "", p); return p } function ends(s, t) { return length(s) >= length(t) && substr(s, length(s) - length(t) + 1) == t } # Plain names: a basename that is itself a path token gets the exact test; - # anything else keeps the substring test rather than losing coverage. The - # directories that carry each name feed the same-directory arm of R4. + # anything else keeps the substring test rather than losing coverage. FILENAME == plainf { if ($0 == "") next b = base_of($0) - pdir[b, dir_of($0)] = 1 if (b ~ /^[A-Za-z0-9_.-]+$/) want[b] = 1 else loose[b] = 1 next @@ -610,7 +619,7 @@ token_hits() { } FILENAME == allf { b = base_of($0) - if (b in nrt) { same[b, ++nsame[b]] = $0; here[$0] = 1 } + if (b in nrt) same[b, ++nsame[b]] = $0 next } # uniq_suffix: the shortest path suffix, two components or more, that no @@ -634,11 +643,10 @@ token_hits() { # mention from the directory of t does. A bare name does from a directory # above t when no other file of that name sits below that directory. A path # does when it ends in the shortest unique suffix of t, or in the path of t - # relative to a directory holding both files ($SCRIPT_DIR/lib/x.sh, + # relative to a directory below the root holding both files, when that + # relative path has a directory in it ($SCRIPT_DIR/lib/x.sh, # $PLUGIN_DIR/skills//SKILL.md). - # A path spelling only the name, `$DIR/README.md`, means the nearest file of - # that name above the namer: a closer one claims it from any farther one. - function resolves(namer, pt, t, u, a, i, b, rel, claimed) { + function resolves(namer, pt, t, u, a, i, b, rel) { a = dir_of(namer) if (a == dir_of(t)) return 1 b = base_of(t) @@ -651,29 +659,24 @@ token_hits() { if (!(t in usuf)) usuf[t] = uniq_suffix(t) u = usuf[t] if (u != "" && (pt == u || ends(pt, "/" u))) return 1 - claimed = 0 - while (1) { - if (a == "" || index(t, a) == 1) { - rel = substr(t, length(a) + 1) - if ((pt == rel || ends(pt, "/" rel)) && (index(rel, "/") || !claimed)) return 1 - } - if (((a b) in here) && (a b) != t) claimed = 1 - if (a == "") return 0 - sub(/[^\/]*\/$/, "", a) + for (; a != ""; sub(/[^\/]*\/$/, "", a)) { + if (index(t, a) != 1) continue + rel = substr(t, length(a) + 1) + if (index(rel, "/") && (pt == rel || ends(pt, "/" rel))) return 1 } + return 0 } - # runs_or_loads: R4. An interpreter or process API on the line, a path to - # the file, a shell script as the named file, or a file of the same - # directory: a wrapper suite hands its sibling to a runner by bare name. - function runs_or_loads(path, name, n, j) { - if (exec_line || name ~ /\.(sh|bash)$/ || ((name SUBSEP dir_of(path)) in pdir)) return 1 + # runs_or_loads: R4. An interpreter or process API on the line, or a path + # to the file. + function runs_or_loads(name, n, j) { + if (exec_line) return 1 for (j = 1; j <= n; j++) if (ends(ptok[j], "/" name)) return 1 return 0 } function keep(path, name, n) { key = path SUBSEP name if (!(key in kept)) { kept[key] = 0; order[++nkept] = key } - if (!kept[key] && runs_or_loads(path, name, n)) kept[key] = 1 + if (!kept[key] && runs_or_loads(name, n)) kept[key] = 1 } # comment_only: a whole-line comment names nothing (COMMENTS in the header), # except a shellcheck source directive and a JSDoc type import. @@ -690,7 +693,8 @@ token_hits() { path = substr($0, 1, i - 1) text = substr($0, i + 1) if (comment_only(path, text)) next - exec_line = text ~ /(^|[^A-Za-z0-9_-])(bash|sh|zsh|python3?|node|deno|pwsh|powershell|uv|npx|source|subprocess|Popen|check_output|check_call|spawn[A-Za-z0-9_]*|exec[A-Za-z0-9_]*|execa|child_process|Start-Process|Invoke-Expression)([^A-Za-z0-9_-]|$)/ + # A word, not an extension: the `.sh` of `x.sh` is no interpreter. + exec_line = text ~ /(^|[^A-Za-z0-9_.-])(bash|sh|zsh|python3?|node|deno|pwsh|powershell|uv|npx|source|subprocess|Popen|check_output|check_call|spawn[A-Za-z0-9_]*|exec[A-Za-z0-9_]*|execa|child_process|Start-Process|Invoke-Expression)([^A-Za-z0-9_-]|$)/ # Path tokens. A leading `.` stays: `./x`, `../x` and `.claude-plugin/x` # are paths, not punctuation. np = split(text, ptok, /[^A-Za-z0-9_.\/-]+/) @@ -903,6 +907,8 @@ select_for() { *) ;; esac b="${p##*/}" + # MANIFESTS: no mention reaches a suite from a manifest or changelog. + [[ "$MANIFEST_BASENAMES" == *" $b "* ]] && continue printf '%s\n' "$b" >>"$WORK_DIR/patterns" if [[ "$STRUCTURAL_BASENAMES" == *" $b "* ]] || [[ -n "${AMBIGUOUS[$b]:-}" && -z "${SYNC_MEMBER[$p]:-}" ]]; then diff --git a/scripts/affected-tests.test.sh b/scripts/affected-tests.test.sh index 67cef40b60..dd11ddfe75 100755 --- a/scripts/affected-tests.test.sh +++ b/scripts/affected-tests.test.sh @@ -1048,8 +1048,8 @@ fi # A file in another language that merely contains the name (a string, a log # message) is not a dependent and its suite is not selected: across languages # the text says nothing about a dependency unless the line runs or loads the -# file. An interpreter on the line, a path to the file, or a bare name from the -# file's own directory (a wrapper handing its sibling to a runner) does. +# file. An interpreter on the line or a path to the file does; a bare name from +# the file's own directory, with no interpreter on the line, does not. mkdir -p "$repo/eco/hop" "$repo/eco/elsewhere" printf 'export const c = 3;\n' >"$repo/eco/hop/origin.js" printf 'echo "origin.js is the entry point"\n' >"$repo/eco/elsewhere/mention.test.sh" @@ -1058,6 +1058,11 @@ printf 'node "$ROOT/eco/hop/origin.js"\n' >"$repo/eco/elsewhere/node-runs.test.s # shellcheck disable=SC2016 # deliberate: the emitted fixture must expand these printf 'cp "$SRC/eco/hop/origin.js" "$DEST"\n' >"$repo/eco/elsewhere/path-loads.test.sh" printf 'run_suite origin.js\n' >"$repo/eco/hop/wrapper.test.sh" +printf 'node origin.js\n' >"$repo/eco/hop/node-wrapper.test.sh" +# A .py naming a shell script by bare name, with no interpreter on the line, +# is not a dependent of it either. +printf 'echo hop\n' >"$repo/eco/hop/hop-tool.sh" +printf 'TOOL = "hop-tool.sh"\n' >"$repo/eco/hop/test_hop_listing.py" # A .ps1 that runs the js, so the walk crosses into it once, and a shell file # that runs the .ps1: reaching ITS suite takes a second transition. printf "node origin.js\nfunction Get-Far { 2 }\n" >"$repo/eco/hop/Far.ps1" @@ -1068,7 +1073,8 @@ run_sel "$repo" eco/hop/origin.js if ! has_line "$OUT" eco/elsewhere/mention.test.sh && has_line "$OUT" eco/elsewhere/node-runs.test.sh && has_line "$OUT" eco/elsewhere/path-loads.test.sh && - has_line "$OUT" eco/hop/wrapper.test.sh; then + has_line "$OUT" eco/hop/node-wrapper.test.sh && + ! has_line "$OUT" eco/hop/wrapper.test.sh; then ok "R4: another language's suite runs only where its line runs or loads the file" else fail "R4: cross-language selection wrong (rc=$RC): $OUT" @@ -1078,6 +1084,12 @@ if has_line "$OUT" eco/hop/Far.Tests.ps1 && ! has_line "$OUT" eco/hop/far-runner else fail "R4: the transition budget was not applied (rc=$RC): $OUT" fi +run_sel "$repo" eco/hop/hop-tool.sh +if ! has_line "$OUT" eco/hop/test_hop_listing.py; then + ok "R4: a bare shell-script name in another language is not a dependency" +else + fail "R4: a bare .sh name selected a Python suite (rc=$RC): $OUT" +fi # --- --run refuses to guess a runner for another ecosystem ----------------- # Selected-but-not-run must never report as success. The invocations differ per @@ -1379,7 +1391,7 @@ mk_cmt_dependent sh-code sh 'source "$(dirname "$0")/hub-target.sh"\n' mk_cmt_dependent sh-trailing sh 'echo ok # runs after hub-target.sh\n' # shellcheck disable=SC2016 # deliberate: the emitted fixture must expand these mk_cmt_dependent sh-directive sh '# shellcheck source=hub-target.sh\n. "$HUB"\n' -mk_cmt_dependent js-code js 'const target = "hub-target.sh";\n' +mk_cmt_dependent js-code js 'spawnSync("bash", ["hub-target.sh"]);\n' mk_cmt_dependent js-typeimport js '/** @import { T } from "./hub-target.sh" */\n/** @param {import("./hub-target.sh").T} t */\nexport const y = 2;\n' printf '#!/usr/bin/env bash\n# covers hub-target.sh\n' >"$repo3/eco/cmt/hub-prose.test.sh" # A Python import never spells the .py, so a comment naming the module is the @@ -1683,8 +1695,8 @@ done # Skills reuse reference names freely, so a bare `probe-doc.md` says nothing # about which one. A mention names a file only when it resolves to it: a path # suffix no other file of that name ends in, any mention from the file's own -# directory, or a path relative to a directory holding both files. The -# structural names (SKILL.md, plugin.json and the like) always resolve this way. +# directory, or a path relative to a directory below the root holding both +# files. The structural docs (SKILL.md and the like) always resolve this way. mk_repo repo own_a=plugins/alpha/skills/sa own_b=plugins/alpha/skills/sb @@ -1772,35 +1784,44 @@ else fail "ambiguous: SKILL.md resolution wrong (rc=$alpha_rc/$RC): [$alpha_out] [$OUT]" fi -mkdir -p "$repo/plugins/beta/.claude-plugin" -printf '{ "name": "beta" }\n' >"$repo/plugins/beta/.claude-plugin/plugin.json" +# MANIFESTS: a suite that spells its plugin's manifest exactly is still not +# selected by it; the manifest gates own it. run_sel "$repo" plugins/alpha/.claude-plugin/plugin.json -alpha_out="$OUT" alpha_rc="$RC" -run_sel "$repo" plugins/beta/.claude-plugin/plugin.json -if [[ "$alpha_rc" -eq 0 ]] && has_line "$alpha_out" plugins/alpha/tests/manifest.test.sh && - [[ "$RC" -eq 0 && -z "$OUT" ]]; then - ok "ambiguous: a manifest named relative to its plugin selects that suite; another plugin's does not" +if [[ "$RC" -eq 0 && -z "$OUT" ]]; then + ok "manifests: a plugin.json selects no suite through a mention" else - fail "ambiguous: plugin.json resolution wrong (rc=$alpha_rc/$RC): [$alpha_out] [$OUT]" + fail "manifests: plugin.json selected a suite (rc=$RC): $OUT" fi -# `$DIR/README.md` spells only the name, so it means the nearest README.md above -# the naming file: the plugin's own from inside the plugin, the root one from -# scripts/. +# A path that spells only the name, or a path from the repository root alone, +# could be a file the suite builds under a temporary directory, so neither +# resolves; a mention from the file's own directory does. printf '# root\n' >"$repo/README.md" # shellcheck disable=SC2016 # deliberate: the emitted fixtures must expand these { printf 'grep -q x "$PLUGIN_DIR/README.md"\n' >"$repo/plugins/alpha/tests/readme.test.sh" printf 'grep -q x "$REPO_ROOT/README.md"\n' >"$repo/scripts/zz-root-readme.test.sh" + # docs/guide.md at the root and a fixture copy ending in the same path. + printf 'grep -q x "$REPO_ROOT/docs/guide.md"\n' >"$repo/scripts/zz-root-path.test.sh" + printf 'grep -q x README.md\n' >"$repo/zz-root-local.test.sh" } +mkdir -p "$repo/docs" "$repo/plugins/alpha/fixtures/docs" +printf '# guide\n' >"$repo/docs/guide.md" +printf '# guide\n' >"$repo/plugins/alpha/fixtures/docs/guide.md" run_sel "$repo" README.md root_out="$OUT" run_sel "$repo" plugins/alpha/README.md -if has_line "$root_out" scripts/zz-root-readme.test.sh && ! has_line "$root_out" plugins/alpha/tests/readme.test.sh && - has_line "$OUT" plugins/alpha/tests/readme.test.sh && ! has_line "$OUT" scripts/zz-root-readme.test.sh; then - ok "ambiguous: a path spelling only the name means the nearest file of that name" +if has_line "$root_out" zz-root-local.test.sh && ! has_line "$root_out" scripts/zz-root-readme.test.sh && + ! has_line "$root_out" plugins/alpha/tests/readme.test.sh && ! has_line "$OUT" plugins/alpha/tests/readme.test.sh; then + ok "ambiguous: a path spelling only the name does not resolve; the file's own directory does" +else + fail "ambiguous: name-only path resolution wrong: [$root_out] [$OUT]" +fi +run_sel "$repo" docs/guide.md +if [[ "$RC" -eq 0 ]] && ! has_line "$OUT" scripts/zz-root-path.test.sh; then + ok "ambiguous: a path from the repository root alone does not resolve" else - fail "ambiguous: nearest-file resolution wrong: [$root_out] [$OUT]" + fail "ambiguous: a root-relative path resolved (rc=$RC): $OUT" fi rm -rf "$repo" From 469cdfbcd3b08b0f91b9a3aaf27b5d2e4c8abe6c Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 05:23:22 -0400 Subject: [PATCH 11/18] test(scripts): pin that `from . import` maps the imported module Co-Authored-By: Claude Opus 5.5 (1M context) --- scripts/affected-tests.test.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/scripts/affected-tests.test.sh b/scripts/affected-tests.test.sh index dd11ddfe75..ebc55d6b80 100755 --- a/scripts/affected-tests.test.sh +++ b/scripts/affected-tests.test.sh @@ -1157,6 +1157,8 @@ printf 'import runner\n' >"$repo/plugins/alpha/scripts/test_runner.py" printf 'import sys\nimport tool as t\n' >"$repo/plugins/alpha/scripts/tests/test_tool_behavior.py" printf 'import tool\n' >"$repo/plugins/alpha/skills/one/scripts/use_tool.py" printf 'import use_tool\n' >"$repo/plugins/alpha/skills/one/scripts/test_use_tool.py" +printf 'from . import tool\n' >"$repo/plugins/alpha/scripts/rel_user.py" +printf 'import rel_user\n' >"$repo/plugins/alpha/scripts/test_rel_user.py" printf 'import tool\n' >"$repo/plugins/beta/scripts/other.py" printf 'import other\n' >"$repo/plugins/beta/scripts/test_other.py" printf 'X = 1\n' >"$repo/plugins/alpha/pkg/sub/deep.py" @@ -1168,8 +1170,9 @@ run_sel "$repo" plugins/alpha/scripts/tool.py if [[ "$RC" -eq 0 ]] && has_line "$OUT" plugins/alpha/scripts/test_runner.py && has_line "$OUT" plugins/alpha/scripts/tests/test_tool_behavior.py && has_line "$OUT" plugins/alpha/skills/one/scripts/test_use_tool.py && + has_line "$OUT" plugins/alpha/scripts/test_rel_user.py && ! has_line "$OUT" plugins/beta/scripts/test_other.py; then - ok "python: an import selects from the module's directory, below it, and across its plugin" + ok "python: an import (also 'from . import') selects from the module's directory, below it, and across its plugin" else fail "python: import selection wrong for tool.py (rc=$RC): $OUT" fi From e5541a14022b99e92765cfb45d425af7f8a15107 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 06:14:48 -0400 Subject: [PATCH 12/18] fix(scripts): declare test scopes in suite headers and drop the Python import rule The selector now reads each scanning suite's declared scope from a `# test-scope: ...` line in the suite's leading comment block, as design rule S7 specifies, instead of the central scripts/affected-tests-scopes.txt. The 87 entries moved verbatim into their 87 suites; reading the headers back gives the same 87 lines. A changed suite whose glob matches no file fails the run, and --replay carries this tree's declarations to older commits through AFFECTED_TESTS_SCOPES (against , it reads 's own headers). The Python import rule (`import foo` names foo.py) is removed: design rules S1-S9 do not include it, and with it the replay selected 14.1% more suites than the design model. A module only an import reaches is now unmapped and falls back to the Python corpus (S9). Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/ci-runner-routing.md | 6 +- .../actionlint/hooks/actionlint-check.test.sh | 1 + plugins/animation/scripts/animation.test.sh | 1 + ...heck-prerequisite-resolution-slice.test.sh | 1 + .../check-security-binding.fixtures.test.sh | 1 + .../generate-identity-prerequisites.test.sh | 1 + .../resolve-prerequisites.fixtures.test.sh | 1 + plugins/code-metrics/scripts/dispatch.test.sh | 1 + .../scripts/tool-free-path.test.sh | 1 + .../scripts/audit-complexity.test.sh | 1 + .../scripts/audit-coverage.test.sh | 1 + .../scripts/audit-duplication.test.sh | 1 + .../audit-size/scripts/audit-size.test.sh | 1 + .../scripts/audit-type-debt.test.sh | 1 + .../skills/setup/scripts/setup-check.test.sh | 1 + .../scripts/allowed-tools-pairing.test.sh | 1 + .../scripts/evals-fixtures.test.sh | 1 + .../hooks/zone-crossing-inject.test.sh | 1 + plugins/discovery/agents/tool-honesty.test.sh | 1 + plugins/discovery/scripts/contract.test.sh | 1 + .../skills/clean/scripts/hygiene.test.sh | 1 + .../clean/scripts/owner_registry.test.sh | 1 + .../skills/clean/scripts/test_hygiene.py | 1 + .../setup/scripts/kill_switch_probe.test.sh | 1 + .../scripts/allowed-tools-pairing.test.sh | 1 + .../compress/scripts/detect-caveman.test.sh | 1 + .../scripts/allowed-tools-pairing.test.sh | 1 + .../scripts/calibrate-judge.test.sh | 1 + .../plugin-eval/scripts/run-validity.test.sh | 1 + .../validate/scripts/validate-cases.test.sh | 1 + plugins/github/github.test.sh | 1 + .../guardrails/hooks/abort-boundary.test.sh | 1 + .../hooks/coverage-manifest.test.sh | 1 + .../hooks/require-jq-notice-isolation.test.sh | 1 + .../hooks/require-jq-posture.test.sh | 1 + plugins/guardrails/hooks/run-guards.test.sh | 1 + .../scripts/inventory.test.sh | 1 + .../scripts/audit.test.sh | 1 + .../hooks/audit-session-id.test.sh | 1 + .../plugins/scripts/fleet-state.test.sh | 1 + .../skills/plugins/scripts/sync-run.test.sh | 1 + .../scripts/check-html-rows.test.sh | 1 + .../scripts/extract_blog_body.test.sh | 1 + .../scripts/allowed-tools-pairing.test.sh | 1 + plugins/multi-agent/tests/drift-audit.test.sh | 1 + plugins/pixel-art/scripts/backends.test.sh | 1 + plugins/planning/surface/surface.test.sh | 1 + plugins/planning/surface/test_exporters.py | 1 + plugins/planning/surface/test_round.py | 1 + plugins/planning/surface/test_schema.py | 1 + plugins/planning/surface/test_server.py | 1 + plugins/planning/surface/watch.test.sh | 1 + plugins/planning/tests/reattach-slice.test.sh | 1 + .../skills/playwright/scripts/update.test.sh | 1 + .../scripts/allowed-tools-pairing.test.sh | 1 + .../scripts/detect-ecosystems.test.sh | 1 + .../scripts/detect-ecosystems.test.sh | 1 + .../scripts/allowed-tools-pairing.test.sh | 1 + .../scripts/allowed-tools-pairing.test.sh | 1 + .../clean/scripts/destructive-guard.test.sh | 1 + plugins/retro-audio/scripts/audio.test.sh | 1 + plugins/review/tests/change-set-block.test.sh | 1 + .../session-flow/scripts/save_point.test.sh | 1 + .../scripts/audit-sessions.test.sh | 1 + .../scripts/babysit-wrapper-help.test.sh | 1 + .../skills/babysit-prs/scripts/engine.test.sh | 1 + .../babysit-prs/scripts/tests/test_guards.py | 1 + .../worktree/nesting-invariant-ssot.test.sh | 1 + plugins/speech/scripts/speech.test.sh | 1 + .../testing/scripts/gen-hook-filters.test.sh | 1 + .../skills/setup/scripts/setup.test.sh | 1 + .../no-hardcoded-priority-scheme.test.sh | 1 + .../adapters/gitea/list-items.test.sh | 1 + .../local-markdown/claim-integrity.test.sh | 1 + .../local-markdown/list-sub-items.test.sh | 1 + .../local-markdown/renew-lease.test.sh | 1 + .../conformance/bindings/jira.test.sh | 1 + .../bindings/local-markdown.test.sh | 1 + .../work-item-tracker.test.sh | 1 + scripts/affected-tests-scopes.txt | 121 ---------- scripts/affected-tests.sh | 211 ++++++------------ scripts/affected-tests.test.sh | 164 +++++--------- scripts/check-docs-only-gate.test.sh | 1 + scripts/check-hook-wiring-liveness.test.sh | 1 + scripts/check-lane-coverage.test.sh | 1 + scripts/check-loop-lane-floor-drift.test.sh | 1 + scripts/ci-fail-a-draft.test.sh | 1 + scripts/lib/gate-entry.test.sh | 1 + scripts/lib/test-harness.test.sh | 1 + scripts/validate-plugin-contracts.test.sh | 1 + 90 files changed, 218 insertions(+), 370 deletions(-) delete mode 100644 scripts/affected-tests-scopes.txt diff --git a/docs/ci-runner-routing.md b/docs/ci-runner-routing.md index 5deec0ed2c..a3d15554c0 100644 --- a/docs/ci-runner-routing.md +++ b/docs/ci-runner-routing.md @@ -116,9 +116,9 @@ four (four on the whole tree or an UNMAPPED file), and, per leg, whether its slice needs the animation wheels, the inventory's parser packages or the DuckDB CLI. A leg installs only those; the shfmt and DuckDB downloads are cached. -A suite that scans a directory never names the file that changed, so -`scripts/affected-tests-scopes.txt` declares what it reads, and the selector's -rule R8 selects it for any changed file matching the glob. The rules, and the +A suite that scans a directory never names the file that changed, so it +declares what it reads in a `# test-scope:` header, and the selector's rule R8 +selects it for any changed file matching the glob. The rules, and the `--replay` mode that shows a selector change's effect on recent main commits, are in the header of `scripts/affected-tests.sh`. diff --git a/plugins/actionlint/hooks/actionlint-check.test.sh b/plugins/actionlint/hooks/actionlint-check.test.sh index b0a8694879..1cf317d07e 100755 --- a/plugins/actionlint/hooks/actionlint-check.test.sh +++ b/plugins/actionlint/hooks/actionlint-check.test.sh @@ -12,6 +12,7 @@ # the caller's working directory would surface (the tool is file-anchored, so a # correct hook needs no cd). actionlint is required to drive the violation # assertions; without it the suite skips (the hook itself no-ops silently). +# test-scope: plugins/actionlint/hooks/* plugins/actionlint/.claude-plugin/plugin.json set -uo pipefail diff --git a/plugins/animation/scripts/animation.test.sh b/plugins/animation/scripts/animation.test.sh index 9d4c3a432a..5bf66ee36d 100755 --- a/plugins/animation/scripts/animation.test.sh +++ b/plugins/animation/scripts/animation.test.sh @@ -5,6 +5,7 @@ # ../requirements.txt hash-locks them) and skip without them, so ANIMATION_REQUIRE_DEPS=1 fails the run # instead: a lane that provisions the pinned requirements sets it, and missing dependencies then read as a # broken environment, not as passing coverage. +# test-scope: plugins/animation/skills/*/SKILL.md plugins/animation/skills/*/scripts/* plugins/animation/hooks/*.sh set -uo pipefail cd "$(dirname "${BASH_SOURCE[0]}")" || exit 1 diff --git a/plugins/autonomy/skills/setup/scripts/check-prerequisite-resolution-slice.test.sh b/plugins/autonomy/skills/setup/scripts/check-prerequisite-resolution-slice.test.sh index d7fa439d0b..3a22b2a460 100755 --- a/plugins/autonomy/skills/setup/scripts/check-prerequisite-resolution-slice.test.sh +++ b/plugins/autonomy/skills/setup/scripts/check-prerequisite-resolution-slice.test.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # Tests for the prerequisite-resolution setup slice wrappers. +# test-scope: plugins/autonomy/skills/setup/scripts/fixtures/prerequisite-resolution/* set -uo pipefail SELF_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.sh b/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.sh index 7a0798ead7..cd5533369f 100755 --- a/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.sh +++ b/plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.sh @@ -1,6 +1,7 @@ #!/usr/bin/env bash # Discovery wrapper: scripts/run-plugin-tests.sh finds plugins/**/*.test.sh, so # this hands off to the Node suite. SKIPs (exit 0) when Node is unavailable. +# test-scope: plugins/autonomy/skills/setup/evals/fixtures/security-binding/* set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/autonomy/skills/setup/scripts/generate-identity-prerequisites.test.sh b/plugins/autonomy/skills/setup/scripts/generate-identity-prerequisites.test.sh index 2c833d05c5..1c60e1df82 100755 --- a/plugins/autonomy/skills/setup/scripts/generate-identity-prerequisites.test.sh +++ b/plugins/autonomy/skills/setup/scripts/generate-identity-prerequisites.test.sh @@ -2,6 +2,7 @@ # Unit tests for generate-identity-prerequisites.mjs. Cases are named in the # co-located manifest; this harness builds throwaway trees where needed and # drives generate / --check / drift / leaf↔emission parity. +# test-scope: plugins/autonomy/reference/routines/*.md set -uo pipefail SELF_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/autonomy/skills/setup/scripts/resolve-prerequisites.fixtures.test.sh b/plugins/autonomy/skills/setup/scripts/resolve-prerequisites.fixtures.test.sh index 18c3e6561c..4134003d5e 100755 --- a/plugins/autonomy/skills/setup/scripts/resolve-prerequisites.fixtures.test.sh +++ b/plugins/autonomy/skills/setup/scripts/resolve-prerequisites.fixtures.test.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # Discovery wrapper: scripts/run-plugin-tests.sh finds plugins/**/*.test.sh. +# test-scope: plugins/autonomy/skills/setup/scripts/fixtures/prerequisite-resolution/* plugins/autonomy/generated/* set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/code-metrics/scripts/dispatch.test.sh b/plugins/code-metrics/scripts/dispatch.test.sh index caa12023a1..35f6aa5c81 100755 --- a/plugins/code-metrics/scripts/dispatch.test.sh +++ b/plugins/code-metrics/scripts/dispatch.test.sh @@ -4,6 +4,7 @@ # carries a fake `scc` that replays fixtures/tool-output/scc.json (design T13; # nothing executable is committed). The adapters themselves (scc.py, # line-counter.py) and report.py are exercised through the dispatcher. +# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* set -uo pipefail unset GIT_DIR GIT_WORK_TREE GIT_CONFIG diff --git a/plugins/code-metrics/scripts/tool-free-path.test.sh b/plugins/code-metrics/scripts/tool-free-path.test.sh index ecf7eeae68..92acc4eeb5 100755 --- a/plugins/code-metrics/scripts/tool-free-path.test.sh +++ b/plugins/code-metrics/scripts/tool-free-path.test.sh @@ -2,6 +2,7 @@ # Regression tests for tool-free-path.sh: the excluded set is derived from # the collector ladder, the filled directory keeps those collectors off PATH, # and the resolvable-collector check fails when one is put back. +# test-scope: plugins/code-metrics/scripts/collectors/*.py set -uo pipefail unset GIT_DIR GIT_WORK_TREE GIT_CONFIG diff --git a/plugins/code-metrics/skills/audit-complexity/scripts/audit-complexity.test.sh b/plugins/code-metrics/skills/audit-complexity/scripts/audit-complexity.test.sh index 2a70c36f1e..1ede4e811f 100755 --- a/plugins/code-metrics/skills/audit-complexity/scripts/audit-complexity.test.sh +++ b/plugins/code-metrics/skills/audit-complexity/scripts/audit-complexity.test.sh @@ -6,6 +6,7 @@ # PATH carries fake `lizard`, `radon`, `multimetric`, `gocognit` and # `shellmetrics` replaying the committed captures under # fixtures/tool-output/ (design T13; nothing executable is committed). +# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* set -uo pipefail unset GIT_DIR GIT_WORK_TREE GIT_CONFIG diff --git a/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh b/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh index 7132e29b4b..eea4ee7fbc 100755 --- a/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh +++ b/plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh @@ -16,6 +16,7 @@ # only, which is what makes Bash CRAP not-applicable). Nothing executable is # committed (design T13). No test command is ever run: this skill reads # artifacts. +# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/parsers/*.py plugins/code-metrics/scripts/fixtures/* set -uo pipefail unset GIT_DIR GIT_WORK_TREE GIT_CONFIG diff --git a/plugins/code-metrics/skills/audit-duplication/scripts/audit-duplication.test.sh b/plugins/code-metrics/skills/audit-duplication/scripts/audit-duplication.test.sh index 293a10f201..308f934f0c 100755 --- a/plugins/code-metrics/skills/audit-duplication/scripts/audit-duplication.test.sh +++ b/plugins/code-metrics/skills/audit-duplication/scripts/audit-duplication.test.sh @@ -20,6 +20,7 @@ # scripts/cross-plugin-source-registry.txt. It runs only when a real `jscpd` # already resolves on PATH, which this plugin never installs, and otherwise # prints a visible SKIP line. +# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* set -uo pipefail unset GIT_DIR GIT_WORK_TREE GIT_CONFIG diff --git a/plugins/code-metrics/skills/audit-size/scripts/audit-size.test.sh b/plugins/code-metrics/skills/audit-size/scripts/audit-size.test.sh index 3824e6d78b..d061f8af16 100755 --- a/plugins/code-metrics/skills/audit-size/scripts/audit-size.test.sh +++ b/plugins/code-metrics/skills/audit-size/scripts/audit-size.test.sh @@ -1,6 +1,7 @@ #!/usr/bin/env bash # Regression tests for the audit-size entry point (audit-size.sh): option parsing, # JSON versus markdown output, and exit-code passthrough from dispatch.sh. +# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* set -uo pipefail unset GIT_DIR GIT_WORK_TREE GIT_CONFIG diff --git a/plugins/code-metrics/skills/audit-type-debt/scripts/audit-type-debt.test.sh b/plugins/code-metrics/skills/audit-type-debt/scripts/audit-type-debt.test.sh index 050d9b0c3b..abd60dae32 100755 --- a/plugins/code-metrics/skills/audit-type-debt/scripts/audit-type-debt.test.sh +++ b/plugins/code-metrics/skills/audit-type-debt/scripts/audit-type-debt.test.sh @@ -13,6 +13,7 @@ # repository-relative path and passed relative, as the dispatcher passes a # scope in a real run, because the type-coverage capture names its file by # that relative path and a file row matches a scope file on the path. +# test-scope: plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* set -uo pipefail unset GIT_DIR GIT_WORK_TREE GIT_CONFIG diff --git a/plugins/code-metrics/skills/setup/scripts/setup-check.test.sh b/plugins/code-metrics/skills/setup/scripts/setup-check.test.sh index 389776e00d..b3318355a2 100755 --- a/plugins/code-metrics/skills/setup/scripts/setup-check.test.sh +++ b/plugins/code-metrics/skills/setup/scripts/setup-check.test.sh @@ -1,6 +1,7 @@ #!/usr/bin/env bash # Regression tests for setup-check.sh: the layer rows, the tracked-file guard, # the resolved references, and one row per collector adapter. +# test-scope: plugins/code-metrics/scripts/collectors/*.py set -uo pipefail unset GIT_DIR GIT_WORK_TREE GIT_CONFIG diff --git a/plugins/code-tidying/scripts/allowed-tools-pairing.test.sh b/plugins/code-tidying/scripts/allowed-tools-pairing.test.sh index a403cec32c..09b72a86dc 100755 --- a/plugins/code-tidying/scripts/allowed-tools-pairing.test.sh +++ b/plugins/code-tidying/scripts/allowed-tools-pairing.test.sh @@ -31,6 +31,7 @@ # fixed string searched for VERBATIM in markdown and frontmatter, where those # placeholders are substituted by Claude Code at load time. Letting the shell # expand any of them would make this gate silently match nothing. +# test-scope: plugins/code-tidying/skills/*.md plugins/code-tidying/skills/*/scripts/* # shellcheck disable=SC2016 set -uo pipefail diff --git a/plugins/code-tidying/scripts/evals-fixtures.test.sh b/plugins/code-tidying/scripts/evals-fixtures.test.sh index e88ee8c82e..6e7c4cf53c 100755 --- a/plugins/code-tidying/scripts/evals-fixtures.test.sh +++ b/plugins/code-tidying/scripts/evals-fixtures.test.sh @@ -5,6 +5,7 @@ # When tree-sitter (or its grammar) is absent the self-certify checks skip # visibly, matching test_change_shape.py. CI sets # CODE_TIDYING_REQUIRE_TREE_SITTER=1 so a missing dependency fails there. +# test-scope: plugins/code-tidying/evals/* set -euo pipefail here="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/context-guard/hooks/zone-crossing-inject.test.sh b/plugins/context-guard/hooks/zone-crossing-inject.test.sh index 596b37127b..4029511ddf 100755 --- a/plugins/context-guard/hooks/zone-crossing-inject.test.sh +++ b/plugins/context-guard/hooks/zone-crossing-inject.test.sh @@ -9,6 +9,7 @@ # # Self-contained: defines its own assertion helpers — installed plugins are # cache-isolated with no shared test lib. +# test-scope: plugins/context-guard/hooks/* set -uo pipefail diff --git a/plugins/discovery/agents/tool-honesty.test.sh b/plugins/discovery/agents/tool-honesty.test.sh index f4e200f7a6..017cd42488 100755 --- a/plugins/discovery/agents/tool-honesty.test.sh +++ b/plugins/discovery/agents/tool-honesty.test.sh @@ -16,6 +16,7 @@ # Scoped to plugins/discovery/agents/*.md on purpose. A sweep over every # plugin's agents would fail this plugin's test on another plugin's drift, which # reports the defect in the wrong place and blocks the wrong change. +# test-scope: plugins/discovery/agents/*.md set -uo pipefail cd "$(dirname "${BASH_SOURCE[0]}")" || exit 1 diff --git a/plugins/discovery/scripts/contract.test.sh b/plugins/discovery/scripts/contract.test.sh index 6f44a1ed13..8b960eebcb 100755 --- a/plugins/discovery/scripts/contract.test.sh +++ b/plugins/discovery/scripts/contract.test.sh @@ -19,6 +19,7 @@ # # SC2016 is disabled file-wide on purpose. Single-quoted `$ARGUMENTS` strings in # assertion labels and grep patterns are literal prose/regex under test. +# test-scope: plugins/discovery/*.md plugins/discovery/*.json # shellcheck disable=SC2016 set -uo pipefail diff --git a/plugins/disk-hygiene/skills/clean/scripts/hygiene.test.sh b/plugins/disk-hygiene/skills/clean/scripts/hygiene.test.sh index 1e387d70da..ec7eb53645 100755 --- a/plugins/disk-hygiene/skills/clean/scripts/hygiene.test.sh +++ b/plugins/disk-hygiene/skills/clean/scripts/hygiene.test.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # Cross-platform contract wrapper for the stdlib Python test suite. +# test-scope: plugins/disk-hygiene/skills/clean/SKILL.md plugins/disk-hygiene/skills/clean/reference/*.json plugins/disk-hygiene/hooks/* set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/disk-hygiene/skills/clean/scripts/owner_registry.test.sh b/plugins/disk-hygiene/skills/clean/scripts/owner_registry.test.sh index bfd30bb1e4..fa48f6419f 100755 --- a/plugins/disk-hygiene/skills/clean/scripts/owner_registry.test.sh +++ b/plugins/disk-hygiene/skills/clean/scripts/owner_registry.test.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # Cross-platform contract wrapper for the owner-registry test suite. +# test-scope: plugins/disk-hygiene/*.py plugins/disk-hygiene/*.sh plugins/disk-hygiene/*.mjs plugins/disk-hygiene/*.json set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/disk-hygiene/skills/clean/scripts/test_hygiene.py b/plugins/disk-hygiene/skills/clean/scripts/test_hygiene.py index 6cd3c89371..3a14270f5e 100755 --- a/plugins/disk-hygiene/skills/clean/scripts/test_hygiene.py +++ b/plugins/disk-hygiene/skills/clean/scripts/test_hygiene.py @@ -1,4 +1,5 @@ #!/usr/bin/env python3 +# test-scope: plugins/disk-hygiene/skills/clean/SKILL.md """Behavioral tests for the disk-hygiene safety engine and scoped guard.""" from __future__ import annotations diff --git a/plugins/disk-hygiene/skills/setup/scripts/kill_switch_probe.test.sh b/plugins/disk-hygiene/skills/setup/scripts/kill_switch_probe.test.sh index f724cc2164..a62aacd461 100755 --- a/plugins/disk-hygiene/skills/setup/scripts/kill_switch_probe.test.sh +++ b/plugins/disk-hygiene/skills/setup/scripts/kill_switch_probe.test.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # Cross-platform contract wrapper for the kill-switch probe test suite. +# test-scope: plugins/disk-hygiene/*.py plugins/disk-hygiene/*.sh plugins/disk-hygiene/*.mjs plugins/disk-hygiene/*.json set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/docs-hygiene/scripts/allowed-tools-pairing.test.sh b/plugins/docs-hygiene/scripts/allowed-tools-pairing.test.sh index 2c8223abc2..c91342a010 100755 --- a/plugins/docs-hygiene/scripts/allowed-tools-pairing.test.sh +++ b/plugins/docs-hygiene/scripts/allowed-tools-pairing.test.sh @@ -27,6 +27,7 @@ # fixed string searched for VERBATIM in markdown and frontmatter, where those # placeholders are substituted by Claude Code at load time. Letting the shell # expand any of them would make this gate silently match nothing. +# test-scope: plugins/docs-hygiene/skills/*.md plugins/docs-hygiene/skills/*/scripts/* # shellcheck disable=SC2016 set -uo pipefail diff --git a/plugins/docs-hygiene/skills/compress/scripts/detect-caveman.test.sh b/plugins/docs-hygiene/skills/compress/scripts/detect-caveman.test.sh index 75f00ea0e7..76263131d5 100755 --- a/plugins/docs-hygiene/skills/compress/scripts/detect-caveman.test.sh +++ b/plugins/docs-hygiene/skills/compress/scripts/detect-caveman.test.sh @@ -4,6 +4,7 @@ # Self-contained: no shared assertion lib; resolves the script under test # relative to this file. Asserts the always-exit-0 contract, both output # labels, and graceful degradation to "unknown" when claude/jq are absent. +# test-scope: .claude/settings.json set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/docs-naming/scripts/allowed-tools-pairing.test.sh b/plugins/docs-naming/scripts/allowed-tools-pairing.test.sh index 952a5f992d..34c603f484 100755 --- a/plugins/docs-naming/scripts/allowed-tools-pairing.test.sh +++ b/plugins/docs-naming/scripts/allowed-tools-pairing.test.sh @@ -27,6 +27,7 @@ # fixed string searched for VERBATIM in markdown and frontmatter, where those # placeholders are substituted by Claude Code at load time. Letting the shell # expand any of them would make this gate silently match nothing. +# test-scope: plugins/docs-naming/skills/*.md plugins/docs-naming/skills/*/scripts/* # shellcheck disable=SC2016 set -uo pipefail diff --git a/plugins/evals/skills/plugin-eval/scripts/calibrate-judge.test.sh b/plugins/evals/skills/plugin-eval/scripts/calibrate-judge.test.sh index 231303fb36..13be9960cf 100755 --- a/plugins/evals/skills/plugin-eval/scripts/calibrate-judge.test.sh +++ b/plugins/evals/skills/plugin-eval/scripts/calibrate-judge.test.sh @@ -5,6 +5,7 @@ # # Exit: 0 all tests passed; 1 a test failed; 2 no usable interpreter (a named # environment error, never a silent skip). +# test-scope: plugins/evals/evals/* plugins/evals/skills/plugin-eval/scripts/fixtures/calibrate-judge/* set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/evals/skills/plugin-eval/scripts/run-validity.test.sh b/plugins/evals/skills/plugin-eval/scripts/run-validity.test.sh index b6ed470ea3..513b624580 100755 --- a/plugins/evals/skills/plugin-eval/scripts/run-validity.test.sh +++ b/plugins/evals/skills/plugin-eval/scripts/run-validity.test.sh @@ -5,6 +5,7 @@ # # Exit: 0 all tests passed; 1 a test failed; 2 no usable interpreter (a named # environment error, never a silent skip). +# test-scope: plugins/evals/skills/plugin-eval/scripts/fixtures/run-validity/* set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/evals/skills/validate/scripts/validate-cases.test.sh b/plugins/evals/skills/validate/scripts/validate-cases.test.sh index a5d8d622c6..d35131ea4d 100755 --- a/plugins/evals/skills/validate/scripts/validate-cases.test.sh +++ b/plugins/evals/skills/validate/scripts/validate-cases.test.sh @@ -7,6 +7,7 @@ # # Exit: 0 all tests passed; 1 a test failed; 2 no usable interpreter (a named # environment error, never a silent skip). +# test-scope: plugins/evals/evals/* set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/github/github.test.sh b/plugins/github/github.test.sh index 0f958be0d3..76686b3f9c 100755 --- a/plugins/github/github.test.sh +++ b/plugins/github/github.test.sh @@ -7,6 +7,7 @@ # (fixture lives here, independent of the file it checks) # - recipe non-hollow contract: every primary-tier recipe carries the six contract sections # and a >=10-question audit checklist +# test-scope: plugins/github/* set -uo pipefail # mapfile (area oracle below) needs bash >= 4; on bash 3.x it fails silently under diff --git a/plugins/guardrails/hooks/abort-boundary.test.sh b/plugins/guardrails/hooks/abort-boundary.test.sh index 27b00eba8b..fa5b4cd4a8 100755 --- a/plugins/guardrails/hooks/abort-boundary.test.sh +++ b/plugins/guardrails/hooks/abort-boundary.test.sh @@ -6,6 +6,7 @@ # code, stderr, and the stdout document. The registered set is read from # hooks.json, never enumerated here, so a hook added without the boundary # fails this suite. Self-contained; no host-repo assertion library. +# test-scope: plugins/guardrails/hooks/* plugins/guardrails/lib/* set -uo pipefail diff --git a/plugins/guardrails/hooks/coverage-manifest.test.sh b/plugins/guardrails/hooks/coverage-manifest.test.sh index 2dd5f76b0a..c657542883 100755 --- a/plugins/guardrails/hooks/coverage-manifest.test.sh +++ b/plugins/guardrails/hooks/coverage-manifest.test.sh @@ -6,6 +6,7 @@ # is registered, every event is one hooks.json declares, every family and # pattern is one the harness-config audit baseline actually lists, and every # lever is a documented option. +# test-scope: plugins/guardrails/README.md set -uo pipefail HOOK_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/guardrails/hooks/require-jq-notice-isolation.test.sh b/plugins/guardrails/hooks/require-jq-notice-isolation.test.sh index 44cecfaba4..b5384bac11 100755 --- a/plugins/guardrails/hooks/require-jq-notice-isolation.test.sh +++ b/plugins/guardrails/hooks/require-jq-notice-isolation.test.sh @@ -12,6 +12,7 @@ # catches this: each isolates its own CLAUDE_PLUGIN_DATA/session, so two # different guardrails hooks sharing one session (the real-world condition) # is never exercised. This test simulates exactly that. +# test-scope: plugins/guardrails/hooks/*.sh set -uo pipefail HOOK_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/guardrails/hooks/require-jq-posture.test.sh b/plugins/guardrails/hooks/require-jq-posture.test.sh index a931739e24..616650a1a8 100755 --- a/plugins/guardrails/hooks/require-jq-posture.test.sh +++ b/plugins/guardrails/hooks/require-jq-posture.test.sh @@ -39,6 +39,7 @@ # that real jq-removal "is not portably simulable" via an isolated bin dir # (which cannot host bash + coreutils across Git Bash and Linux). That is true # of the bin-dir approach, and is exactly why this one overrides the lookup. +# test-scope: plugins/guardrails/hooks/*.sh set -uo pipefail diff --git a/plugins/guardrails/hooks/run-guards.test.sh b/plugins/guardrails/hooks/run-guards.test.sh index 442513237c..2f540d1abc 100755 --- a/plugins/guardrails/hooks/run-guards.test.sh +++ b/plugins/guardrails/hooks/run-guards.test.sh @@ -15,6 +15,7 @@ # # The stub guard bodies below are single-quoted on purpose: they are written # verbatim into stub scripts, so their `$` must not expand here. +# test-scope: plugins/guardrails/hooks/*.sh # shellcheck disable=SC2016 set -uo pipefail diff --git a/plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh b/plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh index b64a1b52d6..fcd6d104d0 100755 --- a/plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh +++ b/plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh @@ -11,6 +11,7 @@ # asserted against a fixture built to a known size, as an exact value and never # as "more than zero", and each hook-location row is pinned across all five of # its fixed columns at once by row6. +# test-scope: plugins/*/.mcp.json plugins/harness-config/skills/*/SKILL.md .claude/settings.json set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/harness-config/skills/audit-permission-state/scripts/audit.test.sh b/plugins/harness-config/skills/audit-permission-state/scripts/audit.test.sh index 51cf3b4149..d9161c3f6a 100755 --- a/plugins/harness-config/skills/audit-permission-state/scripts/audit.test.sh +++ b/plugins/harness-config/skills/audit-permission-state/scripts/audit.test.sh @@ -8,6 +8,7 @@ # # Stage behavior itself belongs to each stage's own suite. Nothing here asserts a # finding. +# test-scope: .claude/settings.json set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/harness-ops/hooks/audit-session-id.test.sh b/plugins/harness-ops/hooks/audit-session-id.test.sh index 99247a2885..ae8abe1792 100755 --- a/plugins/harness-ops/hooks/audit-session-id.test.sh +++ b/plugins/harness-ops/hooks/audit-session-id.test.sh @@ -8,6 +8,7 @@ # on hook_event_name, so every payload here carries that key as Claude Code # sends it; skill-usage-audit.sh and hook-failure-audit.sh keep their own # scripts. +# test-scope: plugins/harness-ops/hooks/*-audit.sh set -uo pipefail HOOK_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/harness-ops/skills/plugins/scripts/fleet-state.test.sh b/plugins/harness-ops/skills/plugins/scripts/fleet-state.test.sh index e846a24c8a..e121f33c79 100755 --- a/plugins/harness-ops/skills/plugins/scripts/fleet-state.test.sh +++ b/plugins/harness-ops/skills/plugins/scripts/fleet-state.test.sh @@ -2,6 +2,7 @@ # Black-box contract tests for fleet-state.sh (self-contained — ships with the plugin). # Fixtures are built per-case into a temp dir via FLEET_STATE_* env overrides, # mirroring the harness-config audit skill's SETTINGS_AUDIT_FIXTURE_DIR pattern. +# test-scope: .claude/settings.json set -uo pipefail # Fixture git isolation: an inherited GIT_DIR/GIT_WORK_TREE/GIT_CONFIG would diff --git a/plugins/harness-ops/skills/plugins/scripts/sync-run.test.sh b/plugins/harness-ops/skills/plugins/scripts/sync-run.test.sh index f0ae8ce3c8..bd9bd82db6 100755 --- a/plugins/harness-ops/skills/plugins/scripts/sync-run.test.sh +++ b/plugins/harness-ops/skills/plugins/scripts/sync-run.test.sh @@ -5,6 +5,7 @@ # and its exit status is the production one. Only the two siblings whose real work # needs a machine — the `claude` CLI and cache-content-check.sh's git compare — are # stubbed, through sync-run.sh's own SYNC_RUN_* overrides. +# test-scope: .claude/settings.json set -uo pipefail # Fixture git isolation: an inherited GIT_DIR/GIT_WORK_TREE/GIT_CONFIG would diff --git a/plugins/knowledge/skills/docpage-digest/scripts/check-html-rows.test.sh b/plugins/knowledge/skills/docpage-digest/scripts/check-html-rows.test.sh index 58edcbd392..7be581d55f 100755 --- a/plugins/knowledge/skills/docpage-digest/scripts/check-html-rows.test.sh +++ b/plugins/knowledge/skills/docpage-digest/scripts/check-html-rows.test.sh @@ -1,6 +1,7 @@ #!/usr/bin/env bash # Cross-platform wrapper so plugin-gate (plugins/**/*.test.sh) runs the # check-html-rows negative-control suite. +# test-scope: plugins/knowledge/skills/docpage-digest/scripts/fixtures/html-rows/* set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/knowledge/skills/docpage-digest/scripts/extract_blog_body.test.sh b/plugins/knowledge/skills/docpage-digest/scripts/extract_blog_body.test.sh index 3883d16f9a..bc4ef87fc4 100755 --- a/plugins/knowledge/skills/docpage-digest/scripts/extract_blog_body.test.sh +++ b/plugins/knowledge/skills/docpage-digest/scripts/extract_blog_body.test.sh @@ -1,6 +1,7 @@ #!/usr/bin/env bash # Cross-platform wrapper so plugin-gate (plugins/**/*.test.sh) runs the # extract_blog_body fixture suite. +# test-scope: plugins/knowledge/skills/docpage-digest/scripts/fixtures/blog-body.html set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/multi-agent/scripts/allowed-tools-pairing.test.sh b/plugins/multi-agent/scripts/allowed-tools-pairing.test.sh index 27f1910cc2..753939de01 100755 --- a/plugins/multi-agent/scripts/allowed-tools-pairing.test.sh +++ b/plugins/multi-agent/scripts/allowed-tools-pairing.test.sh @@ -25,6 +25,7 @@ # fixed string searched for VERBATIM in markdown and frontmatter, where those # placeholders are substituted by Claude Code at load time. Letting the shell # expand any of them would make this gate silently match nothing. +# test-scope: plugins/multi-agent/skills/*.md plugins/multi-agent/skills/*/scripts/* # shellcheck disable=SC2016 set -uo pipefail diff --git a/plugins/multi-agent/tests/drift-audit.test.sh b/plugins/multi-agent/tests/drift-audit.test.sh index 757ba51988..ae853c471f 100755 --- a/plugins/multi-agent/tests/drift-audit.test.sh +++ b/plugins/multi-agent/tests/drift-audit.test.sh @@ -1,6 +1,7 @@ #!/usr/bin/env bash # Discovery wrapper: scripts/run-plugin-tests.sh finds plugins/**/*.test.sh, so # this hands off to the Node suite. SKIPs (exit 0) when Node is unavailable. +# test-scope: plugins/multi-agent/agents/*.md set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/pixel-art/scripts/backends.test.sh b/plugins/pixel-art/scripts/backends.test.sh index aed114640d..fe758f3557 100755 --- a/plugins/pixel-art/scripts/backends.test.sh +++ b/plugins/pixel-art/scripts/backends.test.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # Runs every pixel-art test suite (test_*.py). +# test-scope: plugins/pixel-art/examples/* plugins/pixel-art/palettes/* set -uo pipefail cd "$(dirname "${BASH_SOURCE[0]}")" || exit 1 diff --git a/plugins/planning/surface/surface.test.sh b/plugins/planning/surface/surface.test.sh index eaf24b6c3a..9ed77f8f04 100755 --- a/plugins/planning/surface/surface.test.sh +++ b/plugins/planning/surface/surface.test.sh @@ -7,6 +7,7 @@ # schema/visual.schema.json and schema/ops.schema.json. # The browser suites run only where playwright-cli resolves; elsewhere they print a SKIP with # the number of checks not run, never a pass. +# test-scope: plugins/planning/surface/schema/* plugins/planning/surface/tests/fixtures/* set -u here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) root=$(cd "$here/../../.." && pwd) diff --git a/plugins/planning/surface/test_exporters.py b/plugins/planning/surface/test_exporters.py index dc51e3d01f..e07cae1e83 100644 --- a/plugins/planning/surface/test_exporters.py +++ b/plugins/planning/surface/test_exporters.py @@ -1,3 +1,4 @@ +# test-scope: plugins/planning/surface/tests/fixtures/ledger-legacy/* """Tests for the exporters and import-ledger (AC27 to AC29), driven through round.py's CLI. Sessions are built in temporary data dirs: questions.json written directly, responses.json diff --git a/plugins/planning/surface/test_round.py b/plugins/planning/surface/test_round.py index 2201505e7a..37246550f6 100644 --- a/plugins/planning/surface/test_round.py +++ b/plugins/planning/surface/test_round.py @@ -1,3 +1,4 @@ +# test-scope: plugins/planning/surface/tests/fixtures/*.json """Tests for round.py V1: schema validation, refusals and warnings, --affects, apply, archive, status --latency, the sidecar lock and the rebuild check. diff --git a/plugins/planning/surface/test_schema.py b/plugins/planning/surface/test_schema.py index 3b9bcc9c74..4233502a4c 100644 --- a/plugins/planning/surface/test_schema.py +++ b/plugins/planning/surface/test_schema.py @@ -1,3 +1,4 @@ +# test-scope: plugins/planning/surface/tests/fixtures/*.json """Tests for schema.py, the stdlib JSON Schema subset round.py validates with.""" from __future__ import annotations diff --git a/plugins/planning/surface/test_server.py b/plugins/planning/surface/test_server.py index ebbe72f8b3..dce77257e7 100644 --- a/plugins/planning/surface/test_server.py +++ b/plugins/planning/surface/test_server.py @@ -1,3 +1,4 @@ +# test-scope: plugins/planning/surface/tests/fixtures/*.json """Tests for the interview surface server and its lifecycle commands. Every class starts its own server through `round.py ensure-running --port 0` in a diff --git a/plugins/planning/surface/watch.test.sh b/plugins/planning/surface/watch.test.sh index 853bffb9a0..fbfdc84663 100755 --- a/plugins/planning/surface/watch.test.sh +++ b/plugins/planning/surface/watch.test.sh @@ -9,6 +9,7 @@ # (WAIT_FAILS=1), the one-watcher lease (a second watcher exits 3 naming the holder; a stale # lease is reclaimed after leaseTimeout), the fallback watcher id and a release mid-wait, and # round.sh stop ending the data dir's watch.sh (the lease records its pid). +# test-scope: plugins/planning/surface/schema/* set -u # Every watcher in the suite is one session unless a case names another. export WATCH_ID=suite diff --git a/plugins/planning/tests/reattach-slice.test.sh b/plugins/planning/tests/reattach-slice.test.sh index 39ea76aa65..e815c9a382 100755 --- a/plugins/planning/tests/reattach-slice.test.sh +++ b/plugins/planning/tests/reattach-slice.test.sh @@ -8,6 +8,7 @@ # # shellcheck disable=SC2016 # single quotes are deliberate: assert_gate phrases are verbatim # markdown lines that hold literal backticks, and expanding them would break the match. +# test-scope: plugins/planning/skills/*/SKILL.md set -uo pipefail PLUGIN_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" diff --git a/plugins/playwright/skills/playwright/scripts/update.test.sh b/plugins/playwright/skills/playwright/scripts/update.test.sh index 7917a60d89..e6761284b3 100755 --- a/plugins/playwright/skills/playwright/scripts/update.test.sh +++ b/plugins/playwright/skills/playwright/scripts/update.test.sh @@ -2,6 +2,7 @@ # Self-contained regression tests for update.sh (no external test lib — ships # with the plugin; network-free: exercises help, arg handling, and the sourced # helper functions against local fixtures only). +# test-scope: plugins/playwright/skills/playwright/SKILL.md set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/prototype/scripts/allowed-tools-pairing.test.sh b/plugins/prototype/scripts/allowed-tools-pairing.test.sh index f79f34a302..076987a968 100755 --- a/plugins/prototype/scripts/allowed-tools-pairing.test.sh +++ b/plugins/prototype/scripts/allowed-tools-pairing.test.sh @@ -27,6 +27,7 @@ # fixed string searched for VERBATIM in markdown and frontmatter, where those # placeholders are substituted by Claude Code at load time. Letting the shell # expand any of them would make this gate silently match nothing. +# test-scope: plugins/prototype/skills/*.md plugins/prototype/skills/*/scripts/* # shellcheck disable=SC2016 set -uo pipefail diff --git a/plugins/prototype/skills/explore-directions/scripts/detect-ecosystems.test.sh b/plugins/prototype/skills/explore-directions/scripts/detect-ecosystems.test.sh index b6ba98fd8f..2ae4e8d3df 100755 --- a/plugins/prototype/skills/explore-directions/scripts/detect-ecosystems.test.sh +++ b/plugins/prototype/skills/explore-directions/scripts/detect-ecosystems.test.sh @@ -26,6 +26,7 @@ # allowed-tools-pairing suite disables it. Every single-quoted `${…}` below is a # fixed string searched for VERBATIM in frontmatter or in the wrapper's source # text. Letting the shell expand one would make the assertion match nothing. +# test-scope: plugins/prototype/skills/explore-directions/SKILL.md # shellcheck disable=SC2016 set -uo pipefail diff --git a/plugins/prototype/skills/pressure-test/scripts/detect-ecosystems.test.sh b/plugins/prototype/skills/pressure-test/scripts/detect-ecosystems.test.sh index a5b118c3d2..97bf178834 100755 --- a/plugins/prototype/skills/pressure-test/scripts/detect-ecosystems.test.sh +++ b/plugins/prototype/skills/pressure-test/scripts/detect-ecosystems.test.sh @@ -26,6 +26,7 @@ # allowed-tools-pairing suite disables it. Every single-quoted `${…}` below is a # fixed string searched for VERBATIM in frontmatter or in the wrapper's source # text. Letting the shell expand one would make the assertion match nothing. +# test-scope: plugins/prototype/skills/pressure-test/SKILL.md # shellcheck disable=SC2016 set -uo pipefail diff --git a/plugins/repo-fleet-hygiene/scripts/allowed-tools-pairing.test.sh b/plugins/repo-fleet-hygiene/scripts/allowed-tools-pairing.test.sh index caf400262e..5f8a289309 100755 --- a/plugins/repo-fleet-hygiene/scripts/allowed-tools-pairing.test.sh +++ b/plugins/repo-fleet-hygiene/scripts/allowed-tools-pairing.test.sh @@ -27,6 +27,7 @@ # fixed string searched for VERBATIM in markdown and frontmatter, where those # placeholders are substituted by Claude Code at load time. Letting the shell # expand any of them would make this gate silently match nothing. +# test-scope: plugins/repo-fleet-hygiene/skills/*.md plugins/repo-fleet-hygiene/skills/*/scripts/* # shellcheck disable=SC2016 set -uo pipefail diff --git a/plugins/repo-hygiene/scripts/allowed-tools-pairing.test.sh b/plugins/repo-hygiene/scripts/allowed-tools-pairing.test.sh index 8e12a28cd2..347ff36f0e 100755 --- a/plugins/repo-hygiene/scripts/allowed-tools-pairing.test.sh +++ b/plugins/repo-hygiene/scripts/allowed-tools-pairing.test.sh @@ -21,6 +21,7 @@ # fixed string searched for VERBATIM in markdown and frontmatter, where those # placeholders are substituted by Claude Code at load time. Letting the shell # expand any of them would make this gate silently match nothing. +# test-scope: plugins/repo-hygiene/skills/*.md plugins/repo-hygiene/skills/*/scripts/* # shellcheck disable=SC2016 set -uo pipefail diff --git a/plugins/repo-hygiene/skills/clean/scripts/destructive-guard.test.sh b/plugins/repo-hygiene/skills/clean/scripts/destructive-guard.test.sh index e267ad46d2..8f2aaaa1d3 100755 --- a/plugins/repo-hygiene/skills/clean/scripts/destructive-guard.test.sh +++ b/plugins/repo-hygiene/skills/clean/scripts/destructive-guard.test.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # Regression tests for destructive-guard.sh. +# test-scope: plugins/repo-hygiene/skills/clean/SKILL.md set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/retro-audio/scripts/audio.test.sh b/plugins/retro-audio/scripts/audio.test.sh index 3e860248a4..8ec1d82b64 100755 --- a/plugins/retro-audio/scripts/audio.test.sh +++ b/plugins/retro-audio/scripts/audio.test.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # Contract tests for retro-audio WAV rendering. +# test-scope: plugins/retro-audio/examples/* set -uo pipefail cd "$(dirname "${BASH_SOURCE[0]}")" || exit 1 diff --git a/plugins/review/tests/change-set-block.test.sh b/plugins/review/tests/change-set-block.test.sh index 31db933001..ea67b966e1 100755 --- a/plugins/review/tests/change-set-block.test.sh +++ b/plugins/review/tests/change-set-block.test.sh @@ -3,6 +3,7 @@ # agents/code-reviewer.md step 2, asserts the other reviewer agents carry the # same text, and runs it in scratch repos. It proves the block's output only, # not any model's decline-to-grade behavior. +# test-scope: plugins/review/agents/*.md set -uo pipefail unset GIT_DIR GIT_WORK_TREE GIT_INDEX_FILE GIT_COMMON_DIR GIT_CONFIG diff --git a/plugins/session-flow/scripts/save_point.test.sh b/plugins/session-flow/scripts/save_point.test.sh index 559565c082..6d8b62572b 100755 --- a/plugins/session-flow/scripts/save_point.test.sh +++ b/plugins/session-flow/scripts/save_point.test.sh @@ -3,6 +3,7 @@ # # SKIPs (exit 0) when Python 3.10+ or pytest is unavailable, matching the # repo test-runner convention for optional toolchains. +# test-scope: plugins/session-flow/scripts/tests/fixtures/* set -uo pipefail cd "$(dirname "${BASH_SOURCE[0]}")" || exit 1 diff --git a/plugins/session-flow/skills/audit-sessions/scripts/audit-sessions.test.sh b/plugins/session-flow/skills/audit-sessions/scripts/audit-sessions.test.sh index 77fc0087f4..60ab45d538 100755 --- a/plugins/session-flow/skills/audit-sessions/scripts/audit-sessions.test.sh +++ b/plugins/session-flow/skills/audit-sessions/scripts/audit-sessions.test.sh @@ -4,6 +4,7 @@ # SKIPs (exit 0) when Python 3.10+ or pytest is unavailable, matching the # repo test-runner convention for optional toolchains. -p no:cacheprovider keeps # a .pytest_cache/README.md out of the plugin-wide markdownlint glob. +# test-scope: plugins/session-flow/skills/audit-sessions/scripts/tests/fixtures/* set -uo pipefail cd "$(dirname "${BASH_SOURCE[0]}")" || exit 1 diff --git a/plugins/source-control/scripts/babysit-wrapper-help.test.sh b/plugins/source-control/scripts/babysit-wrapper-help.test.sh index dc5a34b288..2cd027dcda 100755 --- a/plugins/source-control/scripts/babysit-wrapper-help.test.sh +++ b/plugins/source-control/scripts/babysit-wrapper-help.test.sh @@ -8,6 +8,7 @@ # -- reaching no network, needing no allowlist, and exiting 0. If it ever # regressed to a live call, a read-only `check` run would start touching the # fleet it was asked to inspect. So the canary's harmlessness is pinned here. +# test-scope: plugins/source-control/scripts/* plugins/source-control/skills/babysit-prs/scripts/babysit_*.py set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/source-control/skills/babysit-prs/scripts/engine.test.sh b/plugins/source-control/skills/babysit-prs/scripts/engine.test.sh index db6ec4f120..359440834b 100755 --- a/plugins/source-control/skills/babysit-prs/scripts/engine.test.sh +++ b/plugins/source-control/skills/babysit-prs/scripts/engine.test.sh @@ -5,6 +5,7 @@ # the guarded wrappers (whose --allow-unpinned-head rejection is a shell # concern, not a Python one). SKIPs (exit 0) when Python 3.11+ is unavailable, matching the # repo test-runner convention for optional toolchains. +# test-scope: plugins/source-control/*.md plugins/source-control/scripts/babysit-*.sh set -uo pipefail # Resolve before the cd: BASH_SOURCE is the path as invoked, so re-deriving it diff --git a/plugins/source-control/skills/babysit-prs/scripts/tests/test_guards.py b/plugins/source-control/skills/babysit-prs/scripts/tests/test_guards.py index 0a773cf64e..c2cf2b10cc 100644 --- a/plugins/source-control/skills/babysit-prs/scripts/tests/test_guards.py +++ b/plugins/source-control/skills/babysit-prs/scripts/tests/test_guards.py @@ -1,3 +1,4 @@ +# test-scope: plugins/source-control/skills/*/SKILL.md """Executes the guard contract in `guard_contract.py` against the real entry points. Every assertion here exercises a fact a consumer is told to rely on, and fails diff --git a/plugins/source-control/skills/worktree/nesting-invariant-ssot.test.sh b/plugins/source-control/skills/worktree/nesting-invariant-ssot.test.sh index 6fe4dcd476..12b52489e3 100755 --- a/plugins/source-control/skills/worktree/nesting-invariant-ssot.test.sh +++ b/plugins/source-control/skills/worktree/nesting-invariant-ssot.test.sh @@ -17,6 +17,7 @@ # check against the stamp's version arm. The suite never probes the host CLI, # so the result does not depend on where it runs; unset skips that arm with a # counted skip. +# test-scope: plugins/source-control/* set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/plugins/speech/scripts/speech.test.sh b/plugins/speech/scripts/speech.test.sh index 3f66bbf01e..710dbde046 100755 --- a/plugins/speech/scripts/speech.test.sh +++ b/plugins/speech/scripts/speech.test.sh @@ -5,6 +5,7 @@ # hash-locks it) and skip without it, so SPEECH_REQUIRE_DEPS=1 fails the run instead: a lane that provisions the # pinned requirements sets it, and a missing numpy then reads as a broken environment, not as passing coverage. # SPEECH_E2E_DATA_DIR opts into the real end-to-end narration (see test_narrate.py). +# test-scope: plugins/speech/skills/*/SKILL.md plugins/speech/hooks/*.sh plugins/speech/scripts/*.json set -uo pipefail cd "$(dirname "${BASH_SOURCE[0]}")" || exit 1 diff --git a/plugins/testing/scripts/gen-hook-filters.test.sh b/plugins/testing/scripts/gen-hook-filters.test.sh index ed75688c81..3393be9a3f 100755 --- a/plugins/testing/scripts/gen-hook-filters.test.sh +++ b/plugins/testing/scripts/gen-hook-filters.test.sh @@ -2,6 +2,7 @@ # Test for gen-hook-filters.sh: the shipped hooks.json is in sync with the # adapters, every row is gated by an `if`, no row matches a non-test path, no # glob repeats, and --check catches drift. +# test-scope: plugins/testing/skills/audit/adapters/*.yaml # shellcheck disable=SC2016 # check() evals its single-quoted condition set -uo pipefail diff --git a/plugins/testing/skills/setup/scripts/setup.test.sh b/plugins/testing/skills/setup/scripts/setup.test.sh index 6abfa8aa6b..ae331f3f2e 100755 --- a/plugins/testing/skills/setup/scripts/setup.test.sh +++ b/plugins/testing/skills/setup/scripts/setup.test.sh @@ -1,6 +1,7 @@ #!/usr/bin/env bash # Tests for setup.sh: check's four sections, lint findings, the consumer hook # entry, and an apply that writes only the docs convention file or .claude/testing.yaml. +# test-scope: plugins/testing/skills/audit/adapters/*.yaml # shellcheck disable=SC2016 # fence lines in fixtures are literal text set -uo pipefail unset GIT_DIR GIT_WORK_TREE GIT_CONFIG CLAUDE_PROJECT_DIR diff --git a/plugins/work-items/tests/no-hardcoded-priority-scheme.test.sh b/plugins/work-items/tests/no-hardcoded-priority-scheme.test.sh index 84c5a21647..03520d6a2a 100755 --- a/plugins/work-items/tests/no-hardcoded-priority-scheme.test.sh +++ b/plugins/work-items/tests/no-hardcoded-priority-scheme.test.sh @@ -1,6 +1,7 @@ #!/usr/bin/env bash # Regression guard for #1253: plugin prose must never name a `pN-*` priority value, # qualified or bare. CHANGELOG.md is exempt as a historical record. +# test-scope: plugins/work-items/*.md # shellcheck disable=SC2016 # fixture bodies are literal prose in single quotes; expansion is never wanted set -uo pipefail diff --git a/plugins/work-items/tools/work-item-tracker/adapters/gitea/list-items.test.sh b/plugins/work-items/tools/work-item-tracker/adapters/gitea/list-items.test.sh index 564cd5b494..90eecf3bb2 100755 --- a/plugins/work-items/tools/work-item-tracker/adapters/gitea/list-items.test.sh +++ b/plugins/work-items/tools/work-item-tracker/adapters/gitea/list-items.test.sh @@ -2,6 +2,7 @@ # shellcheck disable=SC2154 # FAILED/CASE_NUM initialized by the sourced helper # list-items: offline contract tests. Pagination, PR exclusion, and the per-item # blocker count are all driven through a mocked curl — no live Gitea call. +# test-scope: plugins/work-items/tools/work-item-tracker/adapters/gitea/* set -uo pipefail S="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/list-items.sh" D="$(dirname "$S")" diff --git a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh index 154854a999..27dd47fe73 100755 --- a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh +++ b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh @@ -5,6 +5,7 @@ # field). Covers: (1) an expired lease returns the item to the frontier even though # this adapter has no reclaim, and (2) a failed assignee write fails the claim and # rolls the just-appended lease marker back, leaving no orphaned marker. +# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* # shellcheck disable=SC2154 # FAILED/CASE_NUM initialized by the sourced lib set -uo pipefail diff --git a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh index 0dc3658918..8b3cfec61f 100755 --- a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh +++ b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh @@ -3,6 +3,7 @@ # local-markdown store: direct-child enumeration with state # filtering and parent stamping, the container-scoped frontier (list-frontier # --parent), and container exclusion from every frontier. Runs in CI, offline. +# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* # shellcheck disable=SC2154 # FAILED/CASE_NUM initialized by the sourced lib set -uo pipefail diff --git a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh index d2364cc708..5924a916af 100755 --- a/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh +++ b/plugins/work-items/tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash # shellcheck disable=SC2154 # FAILED/CASE_NUM initialized by the sourced helper +# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* set -uo pipefail TMP_ROOT="$(mktemp -d)" diff --git a/plugins/work-items/tools/work-item-tracker/conformance/bindings/jira.test.sh b/plugins/work-items/tools/work-item-tracker/conformance/bindings/jira.test.sh index 70436cf335..26981e6765 100755 --- a/plugins/work-items/tools/work-item-tracker/conformance/bindings/jira.test.sh +++ b/plugins/work-items/tools/work-item-tracker/conformance/bindings/jira.test.sh @@ -2,6 +2,7 @@ # shellcheck disable=SC2154 # FAILED/CASE_NUM initialized by the sourced lib # RUNS the full abstract suite against the consume-only jira adapter, once normally and # once under a PATH shim that makes gh/curl fail: every exercised path is pre-network. +# test-scope: plugins/work-items/tools/work-item-tracker/adapters/jira/* set -uo pipefail TMP_ROOT="$(mktemp -d)" diff --git a/plugins/work-items/tools/work-item-tracker/conformance/bindings/local-markdown.test.sh b/plugins/work-items/tools/work-item-tracker/conformance/bindings/local-markdown.test.sh index a731ba54d1..34037bafc5 100755 --- a/plugins/work-items/tools/work-item-tracker/conformance/bindings/local-markdown.test.sh +++ b/plugins/work-items/tools/work-item-tracker/conformance/bindings/local-markdown.test.sh @@ -2,6 +2,7 @@ # shellcheck disable=SC2154 # FAILED/CASE_NUM initialized by the sourced lib # RUNS the full abstract conformance suite offline against the local-markdown adapter, # once normally and once under a PATH shim that makes gh/curl fail. +# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* set -uo pipefail TMP_ROOT="$(mktemp -d)" diff --git a/plugins/work-items/tools/work-item-tracker/work-item-tracker.test.sh b/plugins/work-items/tools/work-item-tracker/work-item-tracker.test.sh index b8cf884b8f..19a1fbe21b 100755 --- a/plugins/work-items/tools/work-item-tracker/work-item-tracker.test.sh +++ b/plugins/work-items/tools/work-item-tracker/work-item-tracker.test.sh @@ -1,6 +1,7 @@ #!/usr/bin/env bash # Tests for the core dispatcher: usage, binding resolution, capability gating, and # list-frontier derivation — all against a fake adapter (no network, no gh). +# test-scope: plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/scripts/affected-tests-scopes.txt b/scripts/affected-tests-scopes.txt deleted file mode 100644 index a5ea00b9f9..0000000000 --- a/scripts/affected-tests-scopes.txt +++ /dev/null @@ -1,121 +0,0 @@ -# Declared test scopes, read by scripts/affected-tests.sh (rule R8). A suite -# listed here reads files it never names: it greps or globs a directory, copies -# a tree, or builds a path from parts. A changed file matching one of its globs -# selects the suite and counts as mapped. -# -# Format: one ` [...]` line per suite. Each glob is matched -# against the repo-relative path with bash pattern matching, so `*` crosses -# `/`, as in scripts/affected-tests-no-suite.txt. Blank lines and `#` comments -# are ignored. An entry naming no suite fails every selector run, and a glob -# matching no file fails the run that changes this list. -# -# Each entry was found by tracing the suite under strace: the files it opened -# or listed that no other selection rule connects to it. Declare what the suite -# reads, not the whole plugin, unless it reads the whole plugin. - -# Live-tree suites under scripts/: they scan the repository rather than fixtures. -scripts/affected-tests.test.sh scripts/affected-tests* scripts/sync-*.sh scripts/lib/sync-*.sh .github/workflows/ci.yml # runs every sync manifest, reads the selector's lists and the live workflow -scripts/check-docs-only-gate.test.sh .github/workflows/ci.yml # checks the live workflow against the docs-only contract -scripts/ci-fail-a-draft.test.sh .github/workflows/ci.yml # reads the live workflow -scripts/check-lane-coverage.test.sh .github/workflows/ci.yml # checks the live workflow's lane coverage -scripts/check-hook-wiring-liveness.test.sh .claude/settings.json # reads the live project settings - -# Suites that read a skill body or README through a path that spells only its -# name ($SKILL_DIR/SKILL.md), which does not resolve. -plugins/prototype/skills/explore-directions/scripts/detect-ecosystems.test.sh plugins/prototype/skills/explore-directions/SKILL.md -plugins/prototype/skills/pressure-test/scripts/detect-ecosystems.test.sh plugins/prototype/skills/pressure-test/SKILL.md -plugins/repo-hygiene/skills/clean/scripts/destructive-guard.test.sh plugins/repo-hygiene/skills/clean/SKILL.md -plugins/source-control/skills/babysit-prs/scripts/tests/test_guards.py plugins/source-control/skills/*/SKILL.md -plugins/disk-hygiene/skills/clean/scripts/test_hygiene.py plugins/disk-hygiene/skills/clean/SKILL.md -plugins/playwright/skills/playwright/scripts/update.test.sh plugins/playwright/skills/playwright/SKILL.md -plugins/guardrails/hooks/coverage-manifest.test.sh plugins/guardrails/README.md - -# Suites that read the repository's own .claude/settings.json, which no mention -# reaches (MANIFESTS, and a root path does not resolve). -plugins/docs-hygiene/skills/compress/scripts/detect-caveman.test.sh .claude/settings.json -plugins/harness-config/skills/audit-permission-state/scripts/audit.test.sh .claude/settings.json -plugins/harness-ops/skills/plugins/scripts/fleet-state.test.sh .claude/settings.json -plugins/harness-ops/skills/plugins/scripts/sync-run.test.sh .claude/settings.json -scripts/lib/gate-entry.test.sh scripts/*.sh # scans every script for a hand-rolled base-ref predicate -scripts/lib/test-harness.test.sh scripts/*.test.sh # every harness suite must end on test_harness::report -scripts/validate-plugin-contracts.test.sh plugins/*/retirements.yaml plugins/*/skills/*/evals/evals.json plugins/*/reference/artifact-protocol.md # the validator walks every plugin -scripts/check-loop-lane-floor-drift.test.sh plugins/*/reference/reader-contract.md # compares every copy of the reader contract - -# Plugin prose and manifest scanners: grep -r or find over the plugin. -plugins/github/github.test.sh plugins/github/* -plugins/source-control/skills/worktree/nesting-invariant-ssot.test.sh plugins/source-control/* -plugins/work-items/tests/no-hardcoded-priority-scheme.test.sh plugins/work-items/*.md -plugins/discovery/scripts/contract.test.sh plugins/discovery/*.md plugins/discovery/*.json -plugins/discovery/agents/tool-honesty.test.sh plugins/discovery/agents/*.md -plugins/review/tests/change-set-block.test.sh plugins/review/agents/*.md -plugins/multi-agent/tests/drift-audit.test.sh plugins/multi-agent/agents/*.md -plugins/planning/tests/reattach-slice.test.sh plugins/planning/skills/*/SKILL.md -plugins/harness-config/skills/audit-automation-gaps/scripts/inventory.test.sh plugins/*/.mcp.json plugins/harness-config/skills/*/SKILL.md .claude/settings.json -plugins/source-control/skills/babysit-prs/scripts/engine.test.sh plugins/source-control/*.md plugins/source-control/scripts/babysit-*.sh # test_guards.py walks the plugin - -# allowed-tools pairing: every listed skill's body and bundled scripts. -plugins/code-tidying/scripts/allowed-tools-pairing.test.sh plugins/code-tidying/skills/*.md plugins/code-tidying/skills/*/scripts/* -plugins/docs-hygiene/scripts/allowed-tools-pairing.test.sh plugins/docs-hygiene/skills/*.md plugins/docs-hygiene/skills/*/scripts/* -plugins/docs-naming/scripts/allowed-tools-pairing.test.sh plugins/docs-naming/skills/*.md plugins/docs-naming/skills/*/scripts/* -plugins/multi-agent/scripts/allowed-tools-pairing.test.sh plugins/multi-agent/skills/*.md plugins/multi-agent/skills/*/scripts/* -plugins/prototype/scripts/allowed-tools-pairing.test.sh plugins/prototype/skills/*.md plugins/prototype/skills/*/scripts/* -plugins/repo-fleet-hygiene/scripts/allowed-tools-pairing.test.sh plugins/repo-fleet-hygiene/skills/*.md plugins/repo-fleet-hygiene/skills/*/scripts/* -plugins/repo-hygiene/scripts/allowed-tools-pairing.test.sh plugins/repo-hygiene/skills/*.md plugins/repo-hygiene/skills/*/scripts/* - -# Python dependency probes: glob the skills, scripts and hooks they ship. -plugins/animation/scripts/animation.test.sh plugins/animation/skills/*/SKILL.md plugins/animation/skills/*/scripts/* plugins/animation/hooks/*.sh -plugins/speech/scripts/speech.test.sh plugins/speech/skills/*/SKILL.md plugins/speech/hooks/*.sh plugins/speech/scripts/*.json - -# Registries and dispatchers that enumerate a directory of modules or adapters. -plugins/code-metrics/scripts/dispatch.test.sh plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* -plugins/code-metrics/scripts/tool-free-path.test.sh plugins/code-metrics/scripts/collectors/*.py -plugins/code-metrics/skills/audit-complexity/scripts/audit-complexity.test.sh plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* -plugins/code-metrics/skills/audit-coverage/scripts/audit-coverage.test.sh plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/parsers/*.py plugins/code-metrics/scripts/fixtures/* -plugins/code-metrics/skills/audit-duplication/scripts/audit-duplication.test.sh plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* -plugins/code-metrics/skills/audit-size/scripts/audit-size.test.sh plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* -plugins/code-metrics/skills/audit-type-debt/scripts/audit-type-debt.test.sh plugins/code-metrics/scripts/collectors/*.py plugins/code-metrics/scripts/fixtures/* -plugins/code-metrics/skills/setup/scripts/setup-check.test.sh plugins/code-metrics/scripts/collectors/*.py -plugins/testing/scripts/gen-hook-filters.test.sh plugins/testing/skills/audit/adapters/*.yaml -plugins/testing/skills/setup/scripts/setup.test.sh plugins/testing/skills/audit/adapters/*.yaml -plugins/work-items/tools/work-item-tracker/work-item-tracker.test.sh plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* -plugins/work-items/tools/work-item-tracker/conformance/bindings/local-markdown.test.sh plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* -plugins/work-items/tools/work-item-tracker/conformance/bindings/jira.test.sh plugins/work-items/tools/work-item-tracker/adapters/jira/* -plugins/work-items/tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* -plugins/work-items/tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* -plugins/work-items/tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh plugins/work-items/tools/work-item-tracker/adapters/local-markdown/* -plugins/work-items/tools/work-item-tracker/adapters/gitea/list-items.test.sh plugins/work-items/tools/work-item-tracker/adapters/gitea/* -plugins/harness-ops/hooks/audit-session-id.test.sh plugins/harness-ops/hooks/*-audit.sh -plugins/source-control/scripts/babysit-wrapper-help.test.sh plugins/source-control/scripts/* plugins/source-control/skills/babysit-prs/scripts/babysit_*.py - -# Suites that copy their plugin and run its hooks from the copy. -plugins/guardrails/hooks/abort-boundary.test.sh plugins/guardrails/hooks/* plugins/guardrails/lib/* -plugins/guardrails/hooks/run-guards.test.sh plugins/guardrails/hooks/*.sh -plugins/guardrails/hooks/require-jq-posture.test.sh plugins/guardrails/hooks/*.sh -plugins/guardrails/hooks/require-jq-notice-isolation.test.sh plugins/guardrails/hooks/*.sh -plugins/context-guard/hooks/zone-crossing-inject.test.sh plugins/context-guard/hooks/* -plugins/actionlint/hooks/actionlint-check.test.sh plugins/actionlint/hooks/* plugins/actionlint/.claude-plugin/plugin.json -plugins/disk-hygiene/skills/setup/scripts/kill_switch_probe.test.sh plugins/disk-hygiene/*.py plugins/disk-hygiene/*.sh plugins/disk-hygiene/*.mjs plugins/disk-hygiene/*.json -plugins/disk-hygiene/skills/clean/scripts/owner_registry.test.sh plugins/disk-hygiene/*.py plugins/disk-hygiene/*.sh plugins/disk-hygiene/*.mjs plugins/disk-hygiene/*.json -plugins/disk-hygiene/skills/clean/scripts/hygiene.test.sh plugins/disk-hygiene/skills/clean/SKILL.md plugins/disk-hygiene/skills/clean/reference/*.json plugins/disk-hygiene/hooks/* - -# Fixture directories a suite enumerates or reads by a built path. -plugins/autonomy/skills/setup/scripts/check-security-binding.fixtures.test.sh plugins/autonomy/skills/setup/evals/fixtures/security-binding/* -plugins/autonomy/skills/setup/scripts/resolve-prerequisites.fixtures.test.sh plugins/autonomy/skills/setup/scripts/fixtures/prerequisite-resolution/* plugins/autonomy/generated/* -plugins/autonomy/skills/setup/scripts/check-prerequisite-resolution-slice.test.sh plugins/autonomy/skills/setup/scripts/fixtures/prerequisite-resolution/* -plugins/autonomy/skills/setup/scripts/generate-identity-prerequisites.test.sh plugins/autonomy/reference/routines/*.md -plugins/code-tidying/scripts/evals-fixtures.test.sh plugins/code-tidying/evals/* -plugins/evals/skills/validate/scripts/validate-cases.test.sh plugins/evals/evals/* -plugins/evals/skills/plugin-eval/scripts/calibrate-judge.test.sh plugins/evals/evals/* plugins/evals/skills/plugin-eval/scripts/fixtures/calibrate-judge/* -plugins/evals/skills/plugin-eval/scripts/run-validity.test.sh plugins/evals/skills/plugin-eval/scripts/fixtures/run-validity/* -plugins/knowledge/skills/docpage-digest/scripts/check-html-rows.test.sh plugins/knowledge/skills/docpage-digest/scripts/fixtures/html-rows/* -plugins/knowledge/skills/docpage-digest/scripts/extract_blog_body.test.sh plugins/knowledge/skills/docpage-digest/scripts/fixtures/blog-body.html -plugins/pixel-art/scripts/backends.test.sh plugins/pixel-art/examples/* plugins/pixel-art/palettes/* -plugins/retro-audio/scripts/audio.test.sh plugins/retro-audio/examples/* -plugins/planning/surface/surface.test.sh plugins/planning/surface/schema/* plugins/planning/surface/tests/fixtures/* -plugins/planning/surface/watch.test.sh plugins/planning/surface/schema/* -plugins/planning/surface/test_exporters.py plugins/planning/surface/tests/fixtures/ledger-legacy/* -plugins/planning/surface/test_round.py plugins/planning/surface/tests/fixtures/*.json -plugins/planning/surface/test_schema.py plugins/planning/surface/tests/fixtures/*.json -plugins/planning/surface/test_server.py plugins/planning/surface/tests/fixtures/*.json -plugins/session-flow/scripts/save_point.test.sh plugins/session-flow/scripts/tests/fixtures/* -plugins/session-flow/skills/audit-sessions/scripts/audit-sessions.test.sh plugins/session-flow/skills/audit-sessions/scripts/tests/fixtures/* diff --git a/scripts/affected-tests.sh b/scripts/affected-tests.sh index 6589a3872e..ed9873bb42 100755 --- a/scripts/affected-tests.sh +++ b/scripts/affected-tests.sh @@ -24,7 +24,7 @@ # the selector at (see REPLAY) # # --with-always is accepted and changes nothing: the suites that assert against -# the live tree declare what they read in scripts/affected-tests-scopes.txt (R8). +# the live tree declare what they read in a `# test-scope:` header (R8). # # Exit: 0 selected (or nothing to do); 1 an unmapped changed file, or a failing # suite under --run; 2 usage or a broken derivation; 3 --run ran every shell @@ -103,14 +103,15 @@ # R8 declared scope a suite that enumerates a directory of the live tree # (a grep -r, a find, a glob over a plugin or scripts/), or # builds a path from parts, never names the files it reads, -# so scripts/affected-tests-scopes.txt declares them, one -# ` ...` line per suite: -# plugins/github/github.test.sh plugins/github/* +# so it declares them in its leading comment block, before +# any code or docstring, in one or more lines of +# # test-scope: plugins/github/* scripts/x.sh # A changed file matching a glob selects the suite and counts # as mapped. The globs use the dialect of the no-suite list: -# matched against the repo-relative path, `*` crosses `/`. -# An entry naming no suite fails every run (exit 2), and a -# glob matching no file fails the run that changes the list. +# matched against the repo-relative path, `*` crosses `/`; +# a `#` after them starts a comment. Suites whose comments +# start with `#` (shell, Python, Pester) can declare. A glob +# matching no file fails the run that changes its suite. # # MATCHING. One file NAMES another when the basename stands in a line as a WHOLE # PATH TOKEN: bounded on both sides by a character outside [A-Za-z0-9_.-]. `/` @@ -158,29 +159,25 @@ # `import('...')`) are read by tools and still count, as does a trailing # comment on a code line. # -# PYTHON IMPORTS. An import never spells the .py, so R3 also reads Python -# import lines: `import foo`, `from foo import x`, `from . import foo` and the -# dotted forms name foo.py (or the package foo/__init__.py) when the importer -# sits in the directory foo is imported from or below it, when the dotted path -# spells the path of foo, or when foo is the only module of that name in the -# importer's plugin, the reach of a sys.path insert. -# # REPLAY. `--replay ` selects every first-parent commit of # (`git rev-list --first-parent `) against its parent, the squash-merged # pull request's net diff, in a scratch clone checked out at that commit, with -# THIS script's rules, no-suite list and scopes list, so it answers "what would -# this selector have run for those pull requests". It prints one -# `commit ` line per commit, an indented +# THIS script's rules, its no-suite list and its suites' declared scopes, so it +# answers "what would this selector have run for those pull requests". It +# prints one `commit ` line per commit, an indented # `unmapped ` line per unmapped file and one indented # ` ()` line per suite. With `--against ` it also runs the -# selector at , with 's own lists, on the same tree, and prints only -# the suites that differ (`+` this script only, `-` only, each with its -# reason) after a `commit ` line +# selector at , with 's own no-suite list and declared scopes, on the +# same tree, and prints only the suites that differ (`+` this script only, +# `-` only, each with its reason) after a +# `commit ` line # and its `unmapped` lines, so a selector change shows its blast radius. Both # sides run with --allow-unmapped; a summary on stderr counts suites per commit # (p50, p95, max, total) and the commits with an unmapped file on each side. -# Each run is pointed at the scratch clone with AFFECTED_TESTS_ROOT, and at the -# lists with AFFECTED_TESTS_NO_SUITE and AFFECTED_TESTS_SCOPES. +# Each run is pointed at the scratch clone with AFFECTED_TESTS_ROOT and at the +# no-suite list with AFFECTED_TESTS_NO_SUITE, and AFFECTED_TESTS_SCOPES hands it +# the declarations as ` ...` lines in place of the suites' headers, +# since the replayed commits may predate them. # # MECHANICALLY the reverse lookup is two stages. `git grep -F` finds the # candidate LINES with the substring test, which keeps git's fixed-string fast @@ -200,7 +197,7 @@ cd "${AFFECTED_TESTS_ROOT:-$SCRIPT_DIR/..}" || exit 2 . "$SCRIPT_DIR/lib/read-list.sh" || exit 2 NO_SUITE_LIST="${AFFECTED_TESTS_NO_SUITE:-scripts/affected-tests-no-suite.txt}" -SCOPES_LIST="${AFFECTED_TESTS_SCOPES:-scripts/affected-tests-scopes.txt}" +SCOPES_LIST="${AFFECTED_TESTS_SCOPES:-}" # Basenames that name a repository-wide role, reached only through a resolved # mention (AMBIGUOUS NAMES in the header), however few files carry them today. @@ -504,33 +501,19 @@ build_tree_index() { done <<<"${SYNC_SRC_COPIES[$src]}" done - # Every Python import line, read once for PYTHON IMPORTS. Fatal on a git - # error for the same reason as the reverse lookup: no lines reads as no edges. - local rc=0 - git grep --untracked -I -E '^[[:space:]]*(from[[:space:]]+[.A-Za-z_][.A-Za-z0-9_]*[[:space:]]+import|import[[:space:]]+[A-Za-z_])' \ - -- '*.py' >"$WORK_DIR/py-imports" || rc=$? - if [[ "$rc" -gt 1 ]]; then - echo "error: 'git grep' failed (exit $rc) listing the Python import lines." >&2 - exit 2 - fi - - # R8, one ` ...` line per suite. An entry naming no suite fails - # the run: a declaration must not outlive what it declares. A replay hands in - # the list of the tree it started from, whose suites an older commit may lack. + # R8: the suites' `# test-scope:` headers, or the ` ...` lines + # a replay hands in, whose suites an older commit may lack. local -a entries=() words=() - local entry i - if [[ ! -f "$SCOPES_LIST" ]]; then - echo "error: missing $SCOPES_LIST, the declared test scopes (R8)." >&2 - exit 2 + local entry i decls + if [[ -n "$SCOPES_LIST" ]]; then + read_list::into entries "$SCOPES_LIST" --comments inline || exit 2 + else + decls="$(scope_declarations "$WORK_DIR/all-files")" || exit 2 + read_list::into_text entries "$decls" --comments inline || exit 2 fi - read_list::into entries "$SCOPES_LIST" --comments inline || exit 2 for entry in ${entries[@]+"${entries[@]}"}; do read -r -a words <<<"$entry" suite="${words[0]}" - if [[ -z "${AFFECTED_TESTS_SCOPES:-}" ]] && { ! is_suite_path "$suite" || [[ ! -f "$suite" ]]; }; then - echo "error: $SCOPES_LIST names '$suite', which is not a suite; update or remove the entry." >&2 - exit 2 - fi for ((i = 1; i < ${#words[@]}; i++)); do SCOPE_SUITES+=("$suite") SCOPE_GLOBS+=("${words[i]}") @@ -538,6 +521,23 @@ build_tree_index() { done } +# scope_declarations -> one ` ...` line per +# `# test-scope:` line in the leading comment block of each suite the list +# names, read from the current directory (R8). +scope_declarations() { + awk ' + { b = $0; sub(/.*\//, "", b) } + !(/\.test\.sh$|\.Tests\.ps1$/ || b ~ /^test_.*\.py$/) { next } + { + while ((getline line < $0) > 0) { + sub(/\r$/, "", line) + if (line !~ /^#/ && line !~ /^[ \t]*$/) break + if (sub(/^#[ \t]*test-scope:/, "", line)) print $0 " " line + } + close($0) + }' "$1" +} + # --------------------------------------------------------------------------- # Selection # --------------------------------------------------------------------------- @@ -735,81 +735,6 @@ token_hits() { ' "$1" "$2" "$WORK_DIR/all-files" "$3" >"$4" } -# py_hits -> PYTHON IMPORTS: append one -# r -# line for every .py whose import line imports a frontier module, read from the -# import lines build_tree_index listed. -py_hits() { - awk -v front="$1" -v allf="$WORK_DIR/all-files" ' - function dir_of(p) { sub(/[^\/]*$/, "", p); return p } - function root_of(p, c) { split(p, c, "/"); return c[1] == "plugins" ? "plugins/" c[2] "/" : c[1] "/" } - function ends(s, t) { return length(s) >= length(t) && substr(s, length(s) - length(t) + 1) == t } - # The module a file is: foo for foo.py, pkg for pkg/__init__.py. - function modname(p) { - if (p ~ /(^|\/)__init__\.py$/) { p = dir_of(p); sub(/\/$/, "", p) } - sub(/.*\//, "", p) - sub(/\.py$/, "", p) - return p - } - # imports: does dotted name D, imported in P, mean module file t? From the - # directory t is imported from, or below it (a tests/ directory); by a - # dotted path that spells t; or anywhere in the same plugin when t is the - # only module of that name there. - function imports(P, D, t, m, home, pd, path) { - home = dir_of(t) - if (t ~ /(^|\/)__init__\.py$/) { sub(/\/$/, "", home); home = dir_of(home) } - pd = dir_of(P) - if (pd == home || (home != "" && index(pd, home) == 1)) return 1 - if (index(D, ".")) { - path = D - gsub(/\./, "/", path) - if (ends("/" t, "/" path ".py") || ends("/" t, "/" path "/__init__.py")) return 1 - } - return root_of(P) == root_of(t) && cnt[root_of(t), m] == 1 - } - function cand(P, D, m, k, t) { - m = D - sub(/.*\./, "", m) - for (k = 1; k <= nt[m]; k++) { - t = tg[m, k] - if (t != P && imports(P, D, t, m) && !((P SUBSEP t) in seen)) { - seen[P, t] = 1 - print "r\t" P "\t" t - } - } - } - FILENAME == front { if ($0 != "") { m = modname($0); tg[m, ++nt[m]] = $0 } next } - FILENAME == allf { if ($0 ~ /\.py$/) cnt[root_of($0), modname($0)]++; next } - { - i = index($0, ":") - if (i == 0) next - P = substr($0, 1, i - 1) - s = substr($0, i + 1) - sub(/#.*/, "", s) - gsub(/[()\\]/, " ", s) - if (s ~ /^[ \t]*from[ \t]/) { - sub(/^[ \t]*from[ \t]+/, "", s) - base = s - sub(/[ \t].*/, "", base) - sub(/^[^ \t]+[ \t]+import[ \t]+/, "", s) - sub(/^\.+/, "", base) - if (base != "") cand(P, base) - } else { - sub(/^[ \t]*import[ \t]+/, "", s) - base = "" - } - n = split(s, ys, /,/) - for (k = 1; k <= n; k++) { - y = ys[k] - sub(/^[ \t]+/, "", y) - sub(/[ \t].*/, "", y) - if (y !~ /^[A-Za-z_][A-Za-z0-9_.]*$/) continue - cand(P, base == "" ? y : base "." y) - } - } - ' "$1" "$WORK_DIR/all-files" "$WORK_DIR/py-imports" >>"$2" -} - # colocated_suites -> every sibling suite covering it, one per line. # PLURAL on purpose: a .py can carry a co-located test_.py and a # wrapping .test.sh at once, and returning one under-selects. @@ -882,12 +807,10 @@ select_for() { : >"$WORK_DIR/patterns" : >"$WORK_DIR/plain" : >"$WORK_DIR/resolve" - : >"$WORK_DIR/pyfront" next=() for p in "${frontier[@]}"; do [[ -n "${VISITED[$p]:-}" ]] && continue VISITED["$p"]=1 - [[ "$p" == *.py ]] && printf '%s\n' "$p" >>"$WORK_DIR/pyfront" # R1/R2 while IFS= read -r sib; do [[ -n "$sib" ]] || continue @@ -948,8 +871,7 @@ select_for() { fi # Fatal for the same reason: a filter that dies mid-stream hands the walk a # TRUNCATED hit set. - if ! token_hits "$WORK_DIR/plain" "$WORK_DIR/resolve" "$WORK_DIR/matched-lines" "$WORK_DIR/hits" || - { [[ -s "$WORK_DIR/pyfront" ]] && ! py_hits "$WORK_DIR/pyfront" "$WORK_DIR/hits"; }; then + if ! token_hits "$WORK_DIR/plain" "$WORK_DIR/resolve" "$WORK_DIR/matched-lines" "$WORK_DIR/hits"; then echo "error: the token filter over the reverse lookup failed on the current level." >&2 echo " Refusing to continue: a partial filter silently UNDER-selects, and" >&2 echo " under-selection is reported as success by everything downstream." >&2 @@ -1005,12 +927,14 @@ select_for() { done } -# check_scope_globs -> exit 2 when a declared glob matches no file of the tree: -# it declares nothing, so the suite misses the changes it reads. Run when the -# scopes list itself changes, which is when a glob is written or goes stale. +# check_scope_globs -> exit 2 when a glob the suite declares matches no +# file of the tree: it declares nothing, so the suite misses the changes it +# reads. Run when a declaring suite changes, which is when a glob is written. check_scope_globs() { - [[ ${#SCOPE_GLOBS[@]} -gt 0 ]] || return 0 - printf '%s\n' "${SCOPE_GLOBS[@]}" | awk ' + local i + for i in "${!SCOPE_GLOBS[@]}"; do + if [[ "${SCOPE_SUITES[i]}" == "$1" ]]; then printf '%s\n' "${SCOPE_GLOBS[i]}"; fi + done | awk ' # The glob dialect of the lists: `*` any run of characters, `/` included. function to_regex(g, r, i, c) { r = "^" @@ -1027,7 +951,7 @@ check_scope_globs() { { for (g in want) if (!(g in hit) && $0 ~ want[g]) hit[g] = 1 } END { for (i = 1; i <= n; i++) if (!(order[i] in hit)) { print order[i]; bad = 1 } exit bad } ' - "$WORK_DIR/all-files" >"$WORK_DIR/stale-globs" && return 0 - echo "error: $SCOPES_LIST declares globs that match no file of the tree:" >&2 + echo "error: $1 declares test-scope globs that match no file of the tree:" >&2 sed 's/^/ - /' "$WORK_DIR/stale-globs" >&2 exit 2 } @@ -1158,7 +1082,7 @@ replay_select() { } run_replay() { - local tree="$WORK_DIR/replay-tree" against="$WORK_DIR/against" against_scopes="" + local tree="$WORK_DIR/replay-tree" against="$WORK_DIR/against" against_scopes="" against_sha="" local c subject n_new n_old u_new u_old local -a commits=() changed=() against_flags=() if ! git rev-list --first-parent --reverse "$replay_range" >"$WORK_DIR/commits"; then @@ -1170,9 +1094,16 @@ run_replay() { echo "error: '$replay_range' holds no commits to replay." >&2 exit 2 fi - # This tree's rules travel with the replay: its no-suite and scopes lists. + # This tree's rules travel with the replay: its no-suite list and its suites' + # declared scopes. cp "$NO_SUITE_LIST" "$WORK_DIR/replay-no-suite" || exit 2 - cp "$SCOPES_LIST" "$WORK_DIR/replay-scopes" || exit 2 + if [[ -n "$SCOPES_LIST" ]]; then + cp "$SCOPES_LIST" "$WORK_DIR/replay-scopes" || exit 2 + elif ! git ls-files --cached --others --exclude-standard >"$WORK_DIR/replay-files" || + ! scope_declarations "$WORK_DIR/replay-files" >"$WORK_DIR/replay-scopes"; then + echo "error: could not read this tree's declared test scopes." >&2 + exit 2 + fi if ! git clone -q --shared --no-checkout . "$tree"; then echo "error: could not make the scratch clone for the replay." >&2 exit 2 @@ -1197,9 +1128,14 @@ run_replay() { grep -q -- '--with-always)' "$against/scripts/affected-tests.sh" && against_flags+=(--with-always) git show "$against_ref:scripts/affected-tests-always.txt" >"$against/always.txt" 2>/dev/null || rm -f "$against/always.txt" - # A selector that reads a scopes list gets 's own. - if git show "$against_ref:scripts/affected-tests-scopes.txt" >"$against/scopes.txt" 2>/dev/null; then - against_scopes="$against/scopes.txt" + # 's own declared scopes, read from its suites in the scratch clone. + against_scopes="$against/scopes.txt" + if ! against_sha="$(git rev-parse --verify -q "$against_ref^{commit}")" || + ! git -C "$tree" -c advice.detachedHead=false checkout -q --detach "$against_sha" || + ! git -C "$tree" ls-files >"$against/files" || + ! (cd "$tree" && scope_declarations "$against/files") >"$against_scopes"; then + echo "error: could not read the declared test scopes at '$against_ref'." >&2 + exit 2 fi fi @@ -1310,7 +1246,8 @@ build_tree_index declare -a NO_SUITE_FILES=() for f in "${changed[@]}"; do [[ -n "$f" ]] || continue - [[ "$f" == "$SCOPES_LIST" ]] && check_scope_globs + # Only this tree's own headers: handed-in declarations may postdate the tree. + [[ -z "$SCOPES_LIST" && " ${SCOPE_SUITES[*]-} " == *" $f "* ]] && check_scope_globs "$f" select_for "$f" select_scoped "$f" if [[ "$SEED_HITS" -eq 0 ]]; then diff --git a/scripts/affected-tests.test.sh b/scripts/affected-tests.test.sh index ebc55d6b80..8d8fff85b2 100755 --- a/scripts/affected-tests.test.sh +++ b/scripts/affected-tests.test.sh @@ -8,6 +8,7 @@ # against the LIVE repo — the derived shared-lib copy set and the real no-suite # list — because a synthetic fixture cannot show that the derivation still # tracks reality, which is the whole failure mode this tool exists to avoid. +# test-scope: scripts/affected-tests* scripts/sync-*.sh scripts/lib/sync-*.sh .github/workflows/ci.yml set -uo pipefail TMP_ROOT="$(mktemp -d)" @@ -73,8 +74,6 @@ mk_repo() { # mkdir -p "$dir/lib" "$dir/plugins/alpha/hooks" "$dir/plugins/beta/hooks" cp "$NO_SUITE" "$dir/scripts/affected-tests-no-suite.txt" - # The live scopes list names suites this fixture does not have. - printf '# fixture scopes\n' >"$dir/scripts/affected-tests-scopes.txt" # --jobs N delegates to the SIBLING run-plugin-tests.sh rather than spawning # anything itself, so the fixture carries that sibling too. Its serial @@ -1143,55 +1142,6 @@ else fail "py -> sh -> sh chain lost its suite (rc=$RC): $OUT" fi -# --- Python imports name the module they load --------------------------------- -# `import tool` never spells tool.py, so the import line is the edge: from the -# module's directory or below it, by a dotted path that spells it, or from -# anywhere in the plugin when the module name is unique there. A module of the -# same name elsewhere in the plugin makes the bare import ambiguous, and a -# module nothing imports selects only its own suites. -mkdir -p "$repo/plugins/alpha/scripts/tests" "$repo/plugins/alpha/skills/one/scripts" \ - "$repo/plugins/beta/scripts" "$repo/plugins/alpha/pkg/sub" -printf 'def run():\n return 1\n' >"$repo/plugins/alpha/scripts/tool.py" -printf 'from tool import run\n' >"$repo/plugins/alpha/scripts/runner.py" -printf 'import runner\n' >"$repo/plugins/alpha/scripts/test_runner.py" -printf 'import sys\nimport tool as t\n' >"$repo/plugins/alpha/scripts/tests/test_tool_behavior.py" -printf 'import tool\n' >"$repo/plugins/alpha/skills/one/scripts/use_tool.py" -printf 'import use_tool\n' >"$repo/plugins/alpha/skills/one/scripts/test_use_tool.py" -printf 'from . import tool\n' >"$repo/plugins/alpha/scripts/rel_user.py" -printf 'import rel_user\n' >"$repo/plugins/alpha/scripts/test_rel_user.py" -printf 'import tool\n' >"$repo/plugins/beta/scripts/other.py" -printf 'import other\n' >"$repo/plugins/beta/scripts/test_other.py" -printf 'X = 1\n' >"$repo/plugins/alpha/pkg/sub/deep.py" -printf 'from pkg.sub.deep import X\n' >"$repo/plugins/alpha/dotted.py" -printf 'import dotted\n' >"$repo/plugins/alpha/test_dotted.py" -git_test_config "$repo" add plugins >/dev/null -git_test_config "$repo" commit -qm pyimports >/dev/null -run_sel "$repo" plugins/alpha/scripts/tool.py -if [[ "$RC" -eq 0 ]] && has_line "$OUT" plugins/alpha/scripts/test_runner.py && - has_line "$OUT" plugins/alpha/scripts/tests/test_tool_behavior.py && - has_line "$OUT" plugins/alpha/skills/one/scripts/test_use_tool.py && - has_line "$OUT" plugins/alpha/scripts/test_rel_user.py && - ! has_line "$OUT" plugins/beta/scripts/test_other.py; then - ok "python: an import (also 'from . import') selects from the module's directory, below it, and across its plugin" -else - fail "python: import selection wrong for tool.py (rc=$RC): $OUT" -fi -run_sel "$repo" plugins/alpha/pkg/sub/deep.py -if [[ "$RC" -eq 0 ]] && has_line "$OUT" plugins/alpha/test_dotted.py; then - ok "python: a dotted import that spells the module's path selects" -else - fail "python: dotted import lost (rc=$RC): $OUT" -fi -printf 'def run():\n return 2\n' >"$repo/plugins/alpha/skills/one/scripts/tool.py" -run_sel "$repo" plugins/alpha/scripts/tool.py -if [[ "$RC" -eq 0 ]] && has_line "$OUT" plugins/alpha/scripts/test_runner.py && - ! has_line "$OUT" plugins/alpha/skills/one/scripts/test_use_tool.py; then - ok "python: a module name two directories of a plugin carry resolves by directory only" -else - fail "python: an ambiguous module name still selected across the plugin (rc=$RC): $OUT" -fi -rm -f "$repo/plugins/alpha/skills/one/scripts/tool.py" - # --- the crossing budget is aggregated per path, never assigned ------------ # One path can be hit several times in a single round by different patterns, and # those hits can disagree about whether the chain reaching it has already @@ -1397,12 +1347,8 @@ mk_cmt_dependent sh-directive sh '# shellcheck source=hub-target.sh\n. "$HUB"\n' mk_cmt_dependent js-code js 'spawnSync("bash", ["hub-target.sh"]);\n' mk_cmt_dependent js-typeimport js '/** @import { T } from "./hub-target.sh" */\n/** @param {import("./hub-target.sh").T} t */\nexport const y = 2;\n' printf '#!/usr/bin/env bash\n# covers hub-target.sh\n' >"$repo3/eco/cmt/hub-prose.test.sh" -# A Python import never spells the .py, so a comment naming the module is the -# only text edge from an importer; it keeps counting. Prose alone does not. printf 'X = 1\n' >"$repo3/eco/cmt/hubmod.py" printf 'import hubmod\n' >"$repo3/eco/cmt/test_hubmod.py" -printf '# hubmod.py is shared with a sibling\nfrom hubmod import X\n' >"$repo3/eco/cmt/pyimporter.py" -printf 'import pyimporter\n' >"$repo3/eco/cmt/test_pyimporter.py" printf '# see hubmod.py\nimport os\n' >"$repo3/eco/cmt/pyprose.py" printf 'import pyprose\n' >"$repo3/eco/cmt/test_pyprose.py" git_test_config "$repo3" add eco >/dev/null @@ -1434,11 +1380,11 @@ else fi run_sel "$repo3" eco/cmt/hubmod.py -if [[ "$RC" -eq 0 ]] && has_line "$OUT" eco/cmt/test_pyimporter.py && +if [[ "$RC" -eq 0 ]] && has_line "$OUT" eco/cmt/test_hubmod.py && ! has_line "$OUT" eco/cmt/test_pyprose.py; then - ok "a comment naming a module the .py imports by name still selects; prose alone does not" + ok "a Python comment naming a module selects nothing" else - fail "python import-by-name comment edge lost or prose comment kept (rc=$RC): $OUT" + fail "python: a comment-only mention still selected (rc=$RC): $OUT" fi rm -rf "$repo3" @@ -1541,39 +1487,40 @@ fi # --- R8: a declared scope selects the suite that scans a directory ----------- # A suite that greps or globs a directory never spells the files it reads, so -# scripts/affected-tests-scopes.txt declares them, and a matching change selects -# it and counts as mapped. Pinned: the glob crosses `/`, the plugin's other -# suites stay out, an inline comment ends the entry, and a plugin file nothing -# names or declares is still UNMAPPED. +# it declares them in `# test-scope:` lines of its leading comment block, and a +# matching change selects it and counts as mapped. Pinned: the glob crosses +# `/`, a suite may declare on several lines, an inline comment ends the globs, +# a line below the first code line declares nothing, the plugin's other suites +# stay out, and a plugin file nothing names or declares is still UNMAPPED. mk_repo repo mkdir -p "$repo/plugins/alpha/skills/one" "$repo/plugins/alpha/tests" printf -- '---\nname: one\n---\n' >"$repo/plugins/alpha/skills/one/SKILL.md" printf 'kind: probe\n' >"$repo/plugins/alpha/skills/one/probe.yaml" -suite_body alpha-scan >"$repo/plugins/alpha/tests/scan.test.sh" +printf '#!/usr/bin/env bash\n# Scans the skill bodies.\n# test-scope: plugins/alpha/skills/*.md\n\n# test-scope: plugins/alpha/*.yaml # and the probes\necho alpha-scan\n' \ + >"$repo/plugins/alpha/tests/scan.test.sh" +printf '# test-scope: plugins/alpha/skills/*/SKILL.md\n"""Scans the skill bodies."""\nimport unittest\n' \ + >"$repo/plugins/alpha/tests/test_scan.py" +printf '#!/usr/bin/env bash\necho late\n# test-scope: plugins/alpha/skills/*.md\n' >"$repo/plugins/alpha/tests/late.test.sh" printf 'import test from "node:test";\n' >"$repo/plugins/alpha/tests/scan.test.mjs" -printf 'import unittest\n' >"$repo/plugins/alpha/tests/test_scan.py" +printf '#!/usr/bin/env bash\n# test-scope: plugins/nowhere/*\necho stale\n' >"$repo/plugins/alpha/tests/stale.test.sh" printf 'echo orphan\n' >"$repo/plugins/alpha/zzorphan-plugin.sh" -{ - printf '# fixture scopes\n' - printf 'plugins/alpha/tests/scan.test.sh plugins/alpha/skills/*.md plugins/alpha/*.yaml # scans skill bodies\n' - printf 'plugins/alpha/tests/scan.test.mjs plugins/alpha/skills/*/SKILL.md\n' -} >"$repo/scripts/affected-tests-scopes.txt" -git_test_config "$repo" add plugins scripts >/dev/null +git_test_config "$repo" add plugins >/dev/null git_test_config "$repo" commit -qm r8 >/dev/null run_sel "$repo" plugins/alpha/skills/one/SKILL.md if [[ "$RC" -eq 0 ]] && has_line "$OUT" plugins/alpha/tests/scan.test.sh && - has_line "$OUT" plugins/alpha/tests/scan.test.mjs && - ! has_line "$OUT" plugins/alpha/hooks/alpha-hook.test.sh && - ! has_line "$OUT" plugins/alpha/tests/test_scan.py; then - ok "R8: a SKILL.md edit selects the suites that declare it and no other suite of the plugin" + has_line "$OUT" plugins/alpha/tests/test_scan.py && + ! has_line "$OUT" plugins/alpha/tests/late.test.sh && + ! has_line "$OUT" plugins/alpha/tests/scan.test.mjs && + ! has_line "$OUT" plugins/alpha/hooks/alpha-hook.test.sh; then + ok "R8: a SKILL.md edit selects the suites whose headers declare it and no other suite of the plugin" else fail "R8: declared-scope selection wrong for a SKILL.md edit (rc=$RC): $OUT" fi out="$(cd "$repo" && bash scripts/affected-tests.sh --explain plugins/alpha/skills/one/SKILL.md 2>&1)" if contains "$out" "select: plugins/alpha/tests/scan.test.sh (test-scope plugins/alpha/skills/*.md)" && - ! contains "$out" "scans skill bodies"; then + ! contains "$out" "and the probes"; then ok "R8: --explain reports the declared glob" else fail "R8: --explain lacks the declared glob: $out" @@ -1581,7 +1528,7 @@ fi run_sel "$repo" plugins/alpha/skills/one/probe.yaml if [[ "$RC" -eq 0 ]] && has_line "$OUT" plugins/alpha/tests/scan.test.sh; then - ok "R8: a declared file no other rule reaches is mapped, not UNMAPPED" + ok "R8: a file declared on a second header line, which no other rule reaches, is mapped" else fail "R8: a declared file was not mapped by its scope (rc=$RC): $OUT" fi @@ -1593,40 +1540,29 @@ else fail "R8: a plugin file nothing names or declares was mapped (rc=$RC): $OUT" fi -# The list is checked: an entry naming no suite fails every run, and a glob -# matching no file fails the run that changes the list. -cp "$repo/scripts/affected-tests-scopes.txt" "$TMP_ROOT/scopes.keep" -printf 'plugins/alpha/tests/gone.test.sh plugins/alpha/*\n' >>"$repo/scripts/affected-tests-scopes.txt" -out="$(cd "$repo" && bash scripts/affected-tests.sh plugins/beta/hooks/beta-hook.sh 2>&1)" -RC=$? -if [[ "$RC" -eq 2 ]] && contains "$out" "names 'plugins/alpha/tests/gone.test.sh', which is not a suite"; then - ok "R8: an entry naming no suite fails the run" -else - fail "R8: a stale suite entry was not refused (rc=$RC): $out" -fi -cp "$TMP_ROOT/scopes.keep" "$repo/scripts/affected-tests-scopes.txt" -printf 'plugins/alpha/tests/scan.test.sh plugins/nowhere/*\n' >>"$repo/scripts/affected-tests-scopes.txt" +# A glob matching no file fails the run that changes the suite declaring it, +# and only that run. out="$(cd "$repo" && bash scripts/affected-tests.sh plugins/beta/hooks/beta-hook.sh 2>&1)" RC=$? if [[ "$RC" -eq 0 ]]; then - ok "R8: a glob matching nothing is not checked while the list is unchanged" + ok "R8: a glob matching nothing is not checked while its suite is unchanged" else - fail "R8: an unchanged list failed the run (rc=$RC): $out" + fail "R8: an unchanged suite's stale glob failed the run (rc=$RC): $out" fi -out="$(cd "$repo" && bash scripts/affected-tests.sh --allow-unmapped scripts/affected-tests-scopes.txt 2>&1)" +out="$(cd "$repo" && bash scripts/affected-tests.sh plugins/alpha/tests/stale.test.sh 2>&1)" RC=$? -if [[ "$RC" -eq 2 ]] && contains "$out" ' - plugins/nowhere/*'; then - ok "R8: a changed list declaring a glob that matches nothing fails loud" +if [[ "$RC" -eq 2 ]] && contains "$out" "plugins/alpha/tests/stale.test.sh declares test-scope globs" && + contains "$out" ' - plugins/nowhere/*'; then + ok "R8: a changed suite declaring a glob that matches nothing fails loud" else - fail "R8: a stale glob in a changed list was not refused (rc=$RC): $out" + fail "R8: a stale glob in a changed suite was not refused (rc=$RC): $out" fi -cp "$TMP_ROOT/scopes.keep" "$repo/scripts/affected-tests-scopes.txt" -out="$(cd "$repo" && bash scripts/affected-tests.sh --allow-unmapped scripts/affected-tests-scopes.txt 2>&1)" +out="$(cd "$repo" && bash scripts/affected-tests.sh plugins/alpha/tests/scan.test.sh 2>&1)" RC=$? if [[ "$RC" -eq 0 ]]; then - ok "R8: a changed list whose globs all match passes the check" + ok "R8: a changed suite whose globs all match passes the check" else - fail "R8: a valid list was refused (rc=$RC): $out" + fail "R8: a valid declaration was refused (rc=$RC): $out" fi # --- --unmapped-corpus: an unmapped file selects its own language's corpus --- @@ -1664,7 +1600,7 @@ rm -rf "$repo" # --- --with-always is accepted and widens nothing ---------------------------- # A caller that still passes it must neither fail nor get a wider selection: -# the live-tree suites it used to add are declared in the scopes list now. +# the live-tree suites it used to add declare their scopes in their headers now. mk_repo repo run_sel "$repo" --with-always plugins/alpha/hooks/alpha-hook.sh with_out="$OUT" with_rc="$RC" @@ -1867,19 +1803,27 @@ fi rm -rf "$repo" # --- --replay: each commit selected against its parent, with this tree's rules -- -# A replay carries this tree's lists to every commit, since the commits may -# predate them; --against runs the selector at with 's lists and -# prints only the suites the two disagree on. +# A replay carries this tree's no-suite list and declared scopes to every +# commit, since the commits may predate them; --against runs the selector at +# with 's own and prints only the suites the two disagree on. mk_repo repo -printf '#!/usr/bin/env bash\necho old\n' >"$repo/scripts/zz-old-scan.test.sh" -printf '#!/usr/bin/env bash\necho new\n' >"$repo/scripts/zz-new-scan.test.sh" -printf 'scripts/zz-old-scan.test.sh plugins/alpha/*\n' >"$repo/scripts/affected-tests-scopes.txt" +# scan_suite []: a scripts/ suite declaring , or nothing. +scan_suite() { + { + printf '#!/usr/bin/env bash\n' + if [[ -n "${2:-}" ]]; then printf '# test-scope: %s\n' "$2"; fi + printf 'echo %s\n' "$1" + } >"$repo/scripts/zz-$1-scan.test.sh" +} +scan_suite old 'plugins/alpha/*' +scan_suite new git_test_config "$repo" add scripts >/dev/null git_test_config "$repo" commit -qm scans >/dev/null printf '# edited\n' >>"$repo/plugins/alpha/hooks/alpha-hook.sh" git_test_config "$repo" commit -qam 'edit alpha hook' >/dev/null alpha_commit="$(git -C "$repo" rev-parse HEAD)" -printf 'scripts/zz-new-scan.test.sh plugins/alpha/*\n' >"$repo/scripts/affected-tests-scopes.txt" +scan_suite old +scan_suite new 'plugins/alpha/*' out="$(cd "$repo" && bash scripts/affected-tests.sh --replay HEAD~1..HEAD 2>/dev/null)" RC=$? @@ -1904,13 +1848,15 @@ fi # A commit this selector maps to no suite still reports what ran as # dropped, not as added. -printf 'scripts/zz-old-scan.test.sh plugins/beta/*\n' >"$repo/scripts/affected-tests-scopes.txt" +scan_suite old 'plugins/beta/*' +scan_suite new git_test_config "$repo" commit -qam 'scan beta' >/dev/null printf 'notes\n' >"$repo/plugins/beta/zz-notes.yaml" git_test_config "$repo" add plugins >/dev/null git_test_config "$repo" commit -qm 'add beta notes' >/dev/null beta_commit="$(git -C "$repo" rev-parse HEAD)" -printf 'scripts/zz-new-scan.test.sh plugins/alpha/*\n' >"$repo/scripts/affected-tests-scopes.txt" +scan_suite old +scan_suite new 'plugins/alpha/*' out="$(cd "$repo" && bash scripts/affected-tests.sh --replay HEAD~1..HEAD --against HEAD 2>/dev/null)" RC=$? if [[ "$RC" -eq 0 ]] && contains "$out" "commit $beta_commit 0 1 " && diff --git a/scripts/check-docs-only-gate.test.sh b/scripts/check-docs-only-gate.test.sh index de64aff41b..29c46ffc91 100755 --- a/scripts/check-docs-only-gate.test.sh +++ b/scripts/check-docs-only-gate.test.sh @@ -19,6 +19,7 @@ # no scratch git repo is needed. That is deliberate — a fixture repo would need # `git -C

config user.*`, and the un-scoped form of that command writes the # test identity into the CALLER's repo config (claude-code-plugins#2839). +# test-scope: .github/workflows/ci.yml set -uo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" diff --git a/scripts/check-hook-wiring-liveness.test.sh b/scripts/check-hook-wiring-liveness.test.sh index a8f5efeab8..a38d719275 100755 --- a/scripts/check-hook-wiring-liveness.test.sh +++ b/scripts/check-hook-wiring-liveness.test.sh @@ -7,6 +7,7 @@ # repo tree is not sufficient evidence the gate works — these fixtures prove # it goes red on the #2959/#2960 failure class (an unwired hook script) and # green when every non-test script is referenced by a hook command or env. +# test-scope: .claude/settings.json set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/scripts/check-lane-coverage.test.sh b/scripts/check-lane-coverage.test.sh index b89674014d..e024d759b9 100755 --- a/scripts/check-lane-coverage.test.sh +++ b/scripts/check-lane-coverage.test.sh @@ -11,6 +11,7 @@ # names steps of the real ci.yml, and an entry naming a step no fixture defines # is a stale opt-out by construction — so a fixture checked against it would fail # for a reason that has nothing to do with the case under test. +# test-scope: .github/workflows/ci.yml set -uo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" diff --git a/scripts/check-loop-lane-floor-drift.test.sh b/scripts/check-loop-lane-floor-drift.test.sh index c7220d1fa8..2efe559d77 100755 --- a/scripts/check-loop-lane-floor-drift.test.sh +++ b/scripts/check-loop-lane-floor-drift.test.sh @@ -23,6 +23,7 @@ # the SUT's own DATA_CARRIERS for that reason. Note that nothing written here # performs that exemption: the list lives in the gate, so this file cannot # excuse itself, which case 24 asserts. +# test-scope: plugins/*/reference/reader-contract.md set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/scripts/ci-fail-a-draft.test.sh b/scripts/ci-fail-a-draft.test.sh index 507c98d2f5..2a77f96596 100755 --- a/scripts/ci-fail-a-draft.test.sh +++ b/scripts/ci-fail-a-draft.test.sh @@ -6,6 +6,7 @@ # The case that matters is the re-run: a contract-only run drawn while the pull # request was a draft keeps that payload when the full run re-runs it after the # flip to ready, and must then pass. Every other combination stays red. +# test-scope: .github/workflows/ci.yml set -uo pipefail ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" diff --git a/scripts/lib/gate-entry.test.sh b/scripts/lib/gate-entry.test.sh index d184727d78..28643a0225 100755 --- a/scripts/lib/gate-entry.test.sh +++ b/scripts/lib/gate-entry.test.sh @@ -3,6 +3,7 @@ # this process would take the suite down with it, which is the property under # test. # +# test-scope: scripts/*.sh # shellcheck disable=SC2016 # child programs stay single-quoted so this shell does not expand $1 before bash -c set -uo pipefail diff --git a/scripts/lib/test-harness.test.sh b/scripts/lib/test-harness.test.sh index aca8f89f9f..0a4881626d 100755 --- a/scripts/lib/test-harness.test.sh +++ b/scripts/lib/test-harness.test.sh @@ -14,6 +14,7 @@ # forces exit 1 even if report was sabotaged. A throwaway-copy mutation # regression pins both defects: deleting report's return 1, and removing # fail()'s increment. +# test-scope: scripts/*.test.sh set -uo pipefail SELF_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" diff --git a/scripts/validate-plugin-contracts.test.sh b/scripts/validate-plugin-contracts.test.sh index 80afcda3da..302031d337 100755 --- a/scripts/validate-plugin-contracts.test.sh +++ b/scripts/validate-plugin-contracts.test.sh @@ -22,6 +22,7 @@ # aggregate exit code. The one exit-code assertion is the closing real-corpus # case. # +# test-scope: plugins/*/retirements.yaml plugins/*/skills/*/evals/evals.json plugins/*/reference/artifact-protocol.md # shellcheck disable=SC2016 # fixture rows are literal markdown; the backticks they carry are content, never expansion set -uo pipefail From 0b339b51982c0510fcd4a17390532c5552052f64 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 06:18:58 -0400 Subject: [PATCH 13/18] chore(plugins): release the 30 plugins whose suites gained a test-scope header Each plugin that carries a suite with a new `# test-scope:` header gets a patch bump and a CHANGELOG entry naming those suites. Nothing a plugin runs changed; the headers are read only by scripts/affected-tests.sh. Co-Authored-By: Claude Opus 5.5 (1M context) --- plugins/actionlint/.claude-plugin/plugin.json | 2 +- plugins/actionlint/CHANGELOG.md | 6 ++++++ plugins/animation/.claude-plugin/plugin.json | 2 +- plugins/animation/CHANGELOG.md | 6 ++++++ plugins/autonomy/.claude-plugin/plugin.json | 2 +- plugins/autonomy/CHANGELOG.md | 6 ++++++ plugins/code-metrics/.claude-plugin/plugin.json | 2 +- plugins/code-metrics/CHANGELOG.md | 6 ++++++ plugins/code-tidying/.claude-plugin/plugin.json | 2 +- plugins/code-tidying/CHANGELOG.md | 6 ++++++ plugins/context-guard/.claude-plugin/plugin.json | 2 +- plugins/context-guard/CHANGELOG.md | 6 ++++++ plugins/discovery/.claude-plugin/plugin.json | 2 +- plugins/discovery/CHANGELOG.md | 6 ++++++ plugins/disk-hygiene/.claude-plugin/plugin.json | 2 +- plugins/disk-hygiene/CHANGELOG.md | 6 ++++++ plugins/docs-hygiene/.claude-plugin/plugin.json | 2 +- plugins/docs-hygiene/CHANGELOG.md | 6 ++++++ plugins/docs-naming/.claude-plugin/plugin.json | 2 +- plugins/docs-naming/CHANGELOG.md | 6 ++++++ plugins/evals/.claude-plugin/plugin.json | 2 +- plugins/evals/CHANGELOG.md | 6 ++++++ plugins/github/.claude-plugin/plugin.json | 2 +- plugins/github/CHANGELOG.md | 6 ++++++ plugins/guardrails/.claude-plugin/plugin.json | 2 +- plugins/guardrails/CHANGELOG.md | 6 ++++++ plugins/harness-config/.claude-plugin/plugin.json | 2 +- plugins/harness-config/CHANGELOG.md | 6 ++++++ plugins/harness-ops/.claude-plugin/plugin.json | 2 +- plugins/harness-ops/CHANGELOG.md | 6 ++++++ plugins/knowledge/.claude-plugin/plugin.json | 2 +- plugins/knowledge/CHANGELOG.md | 6 ++++++ plugins/multi-agent/.claude-plugin/plugin.json | 2 +- plugins/multi-agent/CHANGELOG.md | 6 ++++++ plugins/pixel-art/.claude-plugin/plugin.json | 2 +- plugins/pixel-art/CHANGELOG.md | 6 ++++++ plugins/planning/.claude-plugin/plugin.json | 2 +- plugins/planning/CHANGELOG.md | 6 ++++++ plugins/playwright/.claude-plugin/plugin.json | 2 +- plugins/playwright/CHANGELOG.md | 6 ++++++ plugins/prototype/.claude-plugin/plugin.json | 2 +- plugins/prototype/CHANGELOG.md | 6 ++++++ plugins/repo-fleet-hygiene/.claude-plugin/plugin.json | 2 +- plugins/repo-fleet-hygiene/CHANGELOG.md | 6 ++++++ plugins/repo-hygiene/.claude-plugin/plugin.json | 2 +- plugins/repo-hygiene/CHANGELOG.md | 6 ++++++ plugins/retro-audio/.claude-plugin/plugin.json | 2 +- plugins/retro-audio/CHANGELOG.md | 6 ++++++ plugins/review/.claude-plugin/plugin.json | 2 +- plugins/review/CHANGELOG.md | 6 ++++++ plugins/session-flow/.claude-plugin/plugin.json | 2 +- plugins/session-flow/CHANGELOG.md | 6 ++++++ plugins/source-control/.claude-plugin/plugin.json | 2 +- plugins/source-control/CHANGELOG.md | 6 ++++++ plugins/speech/.claude-plugin/plugin.json | 2 +- plugins/speech/CHANGELOG.md | 6 ++++++ plugins/testing/.claude-plugin/plugin.json | 2 +- plugins/testing/CHANGELOG.md | 6 ++++++ plugins/work-items/.claude-plugin/plugin.json | 2 +- plugins/work-items/CHANGELOG.md | 6 ++++++ 60 files changed, 210 insertions(+), 30 deletions(-) diff --git a/plugins/actionlint/.claude-plugin/plugin.json b/plugins/actionlint/.claude-plugin/plugin.json index e06705e48d..83fdc97920 100644 --- a/plugins/actionlint/.claude-plugin/plugin.json +++ b/plugins/actionlint/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "actionlint", - "version": "0.12.0", + "version": "0.12.1", "description": "Lint GitHub Actions workflow files on edit via actionlint, surfacing findings as advisory context.", "author": { "name": "Melodic Software", diff --git a/plugins/actionlint/CHANGELOG.md b/plugins/actionlint/CHANGELOG.md index 1441f180e5..0a00491794 100644 --- a/plugins/actionlint/CHANGELOG.md +++ b/plugins/actionlint/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `actionlint` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.12.1] - 2026-10-03 + +### Changed + +- `hooks/actionlint-check.test.sh` declares the files it reads without naming them in a `# test-scope:` header, so CI's test selection runs it when one of them changes. Nothing the plugin runs changed. + ## [0.12.0] - 2026-10-03 ### Added diff --git a/plugins/animation/.claude-plugin/plugin.json b/plugins/animation/.claude-plugin/plugin.json index b644afa983..e639e5c762 100644 --- a/plugins/animation/.claude-plugin/plugin.json +++ b/plugins/animation/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "animation", - "version": "0.3.2", + "version": "0.3.3", "description": "Hand-drawn-style 2D animation as code, rendered in headless Chromium by an ink brush engine. rotoscope copies a reference clip drawing by drawing: trace to vector paths, render, measure against the source (XOR, SSIM), and fit per-shot brush overrides. learn-style measures a clip into a style pack (ships woodcut-ink) and checks films against it. produce turns a brief into approved boards, a shot list, frames, and a delivered file. setup checks ffmpeg, Node, Chromium, and Python.", "author": { "name": "Melodic Software", diff --git a/plugins/animation/CHANGELOG.md b/plugins/animation/CHANGELOG.md index c9e12bac5b..40f46117d5 100644 --- a/plugins/animation/CHANGELOG.md +++ b/plugins/animation/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `animation` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.3.3] - 2026-10-03 + +### Changed + +- `scripts/animation.test.sh` declares the files it reads without naming them in a `# test-scope:` header, so CI's test selection runs it when one of them changes. Nothing the plugin runs changed. + ## [0.3.2] - 2026-10-03 ### Changed diff --git a/plugins/autonomy/.claude-plugin/plugin.json b/plugins/autonomy/.claude-plugin/plugin.json index 048ac11e38..77fdc29a1d 100644 --- a/plugins/autonomy/.claude-plugin/plugin.json +++ b/plugins/autonomy/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "autonomy", - "version": "0.26.0", + "version": "0.26.1", "description": "Contracts for governed autonomous agent operation: role topology, binding seam, telemetry, return accounting, trigger dispatch, a per-work-class guardrail matrix, a standing-routine catalog, and a runner charter. A guided setup skill writes an org's binding, wires OTLP telemetry (file default), human-attested return capture, and signal adapters behind one governed dispatch entrypoint, binds the guardrail matrix to isolation substrates after a live probe, and schedules routines.", "author": { "name": "Melodic Software", diff --git a/plugins/autonomy/CHANGELOG.md b/plugins/autonomy/CHANGELOG.md index b120faf1d3..11b33f9c80 100644 --- a/plugins/autonomy/CHANGELOG.md +++ b/plugins/autonomy/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `autonomy` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.26.1] - 2026-10-03 + +### Changed + +- `skills/setup/scripts/check-prerequisite-resolution-slice.test.sh`, `skills/setup/scripts/check-security-binding.fixtures.test.sh`, `skills/setup/scripts/generate-identity-prerequisites.test.sh`, and `skills/setup/scripts/resolve-prerequisites.fixtures.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.26.0] - 2026-10-03 ### Added diff --git a/plugins/code-metrics/.claude-plugin/plugin.json b/plugins/code-metrics/.claude-plugin/plugin.json index c7d1237a31..eda52c3358 100644 --- a/plugins/code-metrics/.claude-plugin/plugin.json +++ b/plugins/code-metrics/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "code-metrics", - "version": "0.5.2", + "version": "0.5.3", "description": "Read-only code measures for a change, with cited references and no verdict: lines per file (audit-size), cyclomatic, cognitive, and Halstead complexity (audit-complexity), duplication (audit-duplication), per-function coverage and CRAP from existing lcov, Cobertura, coverage.py, or Go artifacts (audit-coverage), TypeScript and Python type debt (audit-type-debt), metric literacy (principles), and setup. Uses only collectors already installed; never installs or runs tests.", "author": { "name": "Melodic Software", diff --git a/plugins/code-metrics/CHANGELOG.md b/plugins/code-metrics/CHANGELOG.md index ca7eb8af13..c4563a7941 100644 --- a/plugins/code-metrics/CHANGELOG.md +++ b/plugins/code-metrics/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `code-metrics` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.5.3] - 2026-10-03 + +### Changed + +- `scripts/dispatch.test.sh`, `scripts/tool-free-path.test.sh`, `skills/audit-complexity/scripts/audit-complexity.test.sh`, `skills/audit-coverage/scripts/audit-coverage.test.sh`, `skills/audit-duplication/scripts/audit-duplication.test.sh`, `skills/audit-size/scripts/audit-size.test.sh`, `skills/audit-type-debt/scripts/audit-type-debt.test.sh`, and `skills/setup/scripts/setup-check.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.5.2] - 2026-10-03 ### Changed diff --git a/plugins/code-tidying/.claude-plugin/plugin.json b/plugins/code-tidying/.claude-plugin/plugin.json index 2248af2647..0d9bbb9acd 100644 --- a/plugins/code-tidying/.claude-plugin/plugin.json +++ b/plugins/code-tidying/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "code-tidying", - "version": "0.28.2", + "version": "0.28.3", "description": "Code tidying and comment hygiene. tidy hunts a rotated lane for Beck-style tidyings and ships one PR. batch-simplify sweeps a branch, time window, or repo in dependency-ordered waves. dissolve-comments deletes zero-information comments and moves the rest into names (safe, aggressive, strip modes). audit-comment-residue flags history, plan, conversational, and ticket residue. audit-dead-code finds dead code with knip, vulture, gopls, and grep. setup scaffolds tidy lanes.", "author": { "name": "Melodic Software", diff --git a/plugins/code-tidying/CHANGELOG.md b/plugins/code-tidying/CHANGELOG.md index 1230ff50b6..e83672a6e0 100644 --- a/plugins/code-tidying/CHANGELOG.md +++ b/plugins/code-tidying/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `code-tidying` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.28.3] - 2026-10-03 + +### Changed + +- `scripts/allowed-tools-pairing.test.sh` and `scripts/evals-fixtures.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.28.2] - 2026-10-03 ### Changed diff --git a/plugins/context-guard/.claude-plugin/plugin.json b/plugins/context-guard/.claude-plugin/plugin.json index 434e4ace27..742e92e120 100644 --- a/plugins/context-guard/.claude-plugin/plugin.json +++ b/plugins/context-guard/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "context-guard", - "version": "0.10.0", + "version": "0.10.1", "description": "Per-session context-window observability. A statusline wrapper writes each session's context_window fields to a snapshot file; a resolver classifies usage into smart, acceptable, and dumb zones from zones.json. Hooks report each move into a worse zone once: the continuation menu to the operator, and only the zone to the model, noting a zone is not a decay signal. Optional blocking mode gates new mutating work in the dumb zone, except handoffs. A PostCompact hook leaves a marker.", "author": { "name": "Melodic Software", diff --git a/plugins/context-guard/CHANGELOG.md b/plugins/context-guard/CHANGELOG.md index 02e328c7fb..0bcc3b174d 100644 --- a/plugins/context-guard/CHANGELOG.md +++ b/plugins/context-guard/CHANGELOG.md @@ -5,6 +5,12 @@ All notable changes to the `context-guard` plugin. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.10.1] - 2026-10-03 + +### Changed + +- `hooks/zone-crossing-inject.test.sh` declares the files it reads without naming them in a `# test-scope:` header, so CI's test selection runs it when one of them changes. Nothing the plugin runs changed. + ## [0.10.0] - 2026-10-03 ### Added diff --git a/plugins/discovery/.claude-plugin/plugin.json b/plugins/discovery/.claude-plugin/plugin.json index 7b66bff722..c1eac81950 100644 --- a/plugins/discovery/.claude-plugin/plugin.json +++ b/plugins/discovery/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "discovery", - "version": "0.28.0", + "version": "0.28.1", "description": "Discovery before changes: explore the local codebase, run multi-source external research, and reconstruct why a past decision was made from evidence outside the code. Each dispatches a subagent by default so the reading stays out of the main thread, with source tiers, falsification, recency gates and a coverage ledger, and persists EXPLORE.md / RESEARCH.md / INTENT.md handoff artifacts. A research sweep workflow (/discovery:research-sweep) backs deep research with adversarial claim verification.", "author": { "name": "Melodic Software", diff --git a/plugins/discovery/CHANGELOG.md b/plugins/discovery/CHANGELOG.md index 6c58244f5a..0c39c99b6b 100644 --- a/plugins/discovery/CHANGELOG.md +++ b/plugins/discovery/CHANGELOG.md @@ -1,5 +1,11 @@ # Changelog: discovery plugin +## [0.28.1] - 2026-10-03 + +### Changed + +- `agents/tool-honesty.test.sh` and `scripts/contract.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.28.0] - 2026-10-03 ### Added diff --git a/plugins/disk-hygiene/.claude-plugin/plugin.json b/plugins/disk-hygiene/.claude-plugin/plugin.json index c5ba742002..334404acfd 100644 --- a/plugins/disk-hygiene/.claude-plugin/plugin.json +++ b/plugins/disk-hygiene/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "disk-hygiene", - "version": "0.43.0", + "version": "0.43.1", "description": "Context-aware disk hygiene for arbitrary directory trees: inventories orphaned and temporary artifacts, classifies evidence into review tiers, and offers exact-path cleanup only after a fresh safety preview and explicit per-tier approval. The target is read-only by default; OS-managed paths, links and mount points, VCS-tracked content without the complete checkout evidence bundle, changed entries, and live-handle uncertainty fail closed.", "author": { "name": "Melodic Software", diff --git a/plugins/disk-hygiene/CHANGELOG.md b/plugins/disk-hygiene/CHANGELOG.md index da26acc618..d38a7c1c8d 100644 --- a/plugins/disk-hygiene/CHANGELOG.md +++ b/plugins/disk-hygiene/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `disk-hygiene` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.43.1] - 2026-10-03 + +### Changed + +- `skills/clean/scripts/hygiene.test.sh`, `skills/clean/scripts/owner_registry.test.sh`, `skills/clean/scripts/test_hygiene.py`, and `skills/setup/scripts/kill_switch_probe.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.43.0] - 2026-10-03 ### Added diff --git a/plugins/docs-hygiene/.claude-plugin/plugin.json b/plugins/docs-hygiene/.claude-plugin/plugin.json index 1b02dbaf9d..2ccd97a882 100644 --- a/plugins/docs-hygiene/.claude-plugin/plugin.json +++ b/plugins/docs-hygiene/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "docs-hygiene", - "version": "0.26.2", + "version": "0.26.3", "description": "Documentation-hygiene toolkit: compress (trim markdown with a semantic-diff check), audit-noise (classify markdown noise), extract-ssot (deduplicate into a single source of truth), audit-encapsulation (citations into skill-private files), rename-references (stale references after renames), audit-derivability (does a doc earn its existence), audit-progressive-disclosure (load tiers), write-for-agents and write-for-humans (authoring). Setup checks markdownlint-cli2.", "author": { "name": "Melodic Software", diff --git a/plugins/docs-hygiene/CHANGELOG.md b/plugins/docs-hygiene/CHANGELOG.md index 37d73a3742..718bc64a86 100644 --- a/plugins/docs-hygiene/CHANGELOG.md +++ b/plugins/docs-hygiene/CHANGELOG.md @@ -1,5 +1,11 @@ # Changelog: docs-hygiene plugin +## [0.26.3] - 2026-10-03 + +### Changed + +- `scripts/allowed-tools-pairing.test.sh` and `skills/compress/scripts/detect-caveman.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.26.2] - 2026-10-03 ### Changed diff --git a/plugins/docs-naming/.claude-plugin/plugin.json b/plugins/docs-naming/.claude-plugin/plugin.json index 980fc39b6d..b88294afe5 100644 --- a/plugins/docs-naming/.claude-plugin/plugin.json +++ b/plugins/docs-naming/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "docs-naming", - "version": "0.2.3", + "version": "0.2.4", "description": "File-name toolkit that plans, applies, and enforces a casing rule across a tracked tree: setup (the one configuration surface), audit-file-names (read-only inventory plus the reference sweep, writing the rename plan), realign-file-names (the executor, one human acceptance per file), and generate-file-name-gate (emits the standalone check that keeps the tree from drifting back).", "author": { "name": "Melodic Software", diff --git a/plugins/docs-naming/CHANGELOG.md b/plugins/docs-naming/CHANGELOG.md index 363338f573..228ea016b8 100644 --- a/plugins/docs-naming/CHANGELOG.md +++ b/plugins/docs-naming/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `docs-naming` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.2.4] - 2026-10-03 + +### Changed + +- `scripts/allowed-tools-pairing.test.sh` declares the files it reads without naming them in a `# test-scope:` header, so CI's test selection runs it when one of them changes. Nothing the plugin runs changed. + ## [0.2.3] - 2026-10-03 ### Changed diff --git a/plugins/evals/.claude-plugin/plugin.json b/plugins/evals/.claude-plugin/plugin.json index 93eece1553..81c38a16d4 100644 --- a/plugins/evals/.claude-plugin/plugin.json +++ b/plugins/evals/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "evals", - "version": "0.5.1", + "version": "0.5.2", "description": "LLM evaluation methodology and eval-suite design, based on Anthropic's evaluation guidance. /evals:methodology answers questions on success criteria, eval design, and grading. /evals:design interviews for measurable success criteria and scaffolds a graded eval suite for an LLM app or Claude Code skill. /evals:plugin-eval preflights the CLI and target, prices a suite before running, and reads the with-versus-without delta. /evals:validate checks case files with no model call.", "author": { "name": "Melodic Software", diff --git a/plugins/evals/CHANGELOG.md b/plugins/evals/CHANGELOG.md index 0ca341208a..9e05e2ef20 100644 --- a/plugins/evals/CHANGELOG.md +++ b/plugins/evals/CHANGELOG.md @@ -1,5 +1,11 @@ # Changelog: evals +## [0.5.2] - 2026-10-03 + +### Changed + +- `skills/plugin-eval/scripts/calibrate-judge.test.sh`, `skills/plugin-eval/scripts/run-validity.test.sh`, and `skills/validate/scripts/validate-cases.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.5.1] - 2026-10-03 ### Changed diff --git a/plugins/github/.claude-plugin/plugin.json b/plugins/github/.claude-plugin/plugin.json index bc869fc0b6..99ddefb275 100644 --- a/plugins/github/.claude-plugin/plugin.json +++ b/plugins/github/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "github", - "version": "0.3.26", + "version": "0.3.27", "description": "GitHub admin-plane audit, advice, and guided setup over the authenticated user's own gh CLI: billing and cost control, security posture, rulesets and settings drift, Actions policy, and every other org/repo/enterprise settings area. Grounded in live state and current official GitHub docs (zero vendored knowledge); read-only by default, every mutation user-in-loop.", "author": { "name": "Melodic Software", diff --git a/plugins/github/CHANGELOG.md b/plugins/github/CHANGELOG.md index 4e45e291a3..deec9aa764 100644 --- a/plugins/github/CHANGELOG.md +++ b/plugins/github/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `github` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.3.27] - 2026-10-03 + +### Changed + +- `github.test.sh` declares the files it reads without naming them in a `# test-scope:` header, so CI's test selection runs it when one of them changes. Nothing the plugin runs changed. + ## [0.3.26] - 2026-10-02 ### Fixed diff --git a/plugins/guardrails/.claude-plugin/plugin.json b/plugins/guardrails/.claude-plugin/plugin.json index d1b8bac6b9..70bafc5eda 100644 --- a/plugins/guardrails/.claude-plugin/plugin.json +++ b/plugins/guardrails/.claude-plugin/plugin.json @@ -170,5 +170,5 @@ "min": 1 } }, - "version": "0.47.0" + "version": "0.47.1" } diff --git a/plugins/guardrails/CHANGELOG.md b/plugins/guardrails/CHANGELOG.md index ba9beee518..0024a59e92 100644 --- a/plugins/guardrails/CHANGELOG.md +++ b/plugins/guardrails/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `guardrails` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.47.1] - 2026-10-03 + +### Changed + +- `hooks/abort-boundary.test.sh`, `hooks/coverage-manifest.test.sh`, `hooks/require-jq-notice-isolation.test.sh`, `hooks/require-jq-posture.test.sh`, and `hooks/run-guards.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.47.0] - 2026-10-03 ### Added diff --git a/plugins/harness-config/.claude-plugin/plugin.json b/plugins/harness-config/.claude-plugin/plugin.json index f44036b0cb..b62a379bae 100644 --- a/plugins/harness-config/.claude-plugin/plugin.json +++ b/plugins/harness-config/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "harness-config", - "version": "1.7.3", + "version": "1.7.4", "description": "Configuration health for a repo's Claude Code: audit (settings, .mcp.json, hooks, plugins, permission drift), audit-automation-gaps, audit-permission-grants, audit-permission-state (effective rules with provenance), draft-auto-mode-rules (prints an autoMode block), audit-instructions (instructions the model no longer needs, conflicts), audit-prompting-postures, audit-pass (one ordered, resumable pass), unhobble (strip instructions, re-add what evidence earns), and setup.", "author": { "name": "Melodic Software", diff --git a/plugins/harness-config/CHANGELOG.md b/plugins/harness-config/CHANGELOG.md index 13879e2756..6e04c2be12 100644 --- a/plugins/harness-config/CHANGELOG.md +++ b/plugins/harness-config/CHANGELOG.md @@ -5,6 +5,12 @@ All notable changes to the `harness-config` plugin are documented here. Format f Versions 0.51.8 to 0.51.9 and 0.51.11 to 0.51.14 were reserved by parallel branches and never released. +## [1.7.4] - 2026-10-03 + +### Changed + +- `skills/audit-automation-gaps/scripts/inventory.test.sh` and `skills/audit-permission-state/scripts/audit.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [1.7.3] - 2026-10-03 ### Changed diff --git a/plugins/harness-ops/.claude-plugin/plugin.json b/plugins/harness-ops/.claude-plugin/plugin.json index 95c9c4d69f..cdbe96909e 100644 --- a/plugins/harness-ops/.claude-plugin/plugin.json +++ b/plugins/harness-ops/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "harness-ops", - "version": "3.6.0", + "version": "3.6.1", "description": "Claude Code operations: audit-skill-visibility (skills the listing budget hides), inventory (all commands, skills, agents, tools, plugins), audit-install-state (~/.claude), audit-performance (slowness), audit-native-overlap (skills duplicating built-ins), observability (telemetry), known-issues (Claude bugs, status), changelog, prerequisites, check, machine-profile, plugins (update the fleet), morning-brief, lanes (background loop sessions), setup. Plus opt-in hook event logs.", "author": { "name": "Melodic Software", diff --git a/plugins/harness-ops/CHANGELOG.md b/plugins/harness-ops/CHANGELOG.md index c15befeca4..6d1cf7d672 100644 --- a/plugins/harness-ops/CHANGELOG.md +++ b/plugins/harness-ops/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `harness-ops` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [3.6.1] - 2026-10-03 + +### Changed + +- `hooks/audit-session-id.test.sh`, `skills/plugins/scripts/fleet-state.test.sh`, and `skills/plugins/scripts/sync-run.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [3.6.0] - 2026-10-03 ### Added diff --git a/plugins/knowledge/.claude-plugin/plugin.json b/plugins/knowledge/.claude-plugin/plugin.json index 22d419169d..08690ba53b 100644 --- a/plugins/knowledge/.claude-plugin/plugin.json +++ b/plugins/knowledge/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "knowledge", - "version": "0.19.2", + "version": "0.19.3", "description": "Ingests external knowledge into synthesized artifacts: book-distill (PDF or EPUB into author-attributed reference files), video-digest (one YouTube or X video: transcript, links, repo applicability), course-digest (Dometrain and Teachable courses into recommendations), docpage-digest (one documentation page into a verified knowledge slice), and map-corpus (a multi-resource corpus into a classified link map and an approved docpage-digest queue). setup sets where artifacts land.", "author": { "name": "Melodic Software", diff --git a/plugins/knowledge/CHANGELOG.md b/plugins/knowledge/CHANGELOG.md index f4e9ae665e..d6e5e44834 100644 --- a/plugins/knowledge/CHANGELOG.md +++ b/plugins/knowledge/CHANGELOG.md @@ -4,6 +4,12 @@ All notable changes to the `knowledge` plugin are recorded here. The `version` i `.claude-plugin/plugin.json` is the delivery vehicle. A consumer receives a change only after that version increases. +## [0.19.3] - 2026-10-03 + +### Changed + +- `skills/docpage-digest/scripts/check-html-rows.test.sh` and `skills/docpage-digest/scripts/extract_blog_body.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.19.2] - 2026-10-03 ### Changed diff --git a/plugins/multi-agent/.claude-plugin/plugin.json b/plugins/multi-agent/.claude-plugin/plugin.json index 2d8b74570a..e01f3ada13 100644 --- a/plugins/multi-agent/.claude-plugin/plugin.json +++ b/plugins/multi-agent/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "multi-agent", - "version": "0.4.0", + "version": "0.4.1", "description": "Workflow and model-routing guidance for multi-agent work: assess (workflow, subagent or single context), route (the role map a workflow script reads from args, keeping fan-out stages off a frontier model), audit-defaults (rechecks bundled defaults against upstream, or sweeps the repo's model and workflow guidance, via a read-only drift-audit workflow), check (whether node resolves and the fetch gate is registered), and setup (writes a user, team or local layer after a preview and a yes).", "author": { "name": "Melodic Software", diff --git a/plugins/multi-agent/CHANGELOG.md b/plugins/multi-agent/CHANGELOG.md index 6d73864603..c6fa4b64f2 100644 --- a/plugins/multi-agent/CHANGELOG.md +++ b/plugins/multi-agent/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `multi-agent` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.4.1] - 2026-10-03 + +### Changed + +- `scripts/allowed-tools-pairing.test.sh` and `tests/drift-audit.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.4.0] - 2026-10-03 ### Added diff --git a/plugins/pixel-art/.claude-plugin/plugin.json b/plugins/pixel-art/.claude-plugin/plugin.json index 32a8917dfc..d2073fe01c 100644 --- a/plugins/pixel-art/.claude-plugin/plugin.json +++ b/plugins/pixel-art/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "pixel-art", - "version": "0.6.1", + "version": "0.6.2", "description": "Creates pixel art with no external tools: sprites (sprite), animation sprite sheets for RPG Maker MZ, Godot, PICO-8, or plain strips with Aseprite-style frame data and GIF previews (animate), tilesets and parallax (tileset), UI skins and bitmap fonts (ui), effect sheets (vfx), and animated scenes as one HTML file (scene). Specs are palette-locked; a humanoid kit and a Python stdlib renderer write PNG, GIF, and frame data with a render-review loop. Optional Aseprite adapter.", "author": { "name": "Melodic Software", diff --git a/plugins/pixel-art/CHANGELOG.md b/plugins/pixel-art/CHANGELOG.md index c51c9b8c05..67b09a361c 100644 --- a/plugins/pixel-art/CHANGELOG.md +++ b/plugins/pixel-art/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `pixel-art` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.6.2] - 2026-10-03 + +### Changed + +- `scripts/backends.test.sh` declares the files it reads without naming them in a `# test-scope:` header, so CI's test selection runs it when one of them changes. Nothing the plugin runs changed. + ## [0.6.1] - 2026-10-03 ### Changed diff --git a/plugins/planning/.claude-plugin/plugin.json b/plugins/planning/.claude-plugin/plugin.json index 49dfb6e307..86dc931fe6 100644 --- a/plugins/planning/.claude-plugin/plugin.json +++ b/plugins/planning/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "planning", - "version": "0.65.3", + "version": "0.65.4", "userConfig": { "surface": { "type": "string", diff --git a/plugins/planning/CHANGELOG.md b/plugins/planning/CHANGELOG.md index 562d2b7a98..548e7378e9 100644 --- a/plugins/planning/CHANGELOG.md +++ b/plugins/planning/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `planning` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.65.4] - 2026-10-03 + +### Changed + +- `surface/surface.test.sh`, `surface/test_exporters.py`, `surface/test_round.py`, `surface/test_schema.py`, `surface/test_server.py`, `surface/watch.test.sh`, and `tests/reattach-slice.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.65.3] - 2026-10-03 ### Changed diff --git a/plugins/playwright/.claude-plugin/plugin.json b/plugins/playwright/.claude-plugin/plugin.json index 81a3c0896c..717abbdf19 100644 --- a/plugins/playwright/.claude-plugin/plugin.json +++ b/plugins/playwright/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "playwright", - "version": "0.8.6", + "version": "0.8.7", "description": "Live E2E browser automation via Microsoft's @playwright/cli: named sessions, accessibility-ref snapshots, click/fill by ref, screenshots, console and network capture, mocking, tracing, video, and auth state, with artifacts written to disk so only paths enter context, plus a vendored upstream baseline and maintainer drift-check update flow.", "author": { "name": "Melodic Software", diff --git a/plugins/playwright/CHANGELOG.md b/plugins/playwright/CHANGELOG.md index ad0913e8b1..9192320bc5 100644 --- a/plugins/playwright/CHANGELOG.md +++ b/plugins/playwright/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `playwright` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.8.7] - 2026-10-03 + +### Changed + +- `skills/playwright/scripts/update.test.sh` declares the files it reads without naming them in a `# test-scope:` header, so CI's test selection runs it when one of them changes. Nothing the plugin runs changed. + ## [0.8.6] - 2026-10-03 ### Changed diff --git a/plugins/prototype/.claude-plugin/plugin.json b/plugins/prototype/.claude-plugin/plugin.json index 0141607a0f..2c93123067 100644 --- a/plugins/prototype/.claude-plugin/plugin.json +++ b/plugins/prototype/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "prototype", - "version": "0.13.9", + "version": "0.13.10", "description": "Builds throwaway code to answer a design question before committing to architecture: a logic facet (an interactive terminal app over a portable state model) and a UI facet (radically different visual variants on one route).", "author": { "name": "Melodic Software", diff --git a/plugins/prototype/CHANGELOG.md b/plugins/prototype/CHANGELOG.md index 7e1af761f2..bd2dcc9201 100644 --- a/plugins/prototype/CHANGELOG.md +++ b/plugins/prototype/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `prototype` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.13.10] - 2026-10-03 + +### Changed + +- `scripts/allowed-tools-pairing.test.sh`, `skills/explore-directions/scripts/detect-ecosystems.test.sh`, and `skills/pressure-test/scripts/detect-ecosystems.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.13.9] - 2026-10-02 ### Fixed diff --git a/plugins/repo-fleet-hygiene/.claude-plugin/plugin.json b/plugins/repo-fleet-hygiene/.claude-plugin/plugin.json index c980f24358..f6911e3fd3 100644 --- a/plugins/repo-fleet-hygiene/.claude-plugin/plugin.json +++ b/plugins/repo-fleet-hygiene/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "repo-fleet-hygiene", - "version": "0.28.1", + "version": "0.28.2", "description": "Cross-repository Git/GitHub fleet discovery, evidence rollup, a gated apply verb, and a sync verb that fast-forwards canonical checkouts onto the remote default branch. Audit stays read-only. apply and sync mutate only with --apply plus interactive confirmation or --yes.", "author": { "name": "Melodic Software", diff --git a/plugins/repo-fleet-hygiene/CHANGELOG.md b/plugins/repo-fleet-hygiene/CHANGELOG.md index dbb7efd02a..371b171300 100644 --- a/plugins/repo-fleet-hygiene/CHANGELOG.md +++ b/plugins/repo-fleet-hygiene/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to `repo-fleet-hygiene` are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.28.2] - 2026-10-03 + +### Changed + +- `scripts/allowed-tools-pairing.test.sh` declares the files it reads without naming them in a `# test-scope:` header, so CI's test selection runs it when one of them changes. Nothing the plugin runs changed. + ## [0.28.1] - 2026-10-03 ### Changed diff --git a/plugins/repo-hygiene/.claude-plugin/plugin.json b/plugins/repo-hygiene/.claude-plugin/plugin.json index cae1c0206a..3bb56101b0 100644 --- a/plugins/repo-hygiene/.claude-plugin/plugin.json +++ b/plugins/repo-hygiene/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "repo-hygiene", - "version": "0.19.2", + "version": "0.19.3", "description": "Repo hygiene action-router: /repo-hygiene:clean sweeps reclaimable caches, build artifacts, and stale git metadata, and can realign the working tree to a fresh-pull state, dry-run-first, with destructive tiers gated behind explicit confirmation and a session-scoped destructive-command guard. Ecosystem targets are detected at runtime; secrets, runtime dependencies, and skill data are preserved by default.", "author": { "name": "Melodic Software", diff --git a/plugins/repo-hygiene/CHANGELOG.md b/plugins/repo-hygiene/CHANGELOG.md index cf67f7f715..48370c89c2 100644 --- a/plugins/repo-hygiene/CHANGELOG.md +++ b/plugins/repo-hygiene/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `repo-hygiene` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.19.3] - 2026-10-03 + +### Changed + +- `scripts/allowed-tools-pairing.test.sh` and `skills/clean/scripts/destructive-guard.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.19.2] - 2026-10-03 ### Changed diff --git a/plugins/retro-audio/.claude-plugin/plugin.json b/plugins/retro-audio/.claude-plugin/plugin.json index a891f613bb..b856e75bd3 100644 --- a/plugins/retro-audio/.claude-plugin/plugin.json +++ b/plugins/retro-audio/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "retro-audio", - "version": "0.2.1", + "version": "0.2.2", "description": "Renders retro sound effects and short chiptune loops to WAV with the Python standard library only: an sfxr-style parameter model and an MML subset with Game Boy and NES pulse duties and channel limits for Game Boy, NES, and PICO-8 (four channels, one noise part). Another plugin can play the WAV; this one does not read that plugin's files.", "author": { "name": "Melodic Software", diff --git a/plugins/retro-audio/CHANGELOG.md b/plugins/retro-audio/CHANGELOG.md index c1112cbb4b..6c9006cf48 100644 --- a/plugins/retro-audio/CHANGELOG.md +++ b/plugins/retro-audio/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `retro-audio` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.2.2] - 2026-10-03 + +### Changed + +- `scripts/audio.test.sh` declares the files it reads without naming them in a `# test-scope:` header, so CI's test selection runs it when one of them changes. Nothing the plugin runs changed. + ## [0.2.1] - 2026-10-03 ### Changed diff --git a/plugins/review/.claude-plugin/plugin.json b/plugins/review/.claude-plugin/plugin.json index 6fdf4b0d5c..f6ba9fba79 100644 --- a/plugins/review/.claude-plugin/plugin.json +++ b/plugins/review/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "review", - "version": "0.37.2", + "version": "0.37.3", "description": "Code-review toolkit: six reviewer agents, read-only over the reviewed code (code, security, architecture, doc drift, build/test/lint, CI-log audit), plus orchestration skills for the quality gate, fan-out, and enforceability audit (/review:audit-enforceability), an offered HTML pull-request explainer (/review:pr-explainer), a fan-out sweep workflow (/review:fanout-sweep), and CI lane commands (/review:code-review, /review:security-review) for org reusable workflows.", "author": { "name": "Melodic Software", diff --git a/plugins/review/CHANGELOG.md b/plugins/review/CHANGELOG.md index 35268e0cd4..49e92f0ca8 100644 --- a/plugins/review/CHANGELOG.md +++ b/plugins/review/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `review` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.37.3] - 2026-10-03 + +### Changed + +- `tests/change-set-block.test.sh` declares the files it reads without naming them in a `# test-scope:` header, so CI's test selection runs it when one of them changes. Nothing the plugin runs changed. + ## [0.37.2] - 2026-10-03 ### Changed diff --git a/plugins/session-flow/.claude-plugin/plugin.json b/plugins/session-flow/.claude-plugin/plugin.json index e976509a4f..52575d3e48 100644 --- a/plugins/session-flow/.claude-plugin/plugin.json +++ b/plugins/session-flow/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "session-flow", - "version": "0.48.0", + "version": "0.48.1", "description": "Session lifecycle: workflow (next stage), handoff (save-point, resume prompt), continue-in-background, keep-going (resume after interruption or stall), find-handoff (recover a lost handoff), clean-stop (durable stopping point), retro and running-retro (retrospectives), audit-sessions, orient (where the session stands), orchestrate (delegation imperatives), reanchor (verify assumptions), reconcile (retire finished work), show-options (ranked skill menu), tidy-work (.work tiers), check, setup.", "author": { "name": "Melodic Software", diff --git a/plugins/session-flow/CHANGELOG.md b/plugins/session-flow/CHANGELOG.md index 879321c396..576ea4bae9 100644 --- a/plugins/session-flow/CHANGELOG.md +++ b/plugins/session-flow/CHANGELOG.md @@ -1,5 +1,11 @@ # Changelog: session-flow plugin +## [0.48.1] - 2026-10-03 + +### Changed + +- `scripts/save_point.test.sh` and `skills/audit-sessions/scripts/audit-sessions.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.48.0] - 2026-10-03 ### Added diff --git a/plugins/source-control/.claude-plugin/plugin.json b/plugins/source-control/.claude-plugin/plugin.json index ffce08d33c..fdd2beea03 100644 --- a/plugins/source-control/.claude-plugin/plugin.json +++ b/plugins/source-control/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "source-control", - "version": "0.79.0", + "version": "0.79.1", "description": "Git and GitHub delivery: /commit (convention-checked subject, Co-authored-by trailer, surgical staging), /pull-request (prep, create, CI monitoring, review triage, merge, CI logs), /babysit-prs (safe-by-default PR fleet loop, opt-in worker and autopilot tiers), /babysit-loop (merge lane; merge is human until the repo adopts it), /worktree, /resolve-conflicts (intent-first), /check, and /setup (layered source-control.md convention config; Conventional Commits by default).", "author": { "name": "Melodic Software", diff --git a/plugins/source-control/CHANGELOG.md b/plugins/source-control/CHANGELOG.md index e7426da742..a68a917fb5 100644 --- a/plugins/source-control/CHANGELOG.md +++ b/plugins/source-control/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `source-control` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.79.1] - 2026-10-03 + +### Changed + +- `scripts/babysit-wrapper-help.test.sh`, `skills/babysit-prs/scripts/engine.test.sh`, `skills/babysit-prs/scripts/tests/test_guards.py`, and `skills/worktree/nesting-invariant-ssot.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.79.0] - 2026-10-03 ### Added diff --git a/plugins/speech/.claude-plugin/plugin.json b/plugins/speech/.claude-plugin/plugin.json index 81577c9b55..82dcbafd61 100644 --- a/plugins/speech/.claude-plugin/plugin.json +++ b/plugins/speech/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "speech", - "version": "0.1.4", + "version": "0.1.5", "description": "Text-to-speech narration. The narrate skill turns a script into narration.wav plus words.json, a start and end time for every word. The kokoro backend runs Kokoro-82M (Apache-2.0) locally through onnxruntime, with timings from the model's own durations. espeak-ng (GPL-3.0) is installed by you, never by the plugin. A SessionStart hook installs the locked Python packages, setup downloads the pinned model files, and check reports each missing prerequisite.", "author": { "name": "Melodic Software", diff --git a/plugins/speech/CHANGELOG.md b/plugins/speech/CHANGELOG.md index 5a289efb80..549e613c46 100644 --- a/plugins/speech/CHANGELOG.md +++ b/plugins/speech/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `speech` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.1.5] - 2026-10-03 + +### Changed + +- `scripts/speech.test.sh` declares the files it reads without naming them in a `# test-scope:` header, so CI's test selection runs it when one of them changes. Nothing the plugin runs changed. + ## [0.1.4] - 2026-10-03 ### Changed diff --git a/plugins/testing/.claude-plugin/plugin.json b/plugins/testing/.claude-plugin/plugin.json index c951f38eb5..5f4e0817aa 100644 --- a/plugins/testing/.claude-plugin/plugin.json +++ b/plugins/testing/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "testing", - "version": "0.22.0", + "version": "0.22.1", "description": "Test-stage discipline: /testing:plan (coverage gaps, test plans), /testing:write (TDD authoring), /testing:run-e2e (live E2E and smoke checks), /testing:diagnose (failing-test root cause, with a fix-until-green workflow across files), /testing:audit (can't-fail test audit with a fail-closed gate), /testing:cleanup (rewrite, quarantine, or delete low-value tests behind a mutation gate), /testing:setup, and opt-in hooks that scan test files Claude writes and flag edits that weaken tests.", "author": { "name": "Melodic Software", diff --git a/plugins/testing/CHANGELOG.md b/plugins/testing/CHANGELOG.md index 35436d36a8..50dd569e5e 100644 --- a/plugins/testing/CHANGELOG.md +++ b/plugins/testing/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `testing` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.22.1] - 2026-10-03 + +### Changed + +- `scripts/gen-hook-filters.test.sh` and `skills/setup/scripts/setup.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.22.0] - 2026-10-03 ### Added diff --git a/plugins/work-items/.claude-plugin/plugin.json b/plugins/work-items/.claude-plugin/plugin.json index 8b0ce17938..6b9f401e9c 100644 --- a/plugins/work-items/.claude-plugin/plugin.json +++ b/plugins/work-items/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "work-items", - "version": "0.47.0", + "version": "0.47.1", "description": "Work items through a provider-neutral tracker seam (github, local-markdown, jira, gitea, linear adapters): track (dashboard, creation, race-safe claims, recurring-schedule checks, stale-lease audits), scan-todos, decompose (plans into vertical-slice items), ship (route a spec container), triage (raw intake and unsolicited PRs), work, work-loop (autonomous PR-only drain), attend-queue (escalations), onboard-adapter (new tracker adapter), and setup (binds the provider).", "author": { "name": "Melodic Software", diff --git a/plugins/work-items/CHANGELOG.md b/plugins/work-items/CHANGELOG.md index b19ad13281..b7931f2247 100644 --- a/plugins/work-items/CHANGELOG.md +++ b/plugins/work-items/CHANGELOG.md @@ -3,6 +3,12 @@ All notable changes to the `work-items` plugin are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); this plugin uses semantic versioning. +## [0.47.1] - 2026-10-03 + +### Changed + +- `tests/no-hardcoded-priority-scheme.test.sh`, `tools/work-item-tracker/adapters/gitea/list-items.test.sh`, `tools/work-item-tracker/adapters/local-markdown/claim-integrity.test.sh`, `tools/work-item-tracker/adapters/local-markdown/list-sub-items.test.sh`, `tools/work-item-tracker/adapters/local-markdown/renew-lease.test.sh`, `tools/work-item-tracker/conformance/bindings/jira.test.sh`, `tools/work-item-tracker/conformance/bindings/local-markdown.test.sh`, and `tools/work-item-tracker/work-item-tracker.test.sh` declare the files they read without naming them in `# test-scope:` headers, so CI's test selection runs them when one of those files changes. Nothing the plugin runs changed. + ## [0.47.0] - 2026-10-03 ### Added From 40f19383119540e75534817d20710fe9ab0eb10f Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 06:23:21 -0400 Subject: [PATCH 14/18] fix(scripts): resolve the --against ref through changed_files::verify_base scripts/lib/gate-entry.test.sh bans a hand-rolled `rev-parse --verify` base-ref predicate outside lib/changed-files.sh; the replay's read of 's declared scopes now validates the ref with the shared helper. Co-Authored-By: Claude Opus 5.5 (1M context) --- scripts/affected-tests.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/affected-tests.sh b/scripts/affected-tests.sh index ed9873bb42..0862f335f7 100755 --- a/scripts/affected-tests.sh +++ b/scripts/affected-tests.sh @@ -1130,7 +1130,7 @@ run_replay() { rm -f "$against/always.txt" # 's own declared scopes, read from its suites in the scratch clone. against_scopes="$against/scopes.txt" - if ! against_sha="$(git rev-parse --verify -q "$against_ref^{commit}")" || + if ! changed_files::verify_base "$against_ref" || ! against_sha="$(git rev-parse "$against_ref")" || ! git -C "$tree" -c advice.detachedHead=false checkout -q --detach "$against_sha" || ! git -C "$tree" ls-files >"$against/files" || ! (cd "$tree" && scope_declarations "$against/files") >"$against_scopes"; then From 663d810d78670df8ea06a0aa5db79af5470fe922 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 07:12:02 -0400 Subject: [PATCH 15/18] fix(scripts): declare the live files affected-tests.test.sh reads by glob The suite's LIVE cases read the github advise and planning interview skill bodies and the autonomy reference docs through globs its test-scope header did not declare. With the always list gone, a pull request renaming or deleting one of those files selected nothing that ran this suite, and the break surfaced on main. The header now declares all three globs. The reference YAML case no longer reads the live files. It runs the selector in a fixture that carries the live no-suite list, on probe files in the two live directories. Declaring the live YAML instead would map it to this suite and stop it failing loud, which the no-suite list wants until the files get a real lane, and a live run failed whenever any other suite began naming one. Co-Authored-By: Claude Opus 5.5 (1M context) --- scripts/affected-tests.test.sh | 57 +++++++++++++++++++--------------- 1 file changed, 32 insertions(+), 25 deletions(-) diff --git a/scripts/affected-tests.test.sh b/scripts/affected-tests.test.sh index 8d8fff85b2..1b7f950535 100755 --- a/scripts/affected-tests.test.sh +++ b/scripts/affected-tests.test.sh @@ -9,6 +9,8 @@ # list — because a synthetic fixture cannot show that the derivation still # tracks reality, which is the whole failure mode this tool exists to avoid. # test-scope: scripts/affected-tests* scripts/sync-*.sh scripts/lib/sync-*.sh .github/workflows/ci.yml +# test-scope: plugins/github/skills/advise/S*.md plugins/planning/skills/interview/S*.md +# test-scope: plugins/autonomy/reference/*.md set -uo pipefail TMP_ROOT="$(mktemp -d)" @@ -826,7 +828,7 @@ else fail "standards-contract fan-out (rc=$RC): $out" fi -# --- LIVE repo: reference YAML with no lane is UNMAPPED, not silently clean -- +# --- LIVE no-suite list: reference YAML with no lane is UNMAPPED ------------- # The reference YAML under plugins/toolchain/ and docs/conventions/ # ecosystem-commands/ is read by NO lane: no yamllint step exists, every # check-jsonschema step names its files and none names these, @@ -839,29 +841,31 @@ fi # class again, this assertion is SUPPOSED to fail — update it together with the # no-suite entry, and make sure the entry names the lane that actually reads # them. Workflow YAML must stay covered via the .github/* entry throughout. -# The probe paths are DISCOVERED with a glob, never written out literally. That -# is not tidiness — a literal basename here would make this file a suite that -# "references" the probe, R3 would select it, and the path would come back -# MAPPED at exit 0. The assertion would then fail for a reason that has nothing -# to do with the no-suite list. This is the MATCHING rule documented in -# affected-tests.sh's header, met head-on: naming a file in a suite is exactly -# what makes the selector consider it covered. -# The exit status of this head pipe is never read, so its early exit is harmless. -mapfile -t eco_yaml < <(cd "$REPO_ROOT" && git ls-files \ - 'plugins/toolchain/reference/ecosystems/*.yaml' \ - 'docs/conventions/ecosystem-commands/examples/*.yaml' | head -2) -if [[ ${#eco_yaml[@]} -eq 0 ]]; then - fail "no reference YAML found to probe — the case below would be vacuous" -fi -for y in ${eco_yaml[@]+"${eco_yaml[@]}"}; do - out="$(cd "$REPO_ROOT" && bash scripts/affected-tests.sh "$y" 2>&1)" - RC=$? - if [[ "$RC" -eq 1 ]] && contains "$out" 'UNMAPPED'; then +# +# The selector runs in a fixture that carries the live no-suite list, on probe +# files in the two live directories whose names no live file has. A live file +# would not do: a suite that names or globs one maps it, so this suite would map +# the files it proves unmapped, and a live run would also fail on main as soon +# as any other suite began naming one. +eco_yaml=(plugins/toolchain/reference/ecosystems/zz-probe.yaml + docs/conventions/ecosystem-commands/examples/zz-probe.yaml) +mk_repo repo +for y in "${eco_yaml[@]}"; do + mkdir -p "$repo/${y%/*}" + printf 'name: probe\n' >"$repo/$y" +done +git_test_config "$repo" add plugins docs >/dev/null +git_test_config "$repo" commit -qm eco-yaml >/dev/null +out="$(cd "$repo" && bash scripts/affected-tests.sh "${eco_yaml[@]}" 2>&1)" +RC=$? +for y in "${eco_yaml[@]}"; do + if [[ "$RC" -eq 1 ]] && has_line "$out" " - $y"; then ok "reference YAML with no covering lane is UNMAPPED: $y" else fail "$y should be UNMAPPED, not silently covered (rc=$RC): $out" fi done +rm -rf "$repo" # ... while YAML under .github/, which actionlint/zizmor/check-jsonschema DO # read, stays covered by the .github/* entry. Without this, the cases above @@ -874,7 +878,8 @@ done # no-suite list is consulted, which would pass a bare rc-0 check while proving # nothing about .github/*. That is not hypothetical — .github/workflows/ci.yml # is named by two suites and reaches exit 0 through R3, so it cannot serve as -# this probe. Discovered by glob for the R3 reason given above. +# this probe. Discovered by glob, never spelled: a path spelled here would make +# this suite name it, and R3 would select this suite the same way. # # The candidate is additionally FILTERED to one that no grepped-language file # names at all, rather than assuming the sole `.github/*.yaml` qualifies. That @@ -1465,9 +1470,11 @@ rm -rf "$repo" # --- LIVE repo: a real autonomy reference doc selects the contract suite ----- # The probe doc is discovered, never spelled: a basename written here would make -# this suite name that file, and R3 would then cover it without R7. The suite -# path above IS spelled on purpose: renaming the suite selects this file, and -# this case then fails instead of R7 silently selecting nothing. +# this suite name that file, and R3 would then cover it without R7. The glob is +# declared in this suite's test-scope header, so deleting the last doc selects +# this suite. The suite path above IS spelled on purpose: renaming the suite +# selects this file, and this case then fails instead of R7 silently selecting +# nothing. # The assertion is on the R7 reason, not on bare selection: a live doc can also # reach the suite through R4 fan-out (a hook naming it), which would pass without # R7. The seed is walked first, so R7's reason is the one recorded. @@ -1616,8 +1623,8 @@ rm -rf "$repo" # Both were a skill body edit breaking a suite that never spells the body's # path the plain way: one scans its plugin's markdown (R8 declares it), the # other spells the body relative to its plugin (AMBIGUOUS NAMES resolves it). -# The probe bodies are discovered, never spelled, so this suite does not name -# them and run on every edit to them. +# The probe bodies are discovered by glob and declared in this suite's +# test-scope header, so renaming or deleting either selects this suite. for probe in 'plugins/github/skills/advise/S*.md|plugins/github/github.test.sh' \ 'plugins/planning/skills/interview/S*.md|plugins/planning/tests/interview-defenses.test.sh'; do body="$(cd "$REPO_ROOT" && git ls-files "${probe%%|*}")" From 74fa5e2a4c25008b68a59f60dd0cb4072eb6dd53 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 11:30:39 -0400 Subject: [PATCH 16/18] fix(scripts): select the .test.sh wrapper of every selected Node suite (R9) CI runs a Node suite that has a sibling .test.sh only through that wrapper: scripts/run-outside-node-suites.sh reports it as owned by the wrapper and runs nothing. The selector stops walking at a reached suite, so a change that reaches the Node suite through its source file (lib/exec-bash.mjs reaching exec-bash.resolver.test.mjs) selected the suite without its wrapper, and CI ran neither. R9 adds the wrapper of every selected .test.{js,mjs,cjs} after every other rule and before --shard. Co-Authored-By: Claude Opus 5.5 (1M context) --- scripts/affected-tests.sh | 13 +++++++++++++ scripts/affected-tests.test.sh | 23 +++++++++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/scripts/affected-tests.sh b/scripts/affected-tests.sh index 0862f335f7..07a4d64e55 100755 --- a/scripts/affected-tests.sh +++ b/scripts/affected-tests.sh @@ -112,6 +112,12 @@ # a `#` after them starts a comment. Suites whose comments # start with `#` (shell, Python, Pester) can declare. A glob # matching no file fails the run that changes its suite. +# R9 wrapper a selected .test.js, .test.mjs or .test.cjs whose +# directory holds .test.sh selects that wrapper too, +# however the Node suite was reached. CI runs such a suite only +# through its wrapper (scripts/run-outside-node-suites.sh +# leaves it to the wrapper), so the Node suite alone runs +# nothing. Applied after every other rule, before --shard. # # MATCHING. One file NAMES another when the basename stands in a line as a WHOLE # PATH TOKEN: bounded on both sides by a character outside [A-Za-z0-9_.-]. `/` @@ -1320,6 +1326,13 @@ fi # an indirect reference and rejects the expanded key list as a variable name. declare -a selected=() if [[ ${#SUITES[@]} -gt 0 ]]; then + # R9. The key list is expanded once, before the loop adds to it. + for s in "${!SUITES[@]}"; do + case "$s" in + *.test.js | *.test.mjs | *.test.cjs) add_suite "${s%.test.*}.test.sh" "wraps $s" || true ;; + *) ;; + esac + done # Checked, and read from a file: a failing `sort` here would empty a # NON-EMPTY selection and report "every changed file is a no-suite class". if ! printf '%s\n' "${!SUITES[@]}" | sort -u >"$WORK_DIR/selected"; then diff --git a/scripts/affected-tests.test.sh b/scripts/affected-tests.test.sh index 1b7f950535..d6e19c530c 100755 --- a/scripts/affected-tests.test.sh +++ b/scripts/affected-tests.test.sh @@ -1017,6 +1017,29 @@ else fail "node .mjs co-located (rc=$RC): $OUT" fi +# --- R9: a wrapped Node suite brings its .test.sh -------------------- +# CI runs a Node suite with a sibling .test.sh only through that wrapper. +# The suite here is reached through a MENTION of the changed file, not through +# its stem, so R2 cannot find the wrapper and the walk stops at the suite. +printf 'export const w = 3;\n' >"$repo/eco/wrapped.mjs" +printf 'import { w } from "./wrapped.mjs";\n' >"$repo/eco/wrapped-cases.test.mjs" +# shellcheck disable=SC2016 # deliberate: the emitted file must expand these, not this shell +printf 'node "$(dirname "$0")/wrapped-cases.test.mjs"\n' >"$repo/eco/wrapped-cases.test.sh" + +run_sel "$repo" eco/wrapped.mjs +if [[ "$RC" -eq 0 ]] && has_line "$OUT" eco/wrapped-cases.test.mjs && has_line "$OUT" eco/wrapped-cases.test.sh; then + ok "R9: a Node suite reached through a mention selects its sibling .test.sh wrapper" +else + fail "R9 wrapper via mention (rc=$RC): $OUT" +fi + +run_sel "$repo" eco/probe.mjs +if [[ "$RC" -eq 0 ]] && [[ "$OUT" != *.test.sh* ]]; then + ok "R9: a Node suite with no sibling .test.sh adds no shell suite" +else + fail "R9 without a wrapper (rc=$RC): $OUT" +fi + run_sel "$repo" eco/ps/Get-Thing.ps1 if [[ "$RC" -eq 0 ]] && has_line "$OUT" eco/pstests/Get-Thing.Tests.ps1; then ok "powershell: a Pester suite in a mirrored tree is found by reference" From ee408ecf9507caa29d4ac203310befeafcb8e2c5 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 11:57:36 -0400 Subject: [PATCH 17/18] fix(scripts): limit R9 to the Node suites the selector recognizes is_suite_path admits .test.js and .test.mjs, not .test.cjs, so the .test.cjs arm of R9 could never fire, and no .test.cjs suite exists. Co-Authored-By: Claude Opus 5.5 (1M context) --- scripts/affected-tests.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/affected-tests.sh b/scripts/affected-tests.sh index 07a4d64e55..88be1e412e 100755 --- a/scripts/affected-tests.sh +++ b/scripts/affected-tests.sh @@ -112,7 +112,7 @@ # a `#` after them starts a comment. Suites whose comments # start with `#` (shell, Python, Pester) can declare. A glob # matching no file fails the run that changes its suite. -# R9 wrapper a selected .test.js, .test.mjs or .test.cjs whose +# R9 wrapper a selected .test.js or .test.mjs whose # directory holds .test.sh selects that wrapper too, # however the Node suite was reached. CI runs such a suite only # through its wrapper (scripts/run-outside-node-suites.sh @@ -1329,7 +1329,7 @@ if [[ ${#SUITES[@]} -gt 0 ]]; then # R9. The key list is expanded once, before the loop adds to it. for s in "${!SUITES[@]}"; do case "$s" in - *.test.js | *.test.mjs | *.test.cjs) add_suite "${s%.test.*}.test.sh" "wraps $s" || true ;; + *.test.js | *.test.mjs) add_suite "${s%.test.*}.test.sh" "wraps $s" || true ;; *) ;; esac done From cb357f3a267ab1fa2711837512d866035dc2ec48 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Sat, 3 Oct 2026 12:05:22 -0400 Subject: [PATCH 18/18] fix(scripts): drop the scripts/lib/sync-*.sh scope main's #6064 emptied #6064 deleted scripts/lib/sync-cluster.sh, the only file the glob matched, and a declared glob that matches no file fails every run that changes its suite. Co-Authored-By: Claude Opus 5.5 (1M context) --- scripts/affected-tests.test.sh | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/scripts/affected-tests.test.sh b/scripts/affected-tests.test.sh index d6e19c530c..ee1f2d6144 100755 --- a/scripts/affected-tests.test.sh +++ b/scripts/affected-tests.test.sh @@ -8,7 +8,7 @@ # against the LIVE repo — the derived shared-lib copy set and the real no-suite # list — because a synthetic fixture cannot show that the derivation still # tracks reality, which is the whole failure mode this tool exists to avoid. -# test-scope: scripts/affected-tests* scripts/sync-*.sh scripts/lib/sync-*.sh .github/workflows/ci.yml +# test-scope: scripts/affected-tests* scripts/sync-*.sh .github/workflows/ci.yml # test-scope: plugins/github/skills/advise/S*.md plugins/planning/skills/interview/S*.md # test-scope: plugins/autonomy/reference/*.md set -uo pipefail @@ -697,10 +697,6 @@ rm -rf "$repo" "$marker" for src in lib/hook-utils.sh lib/parse-concern-value.sh docs/conventions/standards/README.md; do derived="$(cd "$REPO_ROOT" && bash scripts/affected-tests.sh --print-fanout "$src" 2>/dev/null | sort)" manifest="scripts/sync-shared-copies.sh" - case "$src" in - lib/hook-utils.sh) manifest="scripts/sync-hook-utils.sh" ;; - *) ;; - esac expected="$(cd "$REPO_ROOT" && bash "$manifest" --print-manifest | awk -F '\t' -v s="$src" '$1=="src"{on=($2==s)} on && $1=="copy" && $2!=""{print $2}' | while IFS= read -r pat; do if [[ -e "$pat" ]]; then printf '%s\n' "$pat"