test(specs): check the network-denial step count against the workflow - #254
Merged
Merged
Conversation
ADR-0040 action item 7 asked whether the clean-clone-builds network-denial count should be derived rather than restated in four places. I am settling it as checked, not derived. The four statements are hand-written requirement and evidence prose in specs/010-catalog-backstage, and ADR-0040 keeps specs/ out of machine rewriting. What actually went wrong, twice, was a step added to the job without a re-count. scripts/network-denial-count.test.ts parses clean-clone-builds. It requires bun test to be the only post-install step not run through run-network-denied.ts, and it fails when observed-failing-register.md §4.6, the clean-clone-offline README, FR-050, or T093 states a different count, or stops stating one it can read. The last rule keeps a reworded sentence from turning the check into a check of nothing. Observed failing (ADR-0016) three ways: a duplicated wrapped step in the job (all four documents fail), T093 edited to 15 (that file fails), and FR-050's sentence reworded away (that file fails). ADR-0040 item 7 is ticked with the outcome, and §4.6 records why the count is now checked. Signed-off-by: Mark Beacom <m@beacom.dev>
mbeacom
added
the
gate-change-acknowledged
Decisions governing this change
|
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The checker can falsely classify unwrapped steps and omits an existing live count restatement.
Review effort: Balanced
Findings: 2
Open (2)
What changed in this PR
Adds a guard that synchronizes documented network-denial counts with the CI workflow.
Changes:
- Parses
clean-clone-buildsand validates four documentation files. - Records the checked-not-derived decision in ADR-0040.
- Documents the new guard in the evidence register.
Decision-context MCP verification was unavailable.
| File | Description |
|---|---|
scripts/network-denial-count.test.ts |
Adds workflow/count checks. |
docs/adr/0040-keep-derived-surfaces-in-lockstep-with-three-mechanisms-matched-to-three-classes.md |
Closes action item 7. |
specs/010-catalog-backstage/evidence/observed-failing-register.md |
Records the checking mechanism. |
…e command Two review findings on #254. The check counted a clean-clone-builds step as network-denied whenever its run mentioned run-network-denied.ts anywhere. So `bun run build && echo scripts/run-network-denied.ts` counted, and so did `wrapper -- a && curl ...`, where the shell runs curl outside the wrapper. A step now counts only when its run starts with the wrapper and has no shell operator (&&, ||, ;, |, &, a newline, a backtick, or $(...)) outside quotes. The existing `sh -c '... && ...'` steps still count, because their operators run inside the wrapped shell. A unit test covers both sides, and it failed against the old substring check. The comment on the bun test step in ci.yml restated "16 of the 17", a fifth copy that the check didn't read. I removed the number rather than parsing it: the comment now points to §4.6 and to this test. The test also fails if an "N of the M" count or an ordinal step count reappears in ci.yml, and it failed against the old comment. Signed-off-by: Mark Beacom <m@beacom.dev>
github-actions
Bot
removed
the
gate-change-acknowledged
mbeacom
added
the
gate-change-acknowledged
mbeacom
enabled auto-merge (squash)
October 1, 2026 02:34
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
This closes ADR-0040's last open decision, action item 7: should the
clean-clone-buildsnetwork-denial step count be derived rather than restated in four places?I'm settling it as checked, not derived. The count appears in four hand-written places in
specs/010-catalog-backstage/:observed-failing-register.md§4.6;clean-clone-offlineREADME;spec.md;tasks.md.ADR-0040 keeps
specs/out of machine rewriting, so generating these sentences would contradict the record the item belongs to. And the failure was never the prose: twice, a step was added to the job and nobody re-counted.scripts/network-denial-count.test.tsparses the job fromci.ymland:bun testto be the only post-install step not run throughscripts/run-network-denied.ts;ADR-0040 item 7 is ticked with the outcome, and §4.6 now says the count is checked and why.
Checklist
packages/ci/dist: n/aclean-clone-buildsfails all four documents;scripts/suite (1,113 tests),adr lint,check:stale-refs, and typecheck pass.Notes for reviewers
gate-integrityflags this because it adds a file underscripts/.STATEMENTSin the test. The test can't discover new restatements on its own, the same hand-maintained-scope trade-off ADR-0040 accepts for the prose guard.reviewBydate.