Skip to content

Test suite: one home for every check, with coverage measured against the corpus it adopts - #505

Closed
m2ux wants to merge 3 commits into
mainfrom
refactor/lean-test-suite
Closed

Test suite: one home for every check, with coverage measured against the corpus it adopts#505
m2ux wants to merge 3 commits into
mainfrom
refactor/lean-test-suite

Conversation

@m2ux

@m2ux m2ux commented Aug 25, 2026

Copy link
Copy Markdown
Owner

Summary

This branch carries 21 commits of work that had accumulated on top of main without a pull request. It is being opened now because the next two pieces of work — the corpus defects in #491 and the test-location boundary in #497 — both touch the files this branch rewrites, and stacking on an unmerged branch is worse than landing it first.

The work falls into four groups that share a branch rather than a subject.

What is in it

The test suite gets one home per check. Around 2,300 lines come out of tests/ and 2,150 go back. Setups that were repeated across files are named once; checks that had drifted into two places are reduced to one. Several e2e files retire entirely — the all-paths walk, the definition lint, the robot execution harness and the workflow e2e file — because what they measured is now measured in one place instead of four. The number of test files that read the real corpus drops from 29 to 20.

Coverage moves to main and to request. A composite action now handles corpus checkout for every job that needs it, so the walk, the guards and the suite all resolve the corpus the same way. The coverage walk runs on a push to main and on explicit request, and a check verifies that the branch walked the same corpus its merge adopts — the pointer and the measurement can no longer come from different parents unnoticed.

Two guards join the canon set. One reports a set action that builds its value out of the variable it writes. The other measures the branch-as-step class in both directions. The first is registered and runs in the sweep; the second is not, and that is stated below rather than fixed here.

The server accounts for what a run cost. Usage is recorded per dispatch, a child run reports its cost or says it cannot be known, an unpublished progress mark becomes answerable after the fact, and a gate can name a decision its activity did not anticipate. The batch reading moves to where the worker reads it.

Known and not fixed here

The branch-as-step guard has a script and a test but no entry in the guard registry, so the sweep never runs it. Registering it reports 11 violations across 8 files in 5 workflows — all of them the same class, where an indented sub-bullet qualifying an instruction is read as a peer step once the leading whitespace is stripped. Those are corpus files, on the other branch, so the fix cannot land here.

This is finding 3 of #491. The corpus fixes land first, on the corpus branch; the registry entry follows on the code branch once the pointer moves. Registering it in this pull request would make the pull request red for a reason no change here can resolve.

Scope

Landing the work as it stands. No new behaviour is added by opening this.

Acceptance criteria

  • The suite and the full guard sweep pass on this branch.
  • The coverage walk and the guards resolve the corpus through the same checkout mechanism.
  • A merge cannot take a coverage claim from one parent and its corpus from the other without the check saying so.

Non-goals

This does not register the branch-as-step guard, and does not touch the corpus. Both belong to #491.

It does not move any test to the corpus branch or retire the corpus-commit stamps. That is #497, and it stacks on this.

m2ux and others added 3 commits August 25, 2026 05:12
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mike Clay <mike.clay@shielded.io>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mike Clay <mike.clay@shielded.io>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Mike Clay <mike.clay@shielded.io>
@m2ux

m2ux commented Aug 25, 2026

Copy link
Copy Markdown
Owner Author

Closing this: it was opened against a stale reading of main, and its description is wrong.

The measurement that produced it came from a local main that was 64 commits behind the remote. Against the real origin/main, this branch is 42 commits behind and 3 ahead, and all three are .engineering submodule pointer bumps. Everything the description credited to it — the test-suite consolidation, the two canon guards, the coverage workflow, the per-dispatch usage accounting — is already on main, merged through #495, #501 and #502.

What remains is a submodule pointer that is newer than the one on main but older than engineering HEAD, so landing it would move the pointer backwards relative to where that branch now is.

A pointer bump, if wanted, is better taken fresh against engineering HEAD than salvaged from here.

Nothing in the description should be read as a statement about this branch. The one claim in it that survives is about the corpus, and it is correct: the branch-as-step guard has a script and a test but no registry entry on main, and registering it reports 11 violations across 8 files in 5 workflows. That is finding 3 of #491 and is being fixed corpus-first.

@m2ux m2ux closed this Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant