test(guards): fail when a source scan guards nothing - #3878
test(guards): fail when a source scan guards nothing#3878worktrunk-bot wants to merge 3 commits into
Conversation
The walks swallow read errors, so a missing scan root passed as a clean result; and the path allowlists are matched by string, so a rename leaves an exemption armed at the abandoned path for whatever file lands there next. Both walks now return the number of .rs files reached and each test asserts it is nonzero, and a new test pins that every allowlist entry still names a real file.
worktrunk-bot
left a comment
There was a problem hiding this comment.
Self-review. The mechanism is right, and allowlisted_paths_still_exist is the half with teeth — a path allowlist that stops matching anything is an armed exemption, not dead clutter. Two gaps, both of which I'm applying in a follow-up commit rather than leaving for a maintainer, since this PR is bot-authored.
version_build.rs has the identical hole and this PR doesn't cover it. vergen_env_vars_are_read_optionally walks src/ with the same let Ok(entries) = fs::read_dir(dir) else { return }; and then asserts violations.is_empty(), so a missing or relocated scan root passes it having read nothing — exactly the shape this PR pins. It matters more there than here: that guard is what stands between a bare env!("VERGEN_…") and another #3123, where cargo install worktrunk failed to compile off the crates.io archive. The other two source-scanning guards already carry floors — packaged_assets.rs's !assets.is_empty() and snapshot_formatting_guard.rs's seen > 500 — so as written this PR leaves version_build.rs as the only one without.
scanned > 0 is weaker than the precedent it follows. for_each_snapshot in snapshot_formatting_guard.rs asserts seen > 500 over its ~1200-file corpus for this same reason, under the same "would pass vacuously" comment. src/ holds 205 .rs files and the new docstring already says a real run "reaches hundreds of files", so a substantial floor is there for the taking — and it catches the partial walk the docstring itself names ("relocated src/"), which > 0 cannot: a workspace split that leaves a stub src/ behind clears zero while the guard inspects almost nothing.
Left alone deliberately, for a maintainer's call: four guard tests now hand-roll a near-identical recursive walk (output_system_guard, version_build, packaged_assets, and snapshot_formatting_guard's .snap variant), and this PR adds counting to two of them. A single counted walker in tests/common/ would make the floor structural instead of something each new guard has to remember — but it touches four files and is a separate concern from this one.
Verification of the follow-up commit
MIN_SCANNED_FILES = 100 in both files; src/ currently walks to 205, so the headroom is ~2x.
Injected fault — pointing the vergen scan at src-does-not-exist, which passed silently before:
thread '...::version_build::vergen_env_vars_are_read_optionally' panicked at tests/integration_tests/version_build.rs:46:5:
scanned only 0 .rs files under /home/runner/work/worktrunk/worktrunk/src-does-not-exist — the guard passed without inspecting the crate.
The walk swallows read errors, so this is a missing or unreadable scan root, not a clean result.
Unmodified tree:
test integration_tests::output_system_guard::allowlisted_paths_still_exist ... ok
test integration_tests::output_system_guard::check_no_unexpected_stdout_writes ... ok
test integration_tests::output_system_guard::check_stderr_macros_come_from_styling ... ok
test integration_tests::version_build::vergen_env_vars_are_read_optionally ... ok
Also clean: cargo fmt --check, cargo clippy --test integration --all-features.
The nightly-sweep change gave output_system_guard's two walks a
scanned-file floor, but left vergen_env_vars_are_read_optionally with the
identical hole: it walks src/ with the same read-error-swallowing recursion
and then asserts violations.is_empty(), so a missing or relocated scan root
passes it having read nothing. That guard is what stands between a bare
env!("VERGEN_…") and another #3123, and it was the last source-scanning
guard without a floor (packaged_assets and snapshot_formatting_guard both
already carry one).
Raise the floor itself from > 0 to MIN_SCANNED_FILES (100, against src/'s
205 .rs files), matching snapshot_formatting_guard's seen > 500. Zero only
catches a scan root that vanished outright; the docstring's other named
case — a relocated src/ — is partial, and a workspace split leaving a stub
behind clears > 0 while the guard inspects almost nothing.
|
Two changes here, and they're worth separating: Keep the allowlist test. Both lists are matched by path string, so a rename leaves a dead entry armed at the old path — and The floor is a different matter. The root cause it works around is that the walks discard the error: let entries = match fs::read_dir(dir) {
Ok(e) => e,
Err(_) => return,
};Rather than measuring how much the walk reached, let that error surface: let entries = fs::read_dir(dir)
.unwrap_or_else(|e| panic!("scan root {} unreadable: {e}", dir.display()));One line per walk instead of a returned count threaded through three recursive functions, no
|
Replace the scanned-file floor with a panic at the point the walk discards the error, and extend the treatment to the two guards that carried a count-based floor of their own.
worktrunk-bot
left a comment
There was a problem hiding this comment.
Self-review of the two commits since the last round. The requested direction is applied: the floors are gone from all four guards, MIN_SCANNED_FILES with them, and each of the five walks surfaces its read_dir error instead. All 20 allowlist entries resolve today, src/remove_dir.rs included, so allowlisted_paths_still_exist is green on the current tree.
One residual, raised as a decision rather than pushed as a commit. The walks now surface the error at the directory level, but the same swallow sits one level down, inside the very functions this diff touches:
check_fileinoutput_system_guard.rs—match fs::read_to_string(path) { … Err(_) => return }, in both the stdout scan and the stderr scancheck_fileinversion_build.rs—let Ok(contents) = fs::read_to_string(path) else { return; };scan_fileinpackaged_assets.rs— the samelet Ok(…) elseform
A .rs file the walk reaches but can't read is skipped in silence, so the guard reports clean for content it never inspected. That is the shape the PR names, scoped to one file rather than the whole tree. visit_snap_files in snapshot_formatting_guard.rs already treats it as fatal (fs::read_to_string(&path).unwrap()), so as it stands the four guards now disagree about the file-level case while agreeing about the directory-level one. entries.flatten() is the same class again — it drops a mid-iteration DirEntry error in all five walks.
I stopped short of applying it because the last round's point was that this change should stay one line per walk, and pushing the treatment downward is a scope call worth making explicitly rather than assuming. If you want it, it's four one-line edits and I'll push them in a follow-up commit; if the file-level case is too remote to be worth the lines, the inconsistency with snapshot_formatting_guard.rs is the thing to settle, in whichever direction.
|
Applied in Net effect on the two commits you were reading: Verified by repointing each of the five scan roots at a nonexistent directory — all five now fail, all five passed silently before: Checks are green on that head, |
Nightly sweep finding. The guard tests that scan
src/have two ways to pass while guarding nothing: their directory walks discard theread_direrror, andoutput_system_guard's two path allowlists are matched by string, so a rename leaves the exemption armed at the abandoned path. This lets the read error surface at every source-scanning walk, and adds a test that every allowlist entry still names a real file.The stale-allowlist half
STDOUT_ALLOWED_PATHSandSTD_STDERR_ALLOWED_PATHSexempt a path, and an entry exempts the whole file. When a file is renamed, nothing fails — the entry just stops matching anything. It stays armed, so a new file later occupying that path inherits a stdout permission nobody granted it, and the guard stays green while it writes.src/help.rsis the shape of the risk: a plausible name for a future refactor to reuse, currently exempt because--help-pageand--versionare genuinely the command's answer.git log --diff-filter=R -- src/shows renames are routine here, so this is a rot-prevention pin rather than a fix for present drift — all 20 entries resolve today. It converts the rename into a failing test at the moment it happens, while the reviewer still knows whether the exemption should move with the file or go.The vacuous-pass half
Every source-scanning guard walks with
Err(_) => return, so a missing or unreadable scan root produces zero violations and the test reports success having read nothing. Rather than measure how much of the tree the walk reached, the walk now lets the error out:That is one line per walk, and the failure names the actual io error instead of a file count. It covers all five walks, including the two that carried a count-based floor of their own and would otherwise have set the convention:
snapshot_formatting_guard.rs'sseen > 500andpackaged_assets.rs's!assets.is_empty()are both removed in favour of the same treatment.version_build.rsmatters most of the five: that guard is what stands between a bareenv!("VERGEN_…")and another #3123, wherecargo install worktrunkfailed to compile off the crates.io archive.Verification
Each of the five walks was confirmed to panic on an injected fault (its scan root repointed at a nonexistent directory) and to pass on the real tree.
Injected-fault runs
Each panics naming the directory and the error:
All five passed silently before this change. Unmodified tree:
Also clean:
cargo fmt --check,cargo clippy --test integration --all-features.