fix(check): write the report to stdout so it can be piped - #373
Conversation
`check` emitted its entire report — header, per-secret status lines and summary — through `eprintln!`, leaving stdout empty. `secretspec check | grep DATABASE_URL` therefore matched nothing while the command exited 0, which reads as "no secrets found" rather than "the report went elsewhere". The same subcommand already contradicts itself here: `check --json` (`cli/mod.rs:1507`) and `check --explain` (`:1509`) write to stdout. Only the default human rendering did not, so which stream carried the report depended on a flag. Routing it to stdout also makes the existing colour handling correct. `colored` decides on ANSI escapes by testing stdout (`control.rs:108`) while the report went to stderr, so the two streams could disagree in both directions: escapes leaking into a redirected log, or colour stripped from output being read on a terminal. Written through a locked `&mut dyn io::Write` rather than swapping the macros. Putting the report on stdout puts it on a stream a reader can close early, and `println!` panics on EPIPE — stdout is a `LineWriter`, so each report line is its own write and no large output is needed to trigger it. `check | head -1` panicked with exit 101 under a naive swap, where it had exited 0 before only because stdout received nothing. An injected sink is the pattern `write_export` already documents for this reason, so a closed pipe now behaves as `export | head` does: a clean `IO error: Broken pipe`. `ensure_secrets` is untouched — its output is prompts and diagnostics, not report — and the lock is released before it so prompting is unaffected. Three regression tests, each confirmed to fail without the change: the first two against the pre-fix tree, the third against a bare macro swap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Filed the check-stdout report as cachix#372 and opened cachix#373 against it. The PR is built on upstream/main rather than cherry-picked from sudo-main, so it carries only the secrets.rs and check_report_stream.rs hunks plus a hand-written Changed entry under upstream's own Unreleased -- the fork's CHANGELOG diff is 458 fork-local insertions and could not be lifted. All three regression tests were verified against a pure dfa4b10 base, not just against our merged tree. Posting also turned up a hole in a draft marked READY TO POST: the ELI5 section said "if you try to do the obvious thing:" and then jumped straight to "...you get nothing", with the example command block missing entirely. Restored before sending. The larger find is upstream PR cachix#362, which the ledger did not track at all -- it was visible only as a pointer in a comment on cachix#64. It introduces SecretSpec IPC v1 for 0.20+, including `secretspec broker --stdio`, and is close enough in vocabulary to this fork that the distinction has to be written down: upstream's broker is an IPC endpoint inside the caller's own trust domain, not a privilege boundary. Its initialize accepts a caller-supplied manifest, provider and profile, which is exactly what this fork's control plane exists to remove, and its audit is fail-open where ours is fail-closed and hash-chained. The practical consequence is favourable: `secretspec.provider/1` is the exec:// mechanism cachix#345 asked for, and a privileged endpoint can be registered as data without patching upstream internals. Recorded in docs/design/upstream-ipc-v1-and-the-fork.md, along with the finding that cachix#362 does NOT retire the codegen-schema shape debt -- no manifest-shape reflection anywhere in the client protocol, so cachix#371 remains the only route. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tier 2 handoff e439, parent 7c73 (deterministic). Covers releasing 0.19.1-sudo.15, posting upstream issue cachix#372 and PR cachix#373, tracking upstream's IPC v1 PR cachix#362 -- and the vault truncation incident this session caused. The incident is the reason this document leads with it rather than the release: plain `install` without --adopt-existing truncated /var/db/sudo-secretspec/.env to 0 bytes, destroying every stored value. Both shipped docs specify the flag (SKILL.md:197, AI-GUIDANCE.md:87) and I handed over the command without it. The audit ledger brackets the loss to 26 seconds after the install, and Arq's Aug 17 02:10 SYSTEM record predates it, so recovery is available. Three failed approaches are recorded in full because each was expensive: the merge hypothesis presented to the operator before it was cheaply falsifiable, reading a green template-check as reassurance when it was evidence of the overwrite, and concluding the loss predated the session because fs::copy on macOS preserves source mtime. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
domenkozar
left a comment
There was a problem hiding this comment.
Reviewed this against a local checkout and ran the tests. The stream split is the right call, the colour argument holds up, and the description saved me most of the digging. Four things, one of which answers your second open question.
1. Take the sink as a parameter rather than reaching for stdout inside the library
Yes to the impl Write shape you offered. export already established it: cli/mod.rs:1466 acquires stdout in the CLI and passes &mut out into Secrets::export. check should look the same:
pub fn check(&self, no_prompt: bool, out: &mut dyn io::Write) -> Result<ValidatedSecrets>with cli/mod.rs:1519 handing it the stream. One caller in the tree and nothing in secretspec-derive touches check, so the churn is small, and a published library then stays out of its consumer's data channel.
One detail when you move it: pass &mut std::io::stdout() rather than a held .lock(). Holding the lock across self.validate() is the part of the current shape I would not keep. validate() fans out onto std::thread::scope workers whenever a check spans more than one provider group (secrets.rs:5412), so a global lock is held across arbitrary provider I/O running on other threads. Nothing in the tree writes to stdout from those workers today (no provider prints, none use Stdio::inherit), so this is latent rather than a reachable hang, but the lock buys no atomicity when nothing else writes to the stream, and an unlocked handle re-acquires per line exactly like println! does. export has the same pattern already; pre-existing, not yours to fix here.
2. The pipeline test earns its place, and I confirmed it is not vacuous
The report is small enough to fit in a 64KB pipe buffer, so I wanted to be sure head -1 really does close the pipe before the writer is done rather than the whole report landing in the buffer first. Measured against a build of this branch:
- 100 runs of that exact pipeline: 99 produced
Broken pipe, so the post-header writes really do hit a closed pipe. - Substituting a panicking sink for yours (a stand-in for the naive
println!swap): the test failed 10 out of 10 runs.
The timing works because the header is flushed before validate() runs, and validate() then does a provider read plus audit writes, which is the whole window head needs to print and exit. Buffer capacity never comes into it. Roughly 1 run in 100 will not exercise the regression path, which I am happy with; making it deterministic would mean a manifest large enough to fill the pipe buffer, and that is not worth the test's runtime.
3. A closed pipe still prints an error on the pipeline this change exists to enable
check | head now exits 1 with Failed to check secrets: IO error: Broken pipe on stderr. Nothing in the repo touches SIGPIPE, so restoring the default disposition in the CLI entry point would quiet that for check, check --json, and export in one place, and would cover the pre-existing --json | head panic you flagged without visiting each call site. It needs a unix-only libc dependency (already in the lock file transitively), so a follow-up PR is the right size for it. Not blocking this one.
4. Two small ones
tests/check_report_stream.rs:181:envis interpolated raw whilebinandmanifestgo throughshell_quote. The surrounding single quotes cover it in practice, but the helper is right there and the comment above it makes the case.- Changelog: "goes through a locked sink" is implementation detail in a user-facing entry, and it stops being accurate after point 1. The observable half, a closed pipe reporting an error instead of panicking, already carries the meaning.
On your first open question: keeping the constraint violations on stdout with the rest of the report is right, they are report lines rather than diagnostics. On the third: leave import's summary alone, it is a separate decision and point 3 above covers the --json half.
🤖 Review generated with Claude Code. Noted here because the GitHub API has no field for it: cli/cli#13904
Per @domenkozar's review: - check() now takes out: &mut dyn io::Write instead of locking stdout internally, mirroring export's shape (cli/mod.rs:1466). The CLI passes an unlocked &mut std::io::stdout(); validate() fans out onto std::thread::scope workers, so holding a lock across that call bought no atomicity and would have serialized against any output those workers might someday produce. - tests/check_report_stream.rs:181 now shell_quotes the interpolated env path like its neighbors, instead of leaving it raw. - Changelog wording no longer says "locked sink", which stopped being accurate once the lock moved to the caller's discretion. The SIGPIPE follow-up from point 3 is intentionally left for a separate PR, as suggested.
|
Thanks for the thorough review — all addressed in 1b1783c:
|
|
The SIGPIPE follow-up from point 3 is now open as #377. It turned out to be worth doing on its own: |
Rust's runtime ignores SIGPIPE, so a closed output pipe surfaces as an
EPIPE write error rather than terminating the process. `secretspec` then
fails where every other Unix tool exits quietly:
$ secretspec export --format dotenv | head
Error: x Failed to export secrets
|-> IO error: Broken pipe (os error 32)
`-> Broken pipe (os error 32) # exit 1
$ secretspec check --json | head
thread 'main' panicked at library/std/src/io/stdio.rs:1166:9:
failed printing to stdout: Broken pipe (os error 32) # exit 101
Resetting the disposition to SIG_DFL in the binary entry point fixes
every stdout-writing command at once, rather than teaching each call
site to special-case EPIPE. Both commands above now terminate on signal
13 with empty stderr; unpiped output is unchanged.
This is the follow-up promised in #373, where the same broken-pipe
behavior came up on the `check` path.
libc is a new direct dependency, declared only under cfg(unix). It was
already in the lock file transitively, so nothing new is vendored.
|
This one is still needed? |
Fixes #372, where the reasoning and the transcripts are laid out in full.
checkemitted its entire report througheprintln!, so stdout was empty andsecretspec check | grep DATABASE_URLmatched nothing while the command exited0 — which reads as "no secrets found" rather than "the report went elsewhere".
Three things beyond the piping annoyance, in the order I'd weigh them:
checkalready contradicts itself.check --json(cli/mod.rs:1507) andcheck --explain(:1509) write to stdout. Only the default human renderingdid not, so which stream carried the report depended on a flag.
The colour handling is currently wrong in both directions.
coloreddecides on ANSI escapes by testing stdout (
control.rs:108) while thereport went to stderr. So
check 2>logleaked raw escape bytes into the logfile, and
check >filestripped colour from output being read on a terminal.Routing the report to stdout makes the existing detection correct rather than
needing new colour logic.
A naive macro swap introduces a panic. Putting the report on stdout puts
it on a stream a reader can close early, and
println!panics on EPIPE.stdout is a
LineWriter, so every report line is its own write syscall and nolarge output is needed:
That pipeline exits 0 today only because stdout receives nothing. So this
writes the report through a locked
&mut dyn io::Writeinstead — the patternwrite_export's doc comment already documents for exactly this reason ("turnsa broken pipe into a returned error instead of a panic"). A closed pipe now
behaves as
export | headalready does: a cleanIO error: Broken pipe.ensure_secretsis deliberately untouched — its output is prompts anddiagnostics, not report — and the lock is released before it, so prompting is
unaffected. No change to exit codes, to what is printed, or to dependencies.
Tests
secretspec/tests/check_report_stream.rs, three tests, each confirmed to failwithout the change:
a_passing_check_writes_its_whole_report_to_stdouta_failing_check_reports_on_stdout_but_errors_on_stderra_reader_that_closes_the_pipe_early_does_not_panic—#[cfg(unix)], drivesa real
sh/headpipelineThe first two fail against the pre-fix tree; the third fails against a bare
eprintln!→println!swap and passes here.Compatibility
Filed under
Changedrather thanFixed: anyone capturing via2>&1keepsworking, but a script capturing stderr specifically needs to capture stdout
instead. I won't claim that's rare — it caught me, and I had to update three
fixtures in my own downstream post-install suite.
Open questions, happy to go either way
display_validation_errorsare arguablydiagnostics rather than report. I kept them with the rest of the report on
stdout, on the grounds that splitting one human-readable report across two
streams is the same class of bug — say the word and I'll move them.
Secrets::check()is public, so this makes a library write to its consumer'sstdout. It already printed unconditionally, just to the other stream, so no
consumer is spared it today. If you'd rather a library not print at all, the
cleaner shape is
checktaking animpl Writedefaulting toio::stdout(),or moving rendering into the CLI layer — happy to send either instead.
check --json | headpanics the same way already (cli/mod.rs:1456), andimport's summary is on stderr too (secrets.rs:4313/:4320/:4328). Gladto include either if you want them.
🤖 Generated with Claude Code