--reuse-from silently reuses steps whose real output is on disk, resurrecting stale evidence; no way to declare external effects

[khaliqgant]

Summary

flows run --reuse-from <run-id> reuses a step when its step_spec_hash and resolved input match. That is correct for a step whose output is its journaled value. It is wrong for a step whose real output is a side effect on disk, and there is no way for such a step to say so.

The result is silent: the step is skipped, the file it would have rewritten keeps its old contents, and a later gate reads a verdict describing a tree that no longer exists.

Evidence

The flow uses the repair-before-failure pattern that the relay-80-100-workflow skill prescribes. A gate step wraps its command in a recorder that journals {command, exitCode, verdict, tail} to a file and always exits 0, so a red command becomes work for a repair agent rather than a dead run.

That design makes every one of those steps reusable — the step succeeded even though the command it wrapped failed.

Observed across two runs of the same flow:

• Run f: ts-typecheck recorded exit=2 — the repo was missing a workspace symlink (node_modules/@agent-relay/cli-surface).
• I repaired the link. The same command now exits 0 in 13s.
• Run g, launched with --reuse-from <run f>: ts-typecheck was reused. No command ran. The evidence file on disk still said exit=2, describing a tree that had been fixed 20 minutes earlier.

Nothing reported this. The step showed as done.

Why this is not just "the author's problem"

The recorder is not an exotic pattern — it is the only way to express failOnError: false in v2 (see #509), so any flow doing repair-before-failure has this shape, and every one of its gate steps is silently reusable.

More generally: reuse assumes a step's observable output is in the journal. A deterministic step running npm run build, git commit, or a code generator has its real output on disk. Reuse cannot see that, and today it cannot be told.

What to change

Let a step declare that its effects are external and must not be reused:

- id: run-tests
  type: deterministic
  command: 'node gates.mjs record --name unit-tests -- npm test'
  reuse: never        # or: effects: external

Options, not exclusive:

• reuse: never on a step — the minimal, explicit fix.
• Default deterministic steps to no-reuse unless they opt in. Reuse is most valuable for expensive agent and llm steps; a shell command is usually cheap to redo and much more likely to have external effects. This inverts the risk in the safer direction.
• Report what was reused. reuseSummary already computes reusedSteps / executedSteps (packages/sdk/src/cli/reuse.ts). Printing the reused step ids at run start would have made this visible in seconds instead of two runs later.

The last one is worth doing regardless of the others.

Acceptance

• A step can declare that it must not be reused, and --reuse-from honours it.
• A run started with --reuse-from prints which steps were reused.
• A test covers a reused step whose side effect was subsequently undone.

Related

• No advisory deterministic step: v2 gates every deterministic step on exit code, so repair-before-failure flows all re-invent '|| true' #509 — no failOnError equivalent, which is why the recorder pattern exists.
• The worker-parked message names --local-agent only for 'flows run' on a spec path, and 'flows resume --local-agent' silently ignores the flag #504 — flows resume is unusable on a spec run, which is why --reuse-from is the only recovery path and therefore gets used heavily.

Environment

CLI 2.0.22, macOS arm64. Reuse eligibility read from kernel/relayflowd-core/src/memoization.rs (step_spec_hash, resolved_input, candidates).

[khaliqgant]

Raising my own severity assessment. I originally reported this as a nuisance worth surfacing rather than enforcing, on the grounds that a final re-record step heals it before acceptance. That was wrong, and three independent discoveries in one campaign say so.

In a single 67-step run, the stale evidence left behind by --reuse-from was found:

1. By an adversarial reviewer, as a disqualifying blocker. Verdict: "The sealed tree has never been compiled or tested. Every Rust and TypeScript evidence artifact in the set predates the last edits to the files it claims to cover, and the suite is in fact red, including a test the repo labels MUST-FIRE (P1, blocker)."
2. By a second reviewer, from the other vendor, independently — filed as "the implementation seal is stale relative to the reviewed tree."
3. By a fix agent, unprompted, while verifying its own work — "Found while re-running: two -final gates were green on a tree they no longer described."

The pattern that makes it dangerous: a gate step that records evidence and always exits 0 (the only way to express failOnError: false, see #509) is always reuse-eligible, because the step succeeded even when the command did not. So the steps most likely to carry stale results are exactly the ones whose results decide whether the work is acceptable.

Nothing reports it. reuseSummary already computes reusedSteps/executedSteps in packages/sdk/src/cli/reuse.ts — printing the reused step ids at run start would have surfaced all three of the above immediately, and is by far the cheapest item on the original list.