Problem
A repair agent is attached automatically unless onError is continue. For an
implementation step that is the right default: a red typecheck is work to be
done, and an agent that fixes it is doing its job.
For a step whose purpose is to report a red, the same default inverts the
meaning. The step exists to answer "does this gate hold?" and the engine
responds by editing the tree until it says yes.
There is no way to declare "red is a result here, not a failure."
Why this is sharp for verification flows
The failure is silent and it looks like success. A repair agent can turn a gate
green by:
- widening or deleting the assertion that failed,
- relaxing a validator until the input passes,
- adding the missing artifact rather than the behaviour the artifact attests to,
- marking a failing test
#[ignore] / .skip.
All four produce a green run and a tree that is worse than before, and the run
log reads as a clean pass. In a campaign whose entire output is a claim about
correctness, this converts the deliverable into its opposite.
Observed in practice on a relay verification campaign: gates that were "green"
sat on a tree with two double-delivery defects and a mutation probe still live
in product code. onError: continue everywhere is the current workaround, but
it is opt-out, easy to forget on one step out of sixty, and it also disables
repair for the steps that legitimately want it.
Proposal
A step class where repair is structurally unavailable, e.g.
f.check('cargo clippy --all-targets -- -D warnings') // red is data
f.run('npm run build') // red is work
or an explicit property if a new verb is too much:
f.run('cargo test …', { repair: false })
Semantics:
- No repair agent may be attached, and the engine refuses at validation time if
one is configured.
- A red result is recorded as a finding, not a step failure, and the run
continues (so downstream reporting still happens).
- The tree must be unchanged across the step — the engine can assert this
cheaply, and a check step that modifies tracked files is itself a finding.
That last property is the one that matters most: it makes "this gate did not
edit the thing it was measuring" a mechanical guarantee rather than something a
reviewer has to notice.
Relationship to #531 / #532
#531 makes evidence unforgeable. #532 makes proving a test can fail cheap. This
one stops the engine from repairing away the signal those two exist to
capture. All three are the same underlying gap: the engine has no concept of a
step whose output is a measurement.
Problem
A repair agent is attached automatically unless
onErroriscontinue. For animplementation step that is the right default: a red typecheck is work to be
done, and an agent that fixes it is doing its job.
For a step whose purpose is to report a red, the same default inverts the
meaning. The step exists to answer "does this gate hold?" and the engine
responds by editing the tree until it says yes.
There is no way to declare "red is a result here, not a failure."
Why this is sharp for verification flows
The failure is silent and it looks like success. A repair agent can turn a gate
green by:
#[ignore]/.skip.All four produce a green run and a tree that is worse than before, and the run
log reads as a clean pass. In a campaign whose entire output is a claim about
correctness, this converts the deliverable into its opposite.
Observed in practice on a relay verification campaign: gates that were "green"
sat on a tree with two double-delivery defects and a mutation probe still live
in product code.
onError: continueeverywhere is the current workaround, butit is opt-out, easy to forget on one step out of sixty, and it also disables
repair for the steps that legitimately want it.
Proposal
A step class where repair is structurally unavailable, e.g.
or an explicit property if a new verb is too much:
Semantics:
one is configured.
continues (so downstream reporting still happens).
cheaply, and a check step that modifies tracked files is itself a finding.
That last property is the one that matters most: it makes "this gate did not
edit the thing it was measuring" a mechanical guarantee rather than something a
reviewer has to notice.
Relationship to #531 / #532
#531 makes evidence unforgeable. #532 makes proving a test can fail cheap. This
one stops the engine from repairing away the signal those two exist to
capture. All three are the same underlying gap: the engine has no concept of a
step whose output is a measurement.