Skip to content

No step class where a red result is a finding: repair agents can edit a verification gate into passing #533

Description

@khaliqgant

Problem

A repair agent is attached automatically unless onError is continue. For an
implementation step that is the right default: a red typecheck is work to be
done, and an agent that fixes it is doing its job.

For a step whose purpose is to report a red, the same default inverts the
meaning. The step exists to answer "does this gate hold?" and the engine
responds by editing the tree until it says yes.

There is no way to declare "red is a result here, not a failure."

Why this is sharp for verification flows

The failure is silent and it looks like success. A repair agent can turn a gate
green by:

  • widening or deleting the assertion that failed,
  • relaxing a validator until the input passes,
  • adding the missing artifact rather than the behaviour the artifact attests to,
  • marking a failing test #[ignore] / .skip.

All four produce a green run and a tree that is worse than before, and the run
log reads as a clean pass. In a campaign whose entire output is a claim about
correctness, this converts the deliverable into its opposite.

Observed in practice on a relay verification campaign: gates that were "green"
sat on a tree with two double-delivery defects and a mutation probe still live
in product code. onError: continue everywhere is the current workaround, but
it is opt-out, easy to forget on one step out of sixty, and it also disables
repair for the steps that legitimately want it.

Proposal

A step class where repair is structurally unavailable, e.g.

f.check('cargo clippy --all-targets -- -D warnings')     // red is data
f.run('npm run build')                                   // red is work

or an explicit property if a new verb is too much:

f.run('cargo test …', { repair: false })

Semantics:

  • No repair agent may be attached, and the engine refuses at validation time if
    one is configured.
  • A red result is recorded as a finding, not a step failure, and the run
    continues (so downstream reporting still happens).
  • The tree must be unchanged across the step — the engine can assert this
    cheaply, and a check step that modifies tracked files is itself a finding.

That last property is the one that matters most: it makes "this gate did not
edit the thing it was measuring" a mechanical guarantee rather than something a
reviewer has to notice.

Relationship to #531 / #532

#531 makes evidence unforgeable. #532 makes proving a test can fail cheap. This
one stops the engine from repairing away the signal those two exist to
capture. All three are the same underlying gap: the engine has no concept of a
step whose output is a measurement.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    garden-readyScoped and ready for an agent to pick up

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions