Skip to content

Account for checked auxiliary inputs in PBT experiments - #611

Open
CaelmBleidd wants to merge 2 commits into
caelmbleidd/pbt-large-input-exportfrom
caelmbleidd/pbt-auxiliary-measurement-review
Open

CaelmBleidd wants to merge 2 commits into
caelmbleidd/pbt-large-input-exportfrom
caelmbleidd/pbt-auxiliary-measurement-review

Conversation

@CaelmBleidd

Copy link
Copy Markdown
Member

When relation guidance finds an input, the experiment driver previously could omit it from class coverage or accept an incomplete confirmation. This change reuses the existing original/counter replay path for the original confirmed auxiliary input and requires the configured property, sources, selected relation, exact entry/local binding, native refutation event and admitted coverage identity to agree. Reduced inputs never replace the original witness.

Original named-check faults remain HYPOTHESIS candidates and are counted only when their canonical confirmation matches a timely checked auxiliary attempt. Missing, late, mismatched or incomplete confirmations remain explicit auxiliaryUncheckedInputs; summaries retain confirmation diagnostics, relation/original-query costs and dropped/omitted observation counts. The four ordinary controls keep their existing behavior.

Based on #607. Only the experiment driver and its tests change (2 files, +351/-6).

Validation:

  • 15 Python unit tests pass, including the existing ordinary-control cases.
  • Actual public receipts accept complete native coverage and the early canonical auxiliary named-check fault; the 5-second confirmation timeout is omitted with its diagnostic.
  • 19 adversarial variants of the actual public receipt are rejected, including mismatched registration, source pins, selection, target/context, null binding/event, wrong input/check, absent coverage and a local target without compiler retention.
  • git diff --check passes.

The later original/counter replay still measures native class coverage. These receipts verify the measurement contract; they do not establish a comparative scientific advantage. CI and integrated campaign results remain unverified at publication.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant