Skip to content

Add strict pre-sample validation gate profile (promote high-risk warnings to errors) #119

@RandomOscillations

Description

@RandomOscillations

Summary

Current sample flow validates merged spec and proceeds when there are warnings. For high-stakes runs, several warning classes are effectively correctness blockers and should fail early.

Why This Matters

Teams are repeatedly discovering issues downstream (DB contradictions, flat dynamics, no-op modifiers) that could have been blocked before sampling.

A strict gate profile would prevent recursive fix-run cycles.

Current Behavior (Code)

In /Users/adithyasrinivasan/Projects/extropy/extropy/cli/commands/sample.py:

  • merged spec validation runs (line ~165-172)
  • only errors block sampling (line ~175)
  • warnings always pass (line ~196-201)

Proposed Fix

Add a strict gate mode for sample (CLI flag and/or config), e.g. --strict-gates.

In strict mode, promote selected warning categories to errors, including at least:

  • condition value mismatches
  • modifier overlap ambiguity for categorical/boolean outcomes
  • critical scenario validation warnings that imply no-op behavior

Design principles:

  • deterministic and documented category list
  • opt-in initially (to avoid breaking current users)
  • printed summary of promoted warnings

Acceptance Criteria

  • extropy sample ... --strict-gates fails when promoted warning categories are present.
  • Default mode behavior remains backward compatible.
  • JSON output reports promoted categories and counts.

Pipeline Impact

Provides a reliable preflight quality gate for steps 1-3 before expensive downstream runs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions