Repository navigation
Phase 1: make every suite gate measurable #74
Copy link
Copy link
Closed
4 / 44 of 4 issues completedLabels
enhancementNew feature or requestNew feature or requestgate:L0/L1Blocks the layout/lexing and parse gates (D-04 L0, L1)Blocks the layout/lexing and parse gates (D-04 L0, L1)gate:L4 gate:L5 gate:L6/M7 gate:M8 area:harness
Description
Why this phase is first
Three of the eight gate rows in docs/design/D-04-suite-roadmap.md cannot be improved blind, because nothing measures them:
| Gate | Current state |
|---|---|
| L4 core type checking | No scoreboard, and TypeCheckError::error_code maps only EscapedSkolem and InvalidCoercibleInstanceDeclaration. A type mismatch is reported with a message and no code. |
| L5 classes and instances | No scoreboard, and none of the mapped class codes is emitted. |
| L6/M7 runtime | No harness compiles and runs a corpus passing file. |
| M8-W warnings | The corpus's @shouldWarnWith headers are read by nothing. |
| M8-O optimization | optimize is not vendored; its goldens are .out.js, not CoreFn. |
Until these exist, every later phase can claim progress that no number supports. This phase buys the ability to measure; it changes no user-visible language behavior.
Scope
- Official
errorCodemapping for type and class diagnostics.psrs-typecheck'sTypeCheckErrorKindalready distinguishesTypeMismatch,OccursCheck,IntegerOutOfRange,NoInstance,OverlappingInstances,MissingInstanceMethod,AmbiguousConstraint, andFundepConflict, buterror_code()returnsNonefor all of them. Map the ones with an unambiguouspurscounterpart. This alone makes 61 M4 cases and 92 M5 cases measurable. - A lenient type-check entry point.
psrs_driverexposescheck_program_lenient(resolution) andcheck_program_kinds_lenient; there is no type-check equivalent that tolerates missing support libraries, so L4/L5 cannot be scored against the corpus the way L2/L3 are. - L4 and L5 scoreboards, following the existing
l2_resolution_scoreboard_with_annotations/l3_kind_scoreboard_with_annotationspattern: read@shouldFailWith, resolve leniently, report per-code agreement. - Harness module path. The scoreboard loads only a case's same-stem support directory and provides no library modules. Two consequences, both measured: 13
passingfiles fail because a case's other siblings in the same directory are not loaded (passing/Import/M2.purs,passing/Coercible/Lib.purs), and 230 fail purely on library modules. Puttingstdlib/libon the scoreboard's module path and loading a case's sibling modules from the containing directory is a harness-only change. - A runtime compile/run scoreboard for
passing, and a warning-code scoreboard forwarning. - A stated M8-O comparison. Decide what "optimize output agrees" means when the goldens are JavaScript, and record it.
Gate
Every row of the D-04 gate table carries a live measured number, and the L1–L3 numbers in that document are reproducible from the vendored corpus with no external dependency.
Sub-issues
Tracked as children of this epic.
References
docs/design/D-04-suite-roadmap.md— Progress measurement, gate tablecrates/psrs-driver/tests/suite.rs— the L1/L2/L3 scoreboards to extendcrates/psrs-typecheck/src/typecheck/error.rs—TypeCheckErrorKindanderror_codecrates/psrs-driver/src/program/lenient.rs— the lenient entry points
Validation
cargo fmt --all --check
cargo test --workspace
cargo clippy --workspace --all-targets -- -D warnings
PSRS_ORACLE=annotations cargo test -p psrs-driver --test suite -- --ignored --nocapture
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestgate:L0/L1Blocks the layout/lexing and parse gates (D-04 L0, L1)Blocks the layout/lexing and parse gates (D-04 L0, L1)gate:L4 gate:L5 gate:L6/M7 gate:M8 area:harness
Activity
feat(harness): add the L6/M7 compile-and-run scoreboard (#107)
biuld commentedon Oct 2, 2026
Phase 1 is complete. All four slices are closed, and the four gates that were unmeasurable now report numbers.
passingfile.errorCodes, a lenient type-check entry point, and the L4/L5 scoreboards.Every gate the phase set out to measure now has a number:
None of the phase's slices changed a user-visible compiler behavior, which is what the phase promised. The harness work also removed a real defect:
integration.rshad a second privaterun_program_with_wasmtimethat shadowed the shared one and lackedPSRS_REQUIRE_WASMTIMEhandling, so an execution test could silently skip under the required baseline.What the numbers say, honestly: the phase made the gates measurable, not passing. The single largest finding is that most of what remains is not a checker gap — 16 of 26 M4 mismatches, 26 of 37 M5 mismatches, and 214 of 361 unresolved
passingfiles are blocked on a library module that does not exist yet. That is Phase 3 (#94Prelude, #95Effect/Effect.Console/Test.Assert), and the boards now say so per case rather than reporting one number.Evidence for each slice is in its closing comment; CI is green on #107, #108, and #110.
feat(harness): add the L6/M7 compile-and-run scoreboard (#107)
feat(harness): resolve each corpus case the way the compiler loads a …
feat(typecheck): map the official errorCodes and add a lenient type c…