Skip to content

Added invalid handling - #114

Closed
bryantaustin13 wants to merge 1 commit into
mainfrom
equivTupleTestFix
Closed

bryantaustin13 wants to merge 1 commit into
mainfrom
equivTupleTestFix

Conversation

@bryantaustin13

Copy link
Copy Markdown
Contributor

Tests with "invalid": "semantic", handled as purposely failing as a valid test.


Moved from #98, which was raised from a fork before I had committer access on this repository. Same work, same branch name, now hosted here so CI and reviews run against cqframework directly.

@bryantaustin13

Copy link
Copy Markdown
Contributor Author

Superseded by #121.

Re-evaluated this against current main and most of it has been overtaken:

  • The parsedExpected !== undefined guard is already on main (run-test-core.ts), identical.
  • The singletonListKeys guard is unnecessary — singletonListKeysFromExpected already returns an empty set for a non-array input.
  • isExpectedInvalidResult() is superseded by Fix error-expecting test handling in CLI and server runners #116's responseIndicatesError, which checks the response status and looks structurally for an evaluation error parameter rather than regex-matching the stringified actual. Fix error-expecting test handling in CLI and server runners #116's version is the better one and is already merged.
  • Letting invalid tests run without an <output> is already on main (results-shared.ts).

What remained was the capability normalization, which is a real and unfixed bug: a test declaring exactly one <capability> parses to a bare object rather than an array, so Result dropped it — 7,390 tests in the suite versus 513 with two or more. #121 is that fix on its own, with tests, on a branch name that describes it (normalize-single-capability). Since a PR's head branch cannot be repointed, it had to be a new PR.

Closing in favour of #121 — please continue any discussion there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant