Skip to content

Support library-style tests - #111

Closed
bryantaustin13 wants to merge 4 commits into
cqframework:mainfrom
bryantaustin13:library-test-support
Closed

bryantaustin13 wants to merge 4 commits into
cqframework:mainfrom
bryantaustin13:library-test-support

Conversation

@bryantaustin13

Copy link
Copy Markdown
Contributor

Summary

Adds runner support for tests that provide a full CQL <library> and a named
<output>, complementing the schema change in cqframework/cql-tests#141.
Enables cqframework/cql-tests#61 to actually execute, and is the runner side of
cqframework/cql-tests#17.

Changes

  • models/test-types.ts: add TestLibrary; make expression optional; add
    library and TestOutput.name.
  • shared/results-shared.ts: build a Result from a library test (evaluates the
    define named by <output name>, defaulting to output) and pass the library
    inline via the $cql content parameter.
  • commands/build-cql-command.ts: skip library tests (they are evaluated inline,
    not wrapped as generated define statements).

Wire format

The cqframework $cql operation accepts the whole library as a top-level
content (string) parameter plus expression = the unqualified define name;
it returns the result keyed by that define name. (HL7 base $cql only references
libraries by URL; this uses the cqframework inline-content extension.)

⚠️ Authoring note (engine caching)

The engine caches inline libraries by name + version, so every
library-style test must declare a unique library name
or a stale compiled
copy is returned. Discovered while validating; flagging for test authors. A
future hardening could auto-uniquify the name/version per test.

Validation

tsc --noEmit clean; ran the runner against #61's file →
SimpleOverloadMatching status: pass.

Relationship to other work

@brynrhodes brynrhodes left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no content parameter in the $cql operation, and that's probably not the best way to run Library-level tests anyway. They ought to be using the Library$evaluate, which has a library parameter that you can use to pass the Library as a FHIR resource. Note that that will require the CQL content to be wrapped as a base-64 encoded content element in the FHIR library (i.e. the FHIR Library resource SHALL conform to the CQLLibrary profile in the Using CQL IG).

@bryantaustin13

Copy link
Copy Markdown
Contributor Author

Changes since the Library/$evaluate review

The $cql content parameter is gone. Library-style tests now go to Library/$evaluate, with the CQL wrapped as a CQLLibrary-profile FHIR Library (logic-library, one text/cql content element, base64 data).

One deviation: the library is published and referenced by url rather than passed inline. The inline library parameter isn't usable on any current engine — clinical-reasoning's EvaluateProcessor.evaluate passes only the canonical through and discards content, so both the Alphora sandbox and local HAPI 8.10.0 answer "Could not load source for library ...". The runner PUTs the Library, evaluates, and deletes it in a finally.

Also: library tests are skipped rather than mis-sent when a server is configured for $cql; the define name comes from with no silent default; CqlOperation is now constrained to ["$cql", "$evaluate"].

Verified on local HAPI 8.10.0 — 5 of 6 overload-matching tests pass end-to-end, nothing left on the server. The sixth (invalid="semantic", no ) hits a pre-existing skip path for error-expecting tests. 146 unit tests pass, tsc clean.

@bryantaustin13

Copy link
Copy Markdown
Contributor Author

Superseded by #118. Now that I have committer access on cqframework/cql-tests-runner, this work has moved to a branch on the repository itself (same branch name, library-test-support) so CI and reviews run here directly. Closing in favour of #118 — please continue any discussion there.

@brynrhodes — your requested changes here have been addressed on the new PR: library-style tests now publish a CQLLibrary resource and evaluate it by url via Library/$evaluate, with no inline CQL passed to $cql. Would appreciate a fresh look at #118 when you have a moment.

@bryantaustin13
bryantaustin13 deleted the library-test-support branch August 17, 2026 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants