Skip to content

Send an evaluation request timestamp, defaulting to UTC, with every test - #142

Open
bryantaustin13 wants to merge 2 commits into
mainfrom
evaluation-timestamp-and-offset
Open

bryantaustin13 wants to merge 2 commits into
mainfrom
evaluation-timestamp-and-offset

Conversation

@bryantaustin13

@bryantaustin13 bryantaustin13 commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Part of #138. Replaces the approach of #119 (closed).

Problem

CQL evaluates every request against an evaluation request timestamp. Now(), Today() and TimeOfDay() return it, and a DateTime written without an offset takes its offset. In the CQL Author's Guide: "If no timezone offset is specified, the timezone offset of the evaluation request timestamp is used."

The runner sent neither a timestamp nor a timezone. For $cql the request carried only expression, so every result was computed on the server's clock and in the server's zone. The same suite therefore gave different DateTime results depending on where the server ran. Against a server in -06:00:

expected: @2005-05-10T10:20:30
actual  : 2005-05-10T10:20:30-06:00

Change

This is the request side of #138. Comparison is not changed here (see Not in this PR).

Per-test evaluation timing (cqframework/cql-tests#28):

  • A test may set evaluationDateTime (the timestamp itself) or evaluationTimezoneOffset (only the offset).
  • With neither, the test is evaluated at the start of the run, in UTC, as Add support for specifying timezone offset to be used for a test #138 asks.
  • Every test in a run shares the same start instant, each expressed in its own offset.
  • A test that sets both, or an unusable value (an impossible date, an offset beyond ±14:00, a date without a time), is scored error with a clear message and is not sent.
  • cql-tests#28 does not fix an offset format. Both Z / ±HH:MM and CQL's decimal hours (-6.0) are accepted, since the two cannot be confused. The cql-tests schema change could settle on one.

Sent to the server, using both mechanisms Using CQL With FHIR describes (3.0.0-ballot, Timezone and Timezone Offset Handling, FHIR-56046):

  • the timestamp input parameter of $cql and Library/$evaluate, as defined by the IG's OperationDefinitions (0..1 dateTime, "The timestamp of the evaluation request");
  • the Timezone client-timezone header from FHIR's Client Timezone section (R5 http.html#timezones), e.g. Timezone: 2026-09-28T17:12:04.586Z.

A server that supports neither ignores both. This was verified on HAPI FHIR 8.10.0 for $cql and Library/$evaluate.

Reported:

  • At the start of each run the server is probed once:
    • Now() sent with a fixed timestamp shows whether the parameter is used;
    • an offset-less DateTime sent with the header at +05:45 shows whether the header is used.
  • The answer is logged, for example: Evaluation requests default to timezone offset Z; server honours timestamp parameter: no, Timezone header: no.
  • The results file records it under a new top-level evaluationRequest, and each result records the evaluationTimestamp it was sent. Both are optional in the results schema.
  • A failed probe, or a Library/$evaluate configuration (which cannot be probed without publishing a library), records null, meaning unknown.

This matters for reading DateTime failures: against a server that honours neither mechanism, results are in the server's own zone, whatever the test requested.

The README documents the attributes, the defaults, and what is sent.

Verification

Against HAPI FHIR 8.10.0 (CQL engine 5.1.0 via clinical-reasoning 4.10.0), full cql-tests suite:

pass fail skip error
main 1639 170 14 0
this branch 1639 170 14 0
  • 0 status changes and 0 changed actual values across all 1823 records.
  • The probe reports timestampParameterHonored: false, timezoneHeaderHonored: false, which matches direct requests to the server.
  • All 1809 evaluated tests carry the same UTC evaluationTimestamp. The 14 skips carry none, as they are skipped before a request is built.
  • 252 unit tests pass (235 before, 17 new). They cover:
    • offset and date-time parsing, and rejection of bad values;
    • the timestamp on all three request shapes, and the Timezone header sent by runTest;
    • the both-attributes error, with no request made;
    • each probe outcome;
    • a results file with the new fields passing the results schema.
  • test/run-tests.test.ts counts and sequences fetch calls, so it replaces the probe with a fixed answer.

Not in this PR

Comparison. On its own, this change moves no test from fail to pass on any engine, including one that follows the IG. The comparator still compares DateTimes as text, so the expected @2005-05-10T10:20:30 does not match a compliant engine's 2005-05-10T10:20:30Z. The follow-up would:

It is kept separate because it changes scoring, needs decisions (for example, how to report a result from a server that ignored the requested offset), and overlaps #132 (Z / +00:00 equivalence). time-hasOffset also has no engine that emits it to test against yet.

The test attributes. Neither attribute is in testSchema.xsd or used by any test yet. That is cqframework/cql-tests#28.

Notes for the IG

Two small defects in Using CQL With FHIR 3.0.0-ballot, found while implementing this:

  • the prose calls the parameter requestTimestamp, while the OperationDefinitions name it timestamp;
  • the Client Timezone link points to FHIR R4, which has no such section; it is in R5 and later.

tsc --noEmit still reports the two pre-existing rest-routes.ts errors on main, which #139 fixes.

🤖 Generated with Claude Code

bryantaustin13 and others added 2 commits September 28, 2026 11:19
CQL evaluates each request against an evaluation request timestamp: Now(),
Today() and TimeOfDay() return it, and a DateTime written without an offset
takes its offset. The runner sent no timestamp and no timezone, so every
result was computed on the server's clock and in the server's zone, and the
same suite gave different DateTime results depending on where it ran.

This is the request side of #138 (from cqframework/cql-tests#28):

- A test may set evaluationDateTime, or only evaluationTimezoneOffset. With
  neither, it is evaluated at the start of the run in UTC. A test that sets
  both, or an unusable value, is scored as an error and not sent. cql-tests#28
  does not fix an offset format, so both `Z` / `±HH:MM` and CQL decimal hours
  (`-6.0`) are accepted.
- The timestamp is sent as the `timestamp` input of $cql and Library/$evaluate
  (Using CQL With FHIR 3.0.0-ballot OperationDefinitions), and as the
  `Timezone` client-timezone header (FHIR R5 http.html#timezones).
- At the start of a run the server is probed once, and the results record
  whether it honours each mechanism (`evaluationRequest`) and the timestamp
  each test was sent (`evaluationTimestamp`). Both are added to the results
  schema.

Comparison is unchanged. Interpreting offset-less expected DateTimes at the
evaluation offset, and honouring time-hasOffset, is the follow-up step, so
this change on its own moves no test from fail to pass on any engine.

Against HAPI FHIR 8.10.0 / CQFramework engine 4.1.0, which ignores both
mechanisms without error (probe: no / no), the full cql-tests suite is
unchanged at 1639/170/14/0, with no status or actual value changing across
1823 records. Library/$evaluate likewise accepts and ignores them. 252 unit
tests pass (235 before, 17 new).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant