Skip to content

test(compliance): add portability rungs M4-M6 and the FILTER (WHERE) form of M3 - #631

Open
MrBlack1995 wants to merge 5 commits into
apache:mainfrom
MrBlack1995:test/portability-m3filter-m4-m5-m6
Open

MrBlack1995 wants to merge 5 commits into
apache:mainfrom
MrBlack1995:test/portability-m3filter-m4-m5-m6

Conversation

@MrBlack1995

@MrBlack1995 MrBlack1995 commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #596: the next portability rungs, one commit each, on top of the M0-M3 set. Each rung still adds one construct, so a difference in a converter's output localizes to it. Happy to drop or split any of these.

Rung Construct Fixture
M3 (new variants) the same filtered aggregate written with the aggregate FILTER (WHERE ...) clause, which expression_language.md makes REQUIRED, qualified and bare m3/m3_filter_clause_qualified.ossie.yaml, m3/m3_filter_clause_bare.ossie.yaml
M4 two datasets, a declared relationship, and a model-scoped metric across the join m4/m4_cross_dataset.ossie.yaml
M5 a model-scoped metric composed from two other model-scoped metrics referenced by name (Metric Scoping in core-spec/spec.md) m5/m5_metric_ref.ossie.yaml
M6 date and time scalars: YEAR, DATE_TRUNC, part-first DATEDIFF(part, start, end) m6/m6_date_scalars.ossie.yaml

manifest.yaml gets one entry per rung (and the two M3 variants), and the README table lists M0-M6.

M7 and M8 (window and offset functions) are not included: they wait for the default NULL placement in window ORDER BY (#587).

Validation

  • All 13 fixtures pass validation/validate.py on current main.
  • Every manifest entry points to an existing file; the grader's tests pass (13).

Related Issues

Refs apache/ossie-converters#24 (portability measurement). Follows #596.

MrBlack1995 and others added 5 commits October 23, 2026 18:13
One dataset with YEAR and DATE_TRUNC fields and an AVG(DATEDIFF(...)) metric,
using the part-first DATEDIFF(part, start, end) order from
expression_language.md. Adds one construct on top of the baseline.

Co-authored-by: Isaac <no-reply@databricks.com>
expression_language.md makes the aggregate FILTER (WHERE ...) clause REQUIRED.
Two variants (qualified and bare) express the same filtered aggregate as the
existing CASE WHEN fixtures, so a difference between the two forms isolates
handling of the clause itself.

Co-authored-by: Isaac <no-reply@databricks.com>
Two datasets joined by a declared relationship and one model-scoped metric
that aggregates across the join (SUM over one dataset divided by a COUNT
DISTINCT over the other), referencing fields as dataset.field.

Co-authored-by: Isaac <no-reply@databricks.com>
A model-scoped metric (avg_order_value) composed from two other model-scoped
metrics referenced by name, per Metric Scoping in core-spec/spec.md.

Co-authored-by: Isaac <no-reply@databricks.com>
Lists the M3 FILTER (WHERE ...) form and rungs M4-M6 in the README table,
notes that M7 and M8 wait on the window ORDER BY NULL-placement default, and
records that the fixtures live in this repository.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant