Repository navigation
test(compliance): add DuckDB-runnable query tests for the Relational Query Interface - #628
christianeu-db wants to merge 1 commit into
Conversation
Add compliance/query/relational_semantics.sql, a DuckDB-runnable file of 29 test cases for core-spec/relational_semantics.md over the SCOTT schema. Each case pairs an Ossie query that uses AGG() (commented out) with an equivalent DuckDB query and its expected result. - Part A (12 cases): AGG() on one relation: grouping, filters, HAVING, NULL and empty-input values, ROLLUP, CTE propagation, subqueries. - Part B (17 cases): joins, including fan-out and many-to-many joins, and two queries run with each join type. The DuckDB queries use the reference evaluation strategy of Section 9. The model scott_model (commented out) uses dataset-scoped metrics (apache#591). The Ossie queries refer to its relations by two-part name, for example scott_model.emp. This is a light version of the .ost test format (apache#501) and view expander (apache#502). Co-authored-by: Isaac <no-reply@databricks.com>
1b4a613 to
a3dcba5
Compare
|
fyi: @julianhyde |
|
I checked the expected results against Section 8.1 by a second route. For B1-B17 I wrote each measure's input set out directly from 8.1, mostly as Two values depend on how "host dialect" in Section 8.1 is read:
Are the expected values in these cases meant for every engine, or for a host with standard SQL semantics? A1-A4, A6, A8, A10-A12, B1-B9, B13 and B14 also return the expected rows on ClickHouse 26.9 with default settings (translated by SQLGlot 30.24.0). That may help pick the first cases to merge. Queries for B1-B17-- B1: many-to-one, the join repeats no emp row
SELECT d.dname, SUM(e.sal), COUNT(*) FROM scott.emp e JOIN scott.dept d USING (deptno) GROUP BY 1 ORDER BY 1;
-- B2: many-to-one, the join repeats no emp row
SELECT e.job, SUM(e.sal), COUNT(*) FROM scott.emp e JOIN scott.dept d USING (deptno) WHERE d.loc = 'NEW YORK' GROUP BY 1 ORDER BY 1;
-- B3
SELECT e.job, SUM(e.sal) FROM scott.emp e WHERE e.deptno IN
(SELECT deptno FROM scott.emp GROUP BY deptno ORDER BY SUM(sal) DESC LIMIT 2) GROUP BY 1 ORDER BY 1;
-- B4
SELECT j.job,
(SELECT COUNT(*) FROM scott.dept d WHERE EXISTS (SELECT 1 FROM scott.emp e WHERE e.deptno = d.deptno AND e.job = j.job)),
(SELECT COUNT(*) FROM scott.emp e WHERE e.job = j.job AND EXISTS (SELECT 1 FROM scott.dept d WHERE d.deptno = e.deptno))
FROM (SELECT DISTINCT e.job FROM scott.emp e JOIN scott.dept d USING (deptno)) j ORDER BY 1;
-- B5
SELECT j.job,
(SELECT COUNT(*) FROM scott.emp e WHERE e.job = j.job AND EXISTS (SELECT 1 FROM scott.emp m WHERE e.mgr = m.empno)),
(SELECT COUNT(*) FROM scott.emp m WHERE EXISTS (SELECT 1 FROM scott.emp e WHERE e.mgr = m.empno AND e.job = j.job)),
(SELECT SUM(m.sal) FROM scott.emp m WHERE EXISTS (SELECT 1 FROM scott.emp e WHERE e.mgr = m.empno AND e.job = j.job))
FROM (SELECT DISTINCT e.job FROM scott.emp e JOIN scott.emp m ON e.mgr = m.empno) j ORDER BY 1;
-- B6
SELECT g.deptno,
(SELECT COUNT(*) FROM scott.emp e WHERE e.deptno = g.deptno AND EXISTS (SELECT 1 FROM scott.salgrade s WHERE e.sal BETWEEN s.losal AND s.hisal)),
(SELECT COUNT(*) FROM scott.salgrade s WHERE EXISTS (SELECT 1 FROM scott.emp e WHERE e.deptno = g.deptno AND e.sal BETWEEN s.losal AND s.hisal))
FROM (SELECT DISTINCT e.deptno FROM scott.emp e JOIN scott.salgrade s ON e.sal BETWEEN s.losal AND s.hisal) g ORDER BY 1;
-- B7
SELECT g.grade,
(SELECT COUNT(*) FROM scott.emp e WHERE e.sal BETWEEN g.losal AND g.hisal AND EXISTS (SELECT 1 FROM scott.dept d WHERE d.deptno = e.deptno)),
(SELECT SUM(e.sal) FROM scott.emp e WHERE e.sal BETWEEN g.losal AND g.hisal AND EXISTS (SELECT 1 FROM scott.dept d WHERE d.deptno = e.deptno)),
(SELECT COUNT(*) FROM scott.dept d WHERE EXISTS (SELECT 1 FROM scott.emp e WHERE e.deptno = d.deptno AND e.sal BETWEEN g.losal AND g.hisal))
FROM (SELECT DISTINCT s.grade, s.losal, s.hisal FROM scott.emp e JOIN scott.dept d USING (deptno) JOIN scott.salgrade s ON e.sal BETWEEN s.losal AND s.hisal) g ORDER BY 1;
-- B8: every emp row of the department appears on both sides
SELECT g.deptno,
(SELECT SUM(e.sal) FROM scott.emp e WHERE e.deptno = g.deptno),
(SELECT COUNT(*) FROM scott.emp c WHERE c.deptno = g.deptno)
FROM (SELECT DISTINCT e.deptno FROM scott.emp e JOIN scott.emp c ON e.deptno = c.deptno) g ORDER BY 1;
-- B9
SELECT (SELECT COUNT(*) FROM scott.emp e WHERE EXISTS (SELECT 1 FROM scott.emp m WHERE e.mgr = m.empno)),
(SELECT SUM(sal) FROM scott.emp e WHERE EXISTS (SELECT 1 FROM scott.emp m WHERE e.mgr = m.empno)),
(SELECT COUNT(*) FROM scott.emp m WHERE EXISTS (SELECT 1 FROM scott.emp e WHERE e.mgr = m.empno)),
(SELECT SUM(sal) FROM scott.emp m WHERE EXISTS (SELECT 1 FROM scott.emp e WHERE e.mgr = m.empno));
-- B10: every e row survives the LEFT JOIN
SELECT (SELECT COUNT(*) FROM scott.emp), (SELECT SUM(sal) FROM scott.emp),
(SELECT COUNT(*) FROM scott.emp m WHERE EXISTS (SELECT 1 FROM scott.emp e WHERE e.mgr = m.empno)),
(SELECT SUM(sal) FROM scott.emp m WHERE EXISTS (SELECT 1 FROM scott.emp e WHERE e.mgr = m.empno));
-- B11: every m row survives the RIGHT JOIN
SELECT (SELECT COUNT(*) FROM scott.emp e WHERE EXISTS (SELECT 1 FROM scott.emp m WHERE e.mgr = m.empno)),
(SELECT SUM(sal) FROM scott.emp e WHERE EXISTS (SELECT 1 FROM scott.emp m WHERE e.mgr = m.empno)),
(SELECT COUNT(*) FROM scott.emp), (SELECT SUM(sal) FROM scott.emp);
-- B12: every row of both sides survives the FULL JOIN
SELECT (SELECT COUNT(*) FROM scott.emp), (SELECT SUM(sal) FROM scott.emp), (SELECT COUNT(*) FROM scott.emp), (SELECT SUM(sal) FROM scott.emp);
-- B13: every row of both sides survives the CROSS JOIN
SELECT (SELECT COUNT(*) FROM scott.emp), (SELECT SUM(sal) FROM scott.emp), (SELECT COUNT(*) FROM scott.emp), (SELECT SUM(sal) FROM scott.emp);
-- B14
SELECT dc, ec, ss, ss / dc FROM (SELECT
(SELECT COUNT(*) FROM scott.dept d WHERE EXISTS (SELECT 1 FROM scott.emp e WHERE e.deptno = d.deptno)) dc,
(SELECT COUNT(*) FROM scott.emp e WHERE EXISTS (SELECT 1 FROM scott.dept d WHERE e.deptno = d.deptno)) ec,
(SELECT SUM(sal) FROM scott.emp e WHERE EXISTS (SELECT 1 FROM scott.dept d WHERE e.deptno = d.deptno)) ss);
-- B15: every dept row survives the LEFT JOIN
SELECT dc, ec, ss, ss / dc FROM (SELECT (SELECT COUNT(*) FROM scott.dept) dc,
(SELECT COUNT(*) FROM scott.emp e WHERE EXISTS (SELECT 1 FROM scott.dept d WHERE e.deptno = d.deptno)) ec,
(SELECT SUM(sal) FROM scott.emp e WHERE EXISTS (SELECT 1 FROM scott.dept d WHERE e.deptno = d.deptno)) ss);
-- B16: every dept row survives; only clerks match on the emp side
SELECT dc, ec, ss, ss / dc FROM (SELECT (SELECT COUNT(*) FROM scott.dept) dc,
(SELECT COUNT(*) FROM scott.emp e WHERE e.job = 'CLERK' AND EXISTS (SELECT 1 FROM scott.dept d WHERE e.deptno = d.deptno)) ec,
(SELECT SUM(sal) FROM scott.emp e WHERE e.job = 'CLERK' AND EXISTS (SELECT 1 FROM scott.dept d WHERE e.deptno = d.deptno)) ss);
-- B17: every dept row survives the LEFT JOIN
SELECT d.dname,
(SELECT COUNT(*) FROM scott.emp e WHERE e.deptno = d.deptno),
(SELECT SUM(sal) FROM scott.emp e WHERE e.deptno = d.deptno),
(SELECT AVG(sal) FROM scott.emp e WHERE e.deptno = d.deptno)
FROM scott.dept d ORDER BY 1; |
Hi @ekut, thanks for trying these out! I think we'd be aiming for standard SQL semantics as much as possible. So, for the empty result case, we'd be following ISO/IEC 9075 (SQL), which says empty sum / avg => null. The numeric point is an interesting one, esp. since a query optimization within the same engine can lead to slight differences due to floating point math. The most important thing for this suite is the query shapes. For numeric queries, we might need to work with simpler types (e.g. non-decimal) or accept some reasonable margin of error to account for precision |
|
I agree that we should be aiming for sql compliance when using the standard Ossie dialect. I can update the references to host language in #354 to make this clearer. We should update the expression language spec to clarify this as well. There will remain some gaps because the sql standard does not completely define function typing. For example, it allows |
Summary
This PR adds
compliance/query/relational_semantics.sql: 29 test cases for the Relational Query Interface (#354). The file runs in DuckDB with no extensions:duckdb < compliance/query/relational_semantics.sqlThe file loads the SCOTT schema and defines an Ossie model,
scott_model, over it with dataset-scoped metrics (#591). Each test case has three parts:AGG()(in comments).The cases:
GROUP BY,WHERE,HAVING/ORDER BY/LIMIT, scalar expressions overAGG(), NULL and empty-input values,GROUP BY ALL,ROLLUP, a measure through a CTE,AGG()in a scalar subquery, and two-stage aggregation.INNER,LEFT,RIGHT,FULL, andCROSS;INNER, asLEFT, and asLEFTwith a filter inON.For joins, the DuckDB query uses the reference evaluation strategy of Section 9. It carries each dataset's primary key through the join as the row identity, removes duplicate rows by that key, and then aggregates. Thus each expected result follows from the definition in the spec. It does not depend on a separate hand-written rewrite.
Example (fan-out in a self-join). The 4 salesmen all report to BLAKE, so
AGG(m.sum_sal)counts BLAKE's salary once. A plainSUM(m.sal)gives 11400.Scope: behavior first, format later
The purpose of this PR is to pin down behavior now. Each case is a concrete query with a concrete result, so the working group can agree on what the spec requires before an engine implements it. Comments on the expected results are more important than comments on the file format.
The format is deliberately simple: one SQL file, checked by eye or by running DuckDB. A formal test format and runner (#501) and a view expander (#502) are follow-up work. When those are available, each case can become an
.osttest, and the runner can check the Ossie query directly instead of its DuckDB equivalent.Verification
duckdb < compliance/query/relational_semantics.sqlexits with status 0 (DuckDB 1.5.6).Related Issues
Related: #354, #591, #501, #502.
Checklist
Specification
core-spec/and follow the existing structure (N/A: no spec change)Ontology
ontology/are consistent with spec changes (N/A)Converters
Validation
validation/are updated if the spec changed (N/A)Documentation
docs/is updated to reflect any user-facing changes (N/A)compliance/query/README.md)CONTRIBUTING.mdis updated if the contribution process changed (N/A)Examples
examples/are added or updated for any new spec constructs (N/A)Tests
pytest/ CI green) (no code change; waiting for CI)Compliance
NOTICEand/orLICENSEhave been updated per ASF policy (N/A: no code is copied; the SCOTT data values are the same as in the Apache Calcite tests)AI disclosure
Per the ASF Generative Tooling Guidance, this contribution was prepared with AI assistance. I have reviewed and verified every change.