Repository navigation
feat(microsoft): opt-in DAX to OSSIE_SQL_2026 translation for single-column aggregate measures (#584) - #623
Closed
MrBlack1995 wants to merge 2 commits into
Closed
MrBlack1995 wants to merge 2 commits into
MrBlack1995 wants to merge 2 commits into
Conversation
The reverse of _sql_to_dax: a deliberately narrow, deterministic translator for single-column aggregate measures (SUM, AVG, MIN, MAX, COUNT via COUNTA, COUNT DISTINCT via DISTINCTCOUNTNOBLANK), refusing everything else with a reason. Translates exactly the functions _sql_to_dax emits, so an Ossie SQL model exported to Power BI and re-imported round-trips to the same expression. COUNT and DISTINCTCOUNT are refused on purpose: DAX COUNT does not support TRUE/FALSE values and DISTINCTCOUNT counts BLANK as a distinct value, so neither is the exact equivalent of SQL COUNT(x) / COUNT(DISTINCT x); translating them could be a silent wrong number on another engine. Refuse-by-default is the house rule from _sql_to_dax and the argument in apache#437. This is the translator core only; wiring it into _convert_metrics behind an opt-in --translate-dax flag (default off) is the next commit. No existing file is touched, so default behavior is unchanged. 24 unit tests pass. Refs #584 Co-authored-by: Isaac <no-reply@databricks.com>
…(default off) Threads translate_dax (default False) through convert_semantic_model_to_ossie, build_ossie_document and _convert_metrics. When on, a single-column aggregate measure is translated to OSSIE_SQL_2026 as the primary dialect with the original DAX kept as a fallback, so nothing is lost; a measure that cannot be translated safely is kept as DAX with a diagnostic naming the reason. A resolver maps a DAX Table[Column] to the Ossie dataset.field built from the model. Default output is byte-for-byte unchanged (flag off keeps every measure as DAX). Adds the --translate-dax flag to the import subcommand. Integration tests cover default-off, a translated aggregate with DAX fallback, and COUNTROWS refused; _dax_to_sql added to the packaging manifest. Full suite: 468 passed, 2 skipped. Translated output passes validation/validate.py. Refs #584 Co-authored-by: Isaac <no-reply@databricks.com>
Member
|
@MrBlack1995 as shared on the dev mailing list, the converters have moved to a dedicated repository: https://github.com/apache/ossie-converters Can you please create this PR on the new repo? Thanks! |
Contributor
Author
|
Thanks @jbonofre, re-opened as apache/ossie-converters#6. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First slice of apache/ossie-converters#23: an opt-in, deterministic DAX to
OSSIE_SQL_2026translator for Power BI measures on import, so the most common measures leave Power BI in a portable form every Ossie consumer can read.Today every imported measure is DAX only. With
--translate-dax, a measure that is a single aggregate over one fully qualified column getsOSSIE_SQL_2026as its primary dialect, with the original DAX kept as a fallback, so nothing is lost:Everything else is refused and kept exactly as today (DAX only), with a warning naming the reason:
Change
New
ossie_microsoft/_dax_to_sql.py, the reverse of_sql_to_dax.py. It translates exactly the functions_sql_to_daxemits, so an Ossie SQL model exported to Power BI and imported again comes back to the same expression:SUM('T'[c])SUM(T.c)AVERAGE('T'[c])AVG(T.c)MIN('T'[c])/MAX('T'[c])MIN(T.c)/MAX(T.c)COUNTA('T'[c])COUNT(T.c)DISTINCTCOUNTNOBLANK('T'[c])COUNT(DISTINCT T.c)Refused on purpose, following the reasoning already written down in
_sql_to_dax: DAXCOUNT(TRUE/FALSE values unsupported, so it is not SQLCOUNT(x)for an arbitrary column) andDISTINCTCOUNT(counts BLANK as a distinct value). The empty-set convention (DAX BLANK vs SQL 0) is the one_sql_to_daxalready accepts.Also refused: anything with more than one argument or a nested call (
COUNTROWS, iterators,CALCULATE,DIVIDE, arithmetic,IF, time intelligence), unqualified columns, a column that does not resolve to exactly one dataset field, and names that would need SQL quoting. These are later tiers.translate_dax=Falseis threaded throughconvert_semantic_model_to_ossie,build_ossie_documentand_convert_metrics; theimportsubcommand gets--translate-dax. A resolver maps'Table'[Column]to the Ossiedataset.fieldbuilt from the model.Answers to the questions on apache/ossie-converters#23, as implemented (easy to change)
converters/microsoft, in Python, as a sibling of_sql_to_dax.py.catalog.yamlquestion can be settled in the PR that grows it.Measured effect
Running the portability fixtures in #596 (M0 to M3) through Ossie to Power BI to Ossie:
OSSIE_SQL_2026BLANK()placeholder--translate-daxThe 16 returned expressions are identical to the original SQL (
SUM(sales.amount)comes back asSUM(sales.amount),COUNT(DISTINCT sales.customer_id)asCOUNT(DISTINCT sales.customer_id), and so on). The 2 placeholders are the M3 filtered aggregates, which the SQL to DAX export already reduces toBLANK(); that is the export side and out of scope for an import translator.Validation
converters/microsoft: 468 passed, 2 skipped locally;ruff checkclean. 28 new tests intests/test_dax_to_sql.py(golden translations, every refusal, reasons, quoted names, and integration tests for flag off, a translated measure with DAX fallback, and a refused measure).tests/fixtures/sales_model.bim: flag off output is unchanged; flag on output passesvalidation/validate.py; exporting the flag-on document back to Power BI gives the original DAX for every measure.Related Issues
Refs apache/ossie-converters#23. Measured against the fixtures in #596 (#576).
Checklist
Converters
converters/is updated (Microsoft import, opt-in)