Render date-only and time-only analytics-engine columns as SQL DATE / TIME#5524
Render date-only and time-only analytics-engine columns as SQL DATE / TIME#5524ahkcs wants to merge 2 commits into
Conversation
PR Reviewer Guide 🔍(Review updated until commit 6a14c48)Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Latest suggestions up to 6a14c48
Previous suggestionsSuggestions up to commit 88df0b3
|
Consumes the DateType UDT added in opensearch-project/OpenSearch (analytics-api): when the analytics-engine catalog types a date-only column as DateType, OpenSearchTypeFactory.convertAnalyticsEngineRelDataTypeToExprType now maps it to ExprCoreType.DATE (the response schema label), and AnalyticsExecutionEngine materializes the cell as an ExprDateValue by truncating the timestamp result to a LocalDate. Mirrors the existing IpType / BinaryType dispatch in both classes. This flips date-only fields on the analytics route from TIMESTAMP (1984-04-12 00:00:00) to DATE (1984-04-12), matching the v2 / Calcite path. Updates CalciteAnalyticsDatetimeWireFormatIT.testDateRootColumnYmdFormat, which previously asserted the widened-timestamp behavior, to expect the date type and value. Depends on the analytics-api DateType class; compiles once that change is published to the snapshot repo (same cross-repo pattern as IpType/BinaryType). Signed-off-by: Kai Huang <ahkcs@amazon.com>
Phase 2, extending the DateType consumer. Maps a TimeType column to ExprCoreType.TIME in OpenSearchTypeFactory.convertAnalyticsEngineRelDataTypeToExprType and materializes the cell as an ExprTimeValue (timestamp result truncated to a LocalTime) in AnalyticsExecutionEngine. Mirrors the DateType dispatch. Flips time-only fields on the analytics route from TIMESTAMP (1970-01-01 09:00:00) to TIME (09:00:00), matching the v2 / Calcite path. Updates CalciteAnalyticsDatetimeWireFormatIT.testTimeRootColumnHmsFormat, which previously asserted the widened-timestamp behavior, to expect the time type and value. Depends on the analytics-api TimeType class (opensearch-project/OpenSearch#22062). Signed-off-by: Kai Huang <ahkcs@amazon.com>
88df0b3 to
6a14c48
Compare
|
Persistent review updated to latest commit 6a14c48 |
Description
Companion SQL-plugin side of DATE + TIME logical-type support on the analytics-engine route. Pairs with opensearch-project/OpenSearch#22062, which adds
DateType/TimeTypeCalcite UDTs toanalytics-apiand emits them for date-only / time-only-format columns.This change consumes those UDTs in the two SQL-plugin dispatch points that already special-case
IpType/BinaryType:OpenSearchTypeFactory.convertAnalyticsEngineRelDataTypeToExprType— maps aDateTypecolumn toExprCoreType.DATEand aTimeTypecolumn toExprCoreType.TIME(the response schema label).AnalyticsExecutionEngine.toExprValue— when the column isDateType/TimeType, materializes the cell as anExprDateValue/ExprTimeValueby truncating the timestamp result to aLocalDate/LocalTime(epoch interpreted at UTC, matching OpenSearch storage).Net effect on the analytics route: a date-only field renders as a SQL
DATE(1984-04-12) and a time-only field as a SQLTIME(09:00:00), instead of a widenedTIMESTAMP, matching the v2 / Calcite path. Datetime columns are unchanged.Pass rate (analytics route,
-Dtests.analytics.parquet_indices=true)CalcitePPLAggregationIT.testCountByDateTypeSpanForDifferentFormats(+paginating)CalcitePPLAggregationIT.testCountByDateTypeSpanWithDifferentUnits(+paginating)CalcitePPLAggregationIT.testCountByTimeTypeSpanForDifferentFormatsCalcitePPLAggregationIT.testCountByTimeTypeSpanWithDifferentUnitsCalcitePPLAggregationIT.testCountByNullableTimeSpanCalcitePPLAggregationIT.testCountBySpanForCustomFormatsCalciteAnalyticsDatetimeWireFormatIT(date + time cols)CalciteAnalyticsDatetimeWireFormatIT.testDateRootColumnYmdFormat/testTimeRootColumnHmsFormatpreviously asserted the widened-timestamp behavior; they now expectdate/time. The v2 / Calcite path is unaffected (this code path is analytics-route only). No regressions — only columns whose mapping format is date-only or time-only change type.Dependency
Compiles once OpenSearch#22062 is published to the
org.opensearch.sandbox:analytics-apisnapshot (theDateType/TimeTypeclasses), the same cross-repo coupling the existingIpType/BinaryTypeconsumers have. Do not merge before #22062 lands and the snapshot is bumped.Check List
--signoff.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.