Skip to content

feat: add CalendarIntervalType support#4898

Open
peterxcli wants to merge 1 commit into
apache:mainfrom
peterxcli:feat/calendar-interval-type
Open

feat: add CalendarIntervalType support#4898
peterxcli wants to merge 1 commit into
apache:mainfrom
peterxcli:feat/calendar-interval-type

Conversation

@peterxcli

Copy link
Copy Markdown
Member

Which issue does this PR close?

Part of #4540.

Rationale for this change

Comet does not currently support Spark's CalendarIntervalType, causing queries carrying calendar interval columns through Comet operators to fall back to Spark.

This change adds the foundational Arrow and codegen support needed to preserve calendar intervals—months, days, and microseconds—across the Spark, Arrow, and native execution boundaries.

What changes are included in this PR?

  • Map Spark CalendarIntervalType to Arrow Interval(MonthDayNano).
  • Add CALENDAR_INTERVAL to the serialized data type protocol and native Arrow type conversion.
  • Support reading and writing calendar intervals through Arrow vectors, including microsecond-to-nanosecond conversion.
  • Add CalendarIntervalType support to Comet batch-kernel code generation for scalar and nested values.
  • Allow calendar interval columns through Comet’s data type support checks and query plan serialization.
  • Add SQL and unit-test coverage for nullable and non-null calendar interval values.

How are these changes tested?

Added coverage for:

  • Round-tripping CalendarIntervalType through the Arrow writer and CometPlainVector.
  • Round-tripping calendar intervals through the batch-kernel codegen path.
  • SQL queries containing positive, negative, mixed-component, and null calendar interval values with native local table scans enabled.

@peterxcli peterxcli changed the title Add CalendarIntervalType support feat: add CalendarIntervalType support Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant