Nested DATE to numeric casts in structs and maps return the day count or fail

[andygrove]

Describe the bug

In Legacy mode (spark.sql.ansi.enabled=false, the default on Spark 3.x), Spark returns NULL for
a cast from DATE to any numeric or boolean type, including when the date is a struct field or a
map value. Comet runs those nested casts natively and gets them wrong:

• DATE to INT returns the day count since the epoch instead of NULL, silently.
• DATE to BIGINT, SMALLINT, TINYINT, BOOLEAN, FLOAT, DOUBLE or DECIMAL fails the
  query with an internal error (SQLSTATE XX000), for example
  Native cast invoked for unsupported cast from Date32 to Boolean.

The same happens one level further down, for example ARRAY<STRUCT<d: DATE>> to
ARRAY<STRUCT<d: INT>>. Arrays of dates are not affected.

Steps to reproduce

SET spark.sql.ansi.enabled=false;
CREATE TABLE t(s struct<d: date>, m map<string, date>) USING parquet;
INSERT INTO t VALUES (named_struct('d', date '2024-01-15'), map('k', date '2024-01-15'));

SELECT cast(s AS struct<d: int>), cast(m AS map<string, int>) FROM t;
-- Spark: {null}, {k -> null}
-- Comet: {19737}, {k -> 19737}

SELECT cast(s AS struct<d: boolean>) FROM t;
-- Spark: {null}
-- Comet: task failure, Native cast invoked for unsupported cast from Date32 to Boolean

Reproduced on main at 7649361 with Spark 4.1, and the plan is fully native (CometProject).

Cause

CometCast.convert replaces a top-level cast that is always null (isAlwaysCastToNull) with a
null literal, so isSupported reports DATE to a numeric or boolean type as Compatible. The
struct and map arms of isSupported recurse into that same check, but a nested cast does reach
the native kernel. There, (Date32, Int32) reinterprets the day count, which is the kernel
unix_date relies on, and the other targets have no kernel at all. Arrays of dates already have a
guard: ARRAY<DATE> is unsupported to anything other than ARRAY<INT> and ARRAY<STRING>, and
the list kernel returns nulls for DATE to INT.

Expected behavior

Match Spark and return NULL.