fix: report a non-nullable return field for named_struct() and struct()

[adriangb]

Which issue does this PR close?

• Closes named_struct() and struct() report a nullable return field, so IS NOT NULL on them never folds and blocks struct pruning #25305.

Rationale for this change

named_struct(...) and struct(...) never produce a NULL row: invoke_with_args builds the output StructArray with no null buffer. Both functions nevertheless reported a nullable return field, and ExprSchemable::nullable for a scalar function reads that field. As a result the simplifier rule that rewrites a IS NOT NULL to true for a non-nullable a never fired on a struct constructor.

The user-visible cost is lost pruning. A guard such as WHERE s IS NOT NULL on a struct built by a view cannot remove any row, but it keeps the whole struct expression alive, so projection pushdown can no longer prune the scan to the fields that are read:

SET datafusion.explain.format = 'indent';
CREATE TABLE t (a INT, b INT, c INT) AS VALUES (1, 2, 3), (NULL, 5, 6);
CREATE VIEW v AS SELECT named_struct('a', a, 'b', b, 'c', c) AS s FROM t;
EXPLAIN SELECT s['b'] FROM v WHERE s IS NOT NULL;

Before:

logical_plan
Projection: __datafusion_extracted_1 AS v.s[b]
  SubqueryAlias: v
    Projection: __datafusion_extracted_1
      Filter: named_struct(Utf8("a"), t.a, Utf8("b"), t.b, Utf8("c"), t.c) IS NOT NULL
        Projection: t.b AS __datafusion_extracted_1, t.a, t.b, t.c
          TableScan: t projection=[a, b, c]

After:

logical_plan
Projection: __datafusion_extracted_1 AS v.s[b]
  SubqueryAlias: v
    Projection: t.b AS __datafusion_extracted_1
      TableScan: t projection=[b]

This matches the plan the same query already gets without the guard, and lets the existing get_field(named_struct(...), 'f') simplification (#22239) remove the constructor.

What changes are included in this PR?

• datafusion/functions/src/core/named_struct.rs: return_field_from_args now reports a non-nullable field instead of a nullable one.
• datafusion/functions/src/core/struct.rs: adds a return_field_from_args override that reports a non-nullable field. The default implementation it previously used made the field nullable.

Both are a one-line nullability correction; the inner struct fields stay nullable, since an individual member value can still be NULL.

What is the testing strategy for this PR?

• New sqllogictest cases at the end of datafusion/sqllogictest/test_files/struct.slt covering the folded scalar results (named_struct(...) IS NOT NULL, struct(...) IS NULL), the EXPLAIN for WHERE s IS NOT NULL over a view showing TableScan: t projection=[b], the values that query returns, and the CASE WHEN named_struct(...) IS NOT NULL fold.
• Existing datafusion-functions unit tests: cargo test -p datafusion-functions --lib, 356 passed.
• The full sqllogictest suite: all 518 files pass, with no other expectation changes needed.

Are there any user-facing changes?

Yes, but no API change.

• The schema nullability of the output of named_struct() and struct() changes from nullable to non-nullable. This now matches the array those functions actually build.
• IS NULL and IS NOT NULL applied directly to one of these constructors constant-fold to false and true. Query results are unchanged; plans get smaller and can prune more.

[adriangb]

@kosiew would you mind taking a look at this change please?

[codecov-commenter]

Codecov Report

❌ Patch coverage is 88.88889% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 81.91%. Comparing base (add66e4) to head (a793993).

Files with missing lines	Patch %	Lines
datafusion/functions/src/core/struct.rs	87.50%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #25306      +/-   ##
==========================================
- Coverage   81.91%   81.91%   -0.01%     
==========================================
  Files        1134     1134              
  Lines      425655   425663       +8     
  Branches   425655   425663       +8     
==========================================
- Hits       348688   348665      -23     
- Misses      56288    56314      +26     
- Partials    20679    20684       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[adriangb]

thanks @Jefffrey !