Describe the bug
DataFusion 54.0.0 returns an empty result for a grouped aggregate query where the aggregate value is NULL, followed by ORDER BY ... NULLS FIRST LIMIT.
The grouped aggregate should still produce one output row. The final ORDER BY / LIMIT should not remove it.
This looks related to #22190, but I can still reproduce it on the latest datafusion==54.0.0 with a smaller reproducer.
To Reproduce
import datafusion
import pyarrow as pa
from datafusion import SessionContext
print("datafusion", datafusion.__version__)
batch = pa.record_batch(
[
pa.array(["gamma"], type=pa.string()),
pa.array([None], type=pa.float64()),
],
names=["s", "y"],
)
ctx = SessionContext()
ctx.register_record_batches("t0", [[batch]])
query = """
SELECT max_y
FROM (
SELECT s, MAX(y) AS max_y
FROM t0
GROUP BY s
)
ORDER BY max_y DESC NULLS FIRST
LIMIT 3
"""
result = ctx.sql(query).to_pandas()
print(result)
print("row count:", len(result))
### Expected behavior
### Expected behavior
The query should return one row:
max_y
0 NULL
Reason: the input contains one group, s = 'gamma'. MAX(y) over a group where all y values are NULL should produce one grouped output row with max_y = NULL.
### Actual behavior
DataFusion 54.0.0 returns zero rows:
Empty DataFrame
Columns: [max_y]
Index: []
row count: 0
### Additional context
### Environment
- datafusion==54.0.0
- pyarrow==24.0.0
- Python 3.12.3
- Linux x86_64
Describe the bug
DataFusion 54.0.0 returns an empty result for a grouped aggregate query where the aggregate value is
NULL, followed byORDER BY ... NULLS FIRST LIMIT.The grouped aggregate should still produce one output row. The final
ORDER BY/LIMITshould not remove it.This looks related to #22190, but I can still reproduce it on the latest
datafusion==54.0.0with a smaller reproducer.To Reproduce