[Python] Dataset.head(-1) returns a Table with negative num_rows #55295

Dataset.head, Scanner.head and Fragment.head accept a negative num_rows and return a Table that fails Table.validate() with Array length is negative, the check in ValidateLayout (cpp/src/arrow/array/validate.cc:477). len() on it raises SystemError.

import os, tempfile
import pyarrow as pa
import pyarrow.dataset as ds
import pyarrow.parquet as pq

print(pa.__version__)
d = tempfile.mkdtemp()
pq.write_table(pa.table({"a": [1, 2, 3]}), os.path.join(d, "x.parquet"))
dataset = ds.dataset(d)

t = dataset.head(-1)
print(t.num_rows)
try:
    t.validate()
except pa.ArrowInvalid as e:
    print(type(e).__name__, e)
try:
    len(t)
except BaseException as e:
    print(type(e).__name__, e)

print(dataset.scanner().head(-2).num_rows)

$ python -I head_negative.py     # pyarrow 26.0.0 wheel, Python 3.12, Linux x86_64
26.0.0
-1
ArrowInvalid Column 0: In chunk 0: Invalid: Array length is negative
SystemError <built-in function len> returned NULL without setting an exception
-2

The R binding already rejects this: head.Scanner in r/R/dataset-scan.R runs assert_that(n >= 0), with the comment that a negative n would need nrow(x) and so a full scan. That rules out the pandas reading of head(-n). On the same wheel pa.array([1, 2, 3]).slice(0, -1) raises ValueError: Length must be non-negative.

The Python wrappers forward num_rows to Scanner::Head. AsyncScanner::Head (cpp/src/arrow/dataset/scanner.cc:595, 9eb748d) only special-cases 0, then calls Slice(0, num_rows) on each batch, so a negative count yields batches of negative length. I read the source at 9eb748d and ran the 26.0.0 wheel, not a source build.

Expected: ArrowInvalid for num_rows < 0, raised in AsyncScanner::Head so Python and C++ callers both get it. Since ArrowInvalid subclasses ValueError, the Cython layer needs no separate check.

I can open a PR with that check and a test in test_dataset.py. Would you take it?