GH-48473: [CI][Python] Require numpy 2.0

[alippai]

Rationale for this change

Require Numpy 2.0 to support newer features, allow newer numpy API/ABI

What changes are included in this PR?

• Numpy 2.0+ requirement
• Pandas 2.2.2+ requirement (first wheel compatible with numpy 2.0)
• Pyodide 0.27.1 usage (numpy 2.0 api)

Are there any user-facing changes?

Yes, in contrast to pyarrow 25, after merging this PR pyarrow 26 can't be used with Numpy 1.x or pandas 2.0-2.2.1

This PR includes breaking changes to public APIs.

This PR was created using GPT-5.6-Sol-xhigh, every line read & reviewed by me.

• GitHub Issue: [Python] Require numpy 2.x #48473

[github-actions]

⚠️ GitHub issue #48473 has been automatically assigned in GitHub to PR creator.

[alippai]

@tadeja @rok @raulcd @jorisvandenbossche this is ready for review. The failure seems to be an unrelated arrow flight timeout. Once this is on master, I'll file a new StringDType from/to conversion PR.

[kou]

@github-actions crossbow submit -g python

[github-actions]

Revision: d0a51cc

Submitted crossbow builds: ursacomputing/crossbow @ actions-764bbcb6f5

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hdfs-2.9.2
test-conda-python-3.11-hdfs-3.2.1
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-2.2.2-numpy-2.0.2
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-pandas-latest-numpy-latest
test-conda-python-3.13
test-conda-python-3.13-pandas-latest-numpy-2.1.0
test-conda-python-3.13-pandas-latest-numpy-latest
test-conda-python-3.14
test-conda-python-3.14-cpython-debug
test-conda-python-3.14-pandas-nightly-numpy-nightly
test-conda-python-3.14-pandas-upstream_devel-numpy-nightly
test-conda-python-emscripten
test-debian-13-python-3-amd64
test-debian-13-python-3-i386
test-fedora-42-python-3
test-ubuntu-24.04-python-3

[raulcd]

Thanks @alippai for working on this. Changes LGTM in general, let me lauch extra CI and get the ok from @pitrou @jorisvandenbossche @AlenkaF

[raulcd]

@github-actions crossbow submit wheel-cp313

[github-actions]

Revision: d0a51cc

Submitted crossbow builds: ursacomputing/crossbow @ actions-449ef0ee7b

Task	Status
wheel-macos-monterey-cp313-cp313-amd64
wheel-macos-monterey-cp313-cp313-arm64
wheel-manylinux-2-28-cp313-cp313-amd64
wheel-manylinux-2-28-cp313-cp313-arm64
wheel-musllinux-1-2-cp313-cp313-amd64
wheel-musllinux-1-2-cp313-cp313-arm64
wheel-windows-cp313-cp313-amd64

[raulcd]

@github-actions crossbow submit wheel-cp311

[github-actions]

Revision: d0a51cc

Submitted crossbow builds: ursacomputing/crossbow @ actions-012e7c747c

Task	Status
wheel-macos-monterey-cp311-cp311-amd64
wheel-macos-monterey-cp311-cp311-arm64
wheel-manylinux-2-28-cp311-cp311-amd64
wheel-manylinux-2-28-cp311-cp311-arm64
wheel-musllinux-1-2-cp311-cp311-amd64
wheel-musllinux-1-2-cp311-cp311-arm64
wheel-windows-cp311-cp311-amd64

[rok]

Thanks for working on this @alippai !

I found some other places where version checks occur 1, 2, 3, 4, 5. Could you take a look and see if we can simplify/remove those?

[ngoldbaum]

I asked an AI model to review this with an eye toward code that can be deleted to simplify interfacing with Pandas and it found several more places where you can delete or simplify code. I'm happy to take on these cleanups in a followup if you think my suggestions are scope creep.

[alippai]

@ngoldbaum @rok thanks, I’ll address the pandas upgrade too. It was addressing the numpy upgrade only. I believe pandas improvements are in scope for this MR. Addressing upstream fixes and even onboarding the optimal/newer APIs allowed by the new version should land together.

[ngoldbaum]

Just one last nit but this otherwise looks good to me!

[jorisvandenbossche]

Looks good to me!