GH-41670: [C++][Python] Move to DLPack 1.3

[AntoinePrv]

Rationale for this change

Pure version bump to enable implementing more features.

What changes are included in this PR?

• Change the DLPack header file and definition.
• Use raw_data for Tensor similar to Array (instead of mutable_raw_data that is not available on immutable tensors).
• Strides may no longer be NULL
• Add ExportArrayVersioned and ExportTensorVersioned C APIs
• Support for copy
• Support for max_version and copy in __dlpack__

Are these changes tested?

Yes, with existing tests.

Are there any user-facing changes?

• GitHub Issue: [C++][Python] Update DLPack version #41670

[github-actions]

⚠️ GitHub issue #41670 has been automatically assigned in GitHub to PR creator.

[Copilot]

🟡 Changes recommended

Arrow’s current DLPack Array exporter still emits strides = NULL (non-compliant with DLPack v1.2+ requirements) and the new header text contains documentation mismatches that should be corrected.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

Updates Arrow’s vendored DLPack ABI header to DLPack v1.3, expanding the ABI surface (new device/type enums, flags, and the __dlpack_c_exchange_api__ protocol) while bumping the reported minor version.

Changes:

• Bump DLPack minor version to 1.3 and sync header contents to a newer upstream commit.
• Add new DLDeviceType/DLDataTypeCode values and additional DLManagedTensorVersioned flag bitmasks.
• Introduce DLPack fast exchange protocol (DLPackExchangeAPI*) type definitions and documentation.

File summaries

File	Description
cpp/src/arrow/c/dlpack_abi.h	Vendor update to DLPack 1.3 ABI definitions, enums, flags, and fast exchange protocol types.

Review details

Suppressed comments (1)

cpp/src/arrow/c/dlpack_abi.h:426

• The documentation for DLPackManagedTensorFromPyObjectNoSync claims the function returns an owning pointer/NULL, but the typedef returns int and delivers the tensor via out. The return contract should match the signature.

 * \return The owning DLManagedTensorVersioned* or NULL on failure with a
 *         Python exception set. If the data cannot be described using DLPack
 *         this should be a BufferError if possible.
 * \note - As a C function, must not thrown C++ exceptions.

• Files reviewed: 1/1 changed files
• Comments generated: 3
• Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

[Copilot]

🟡 Changes recommended

The new public versioned export APIs are not covered by tests, and there is a small but concrete maintainability issue (std::move on a const shared_ptr reference) to address.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (3)

cpp/src/arrow/c/dlpack.cc:212

• t is a const std::shared_ptr<Tensor>&, so std::move(t) won’t actually move (it will fall back to a copy via operator=(const shared_ptr&)). Using a plain copy here is clearer and avoids the misleading std::move.

  ctx->t = std::move(t);

cpp/src/arrow/c/dlpack_test.cc:49

• New public APIs ExportArrayVersioned/ExportTensorVersioned (and the version/flags fields they populate) aren’t covered by tests here; currently the tests only exercise the legacy ExportArray/ExportTensor paths. Adding a small set of assertions for version/flags on the versioned exports would prevent regressions.

  ASSERT_EQ(1, *dltensor.strides);  // Must be non-null with ndim>0 since 1.2

cpp/src/arrow/c/dlpack.h:48

• ExportTensor also returns the legacy DLManagedTensor (deprecated in DLPack in favor of DLManagedTensorVersioned), but only ExportArray is annotated as deprecated in this header. To keep the API docs consistent and steer callers to the versioned path, add the same deprecation note for ExportTensor.

ARROW_EXPORT
Result<DLManagedTensor*> ExportTensor(const std::shared_ptr<Tensor>& t);

• Files reviewed: 4/4 changed files
• Comments generated: 0 new
• Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

[Copilot]

🟡 Changes recommended

cpp/src/arrow/c/dlpack.cc has missing semicolons after ARROW_ASSIGN_OR_RAISE statements that will break compilation.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (1)

cpp/src/arrow/c/dlpack.cc:175

• Missing semicolon after ARROW_ASSIGN_OR_RAISE; as written this won’t compile because the macro expansion isn’t terminated as a statement.

  ARROW_ASSIGN_OR_RAISE(auto device, ExportDevice(t))

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

[AntoinePrv]

@AlenkaF would you mind providing early feedback before I move on to the Python side.

[Copilot]

🟡 Changes recommended

The legacy ExportTensor path now exposes immutable tensor memory without a read-only signal, which can violate Arrow’s immutability contract unless behavior/tests are adjusted accordingly.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details

Suppressed comments (2)

cpp/src/arrow/c/dlpack.h:48

• ExportTensor is also a legacy (pre-versioned) DLPack API, but only ExportArray is documented as deprecated. This makes the header inconsistent and can mislead users into using the legacy tensor export instead of the versioned API.

ARROW_EXPORT
Result<DLManagedTensor*> ExportTensor(const std::shared_ptr<Tensor>& t);

cpp/src/arrow/c/dlpack_test.cc:265

• This test currently validates exporting an immutable tensor for both legacy and versioned producers. If the legacy API is made to reject immutable tensors (since it cannot express read-only), the test should branch: expect a TypeError for LegacyProducer and continue to validate successful export + read-only flag for VersionedProducer.

  CheckDLTensor<TypeParam>(read_only_tensor, float32(), DLDataTypeCode::kDLFloat, shape,
                           dlpack_strides);
}

• Files reviewed: 4/4 changed files
• Comments generated: 1
• Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

Suppressed comments (2)

python/pyarrow/array.pxi:2297

• copy defaults to None, but the versioned export path passes it directly to ExportArrayVersionedToDLPack(..., c_bool copy). Cython cannot coerce None to c_bool, so arr.__dlpack__(max_version=(1, 0)) (or a consumer that passes copy=None) will raise a TypeError instead of exporting a capsule.

        # Currently no major version other than legacy 0 and current 1.3
        dlm_tensor = GetResultValue(ExportArrayVersionedToDLPack(self.sp_array, copy))
        return PyCapsule_New(dlm_tensor, 'dltensor_versioned', dlpack_versioned_pycapsule_deleter)

python/pyarrow/tensor.pxi:351

• copy defaults to None, but the versioned export path passes it directly to ExportTensorVersionedToDLPack(..., c_bool copy). Cython cannot coerce None to c_bool, so tensor.__dlpack__(max_version=(1, 0)) (or a consumer that passes copy=None) will raise a TypeError instead of exporting a capsule.

        # Currently no major version other than legacy 0 and current 1.3
        dlm_tensor = GetResultValue(ExportTensorVersionedToDLPack(self.sp_tensor, copy))
        return PyCapsule_New(dlm_tensor, 'dltensor_versioned', dlpack_versioned_pycapsule_deleter)

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (2)

python/pyarrow/tensor.pxi:347

• copy is documented as forcing a data copy, but for the legacy (unversioned) capsule path (max_version is None or < (1, 0)), the argument is ignored and the export always shares data. This breaks the documented contract for copy=True. Either implement copying for legacy exports or explicitly reject copy when returning an unversioned capsule (e.g. require max_version >= (1, 0) for copy=True).

        if max_version is None or max_version < (1, 0):
            # Note: from March 2025 onwards, it's okay to raise BufferError here.
            # Still we keep the V0 version that was added in August 2026.
            legacy_tensor = GetResultValue(ExportTensorToDLPack(self.sp_tensor))
            return PyCapsule_New(legacy_tensor, 'dltensor', dlpack_pycapsule_deleter)

python/pyarrow/array.pxi:2293

• copy is documented as forcing a data copy, but when exporting the legacy (unversioned) capsule (max_version is None or < (1, 0)), the argument is ignored and the export always shares data. This violates the method’s documented contract and can surprise consumers attempting to avoid sharing/mutation issues. Either implement copying for the legacy path or reject copy when returning an unversioned capsule (e.g. require max_version >= (1, 0) for copy=True).

        if max_version is None or max_version < (1, 0):
            # Note: from March 2025 onwards, it's okay to raise BufferError here.
            # Still we keep the V0 version that was added in August 2026.
            legacy_tensor = GetResultValue(ExportArrayToDLPack(self.sp_array))
            return PyCapsule_New(legacy_tensor, 'dltensor', dlpack_pycapsule_deleter)