Repository navigation
GH-54920: [C++][Compute] Support unsigned indices in inverse_permutation and scatter - #55169
zanmato1984 wants to merge 1 commit into
Conversation
…atter Signed-off-by: Rossi Sun <zanmato1984@gmail.com>
|
|
github-actions
Bot
added
Component: C++
Component: Python
Component: Documentation
awaiting review
There was a problem hiding this comment.
🟢 Approval recommended
Static review found no blocking issues in dispatch, bounds checks, or validity generation, with added tests covering key boundary cases.
No open issues 🎉
What changed in this PR
Adds unsigned integer support to C++ compute functions inverse_permutation and scatter, addressing GH-54920 while reducing generated code size.
Changes:
- Accept signed and unsigned indices, and allow unsigned inverse-permutation outputs.
- Share input processing and validity generation by integer width while preserving bounds checks.
- Expand boundary, sliced-array, and chunked-array tests and update public documentation.
| File | Description |
|---|---|
python/pyarrow/_compute.pyx |
Document unsigned inverse-permutation output support. |
docs/source/cpp/compute.rst |
List integer input and output support. |
cpp/src/arrow/compute/kernels/vector_swizzle.cc |
Add unsigned support and consolidate kernel implementations. |
cpp/src/arrow/compute/kernels/vector_swizzle_test.cc |
Expand integer-type and boundary-case coverage. |
cpp/src/arrow/compute/api_vector.h |
Document signed and unsigned integer support. |
Review effort: Balanced
| } | ||
|
|
||
| TEST(InversePermutation, DenseValidityBitmapBoundaries) { | ||
| for (int64_t length : {7, 8, 9, 63, 64, 65}) { |
There was a problem hiding this comment.
Can you explain what this function does? Perhaps by giving an example.
| for (const auto& type : IntTypes()) { | ||
| auto indices = ArrayFromJSON(type, "[42, 2, null, 0, 1, 42]")->Slice(1, 4); | ||
| auto expected = ArrayFromJSON(type, "[2, 3, 0, null]"); | ||
| AssertInversePermutation(indices, -1, type, expected, false); |
There was a problem hiding this comment.
Can we make function parameters more explicit?
| AssertInversePermutation(indices, -1, type, expected, false); | |
| AssertInversePermutation(indices, /*xxx=*/ -1, type, expected, /*yyy=*/ false); |
| Datum(ChunkedArrayFromJSON(type, {"[]", "[" + invalid_index + "]"}))}) { | ||
| ASSERT_RAISES_WITH_MESSAGE( | ||
| IndexError, "Index error: Index out of bounds: " + invalid_index, | ||
| InversePermutation(indices, /*max_index=*/1, /*output_type=*/int64())); |
There was a problem hiding this comment.
Perhaps max_index=41 to make sure that boundary cases are treated properly?
| // Keep it separate from the indexed writes, and generate whole bitmap bytes | ||
| // instead of clearing one bit at a time for every missing destination. | ||
| template <typename OutputCType> | ||
| ARROW_NOINLINE Status InversePermutationValidity(KernelContext* ctx, |
There was a problem hiding this comment.
Return a Result<std::shared_ptr<Buffer>> here?
| } | ||
| ARROW_ASSIGN_OR_RAISE(*validity_buf, ctx->AllocateBitmap(output_length)); | ||
| auto* validity = (*validity_buf)->mutable_data(); | ||
| bit_util::SetBitmap(validity, 0, offset); |
There was a problem hiding this comment.
AllocateBitmap should already zero-initialize the bitmap, so I don't think this is necessary.
| // Avoid per-block validity dispatch when every input index is valid. | ||
| if (!span.MayHaveNulls()) { |
There was a problem hiding this comment.
Instead of special-casing this, perhaps rewrite indices visiting using VisitBitRuns?
| {InputType(match::Integer())}, OutputType(ResolveInversePermutationOutputType)); | ||
| kernel.init = InversePermutationState::Init; | ||
| kernel.exec = [](KernelContext* ctx, const ExecSpan& span, ExecResult* result) { | ||
| const auto exec = GenerateInteger<UnsignedInversePermutation, void, ArrayKernelExec>( |
There was a problem hiding this comment.
Sounds almost like we would like a GenerateInteger that takes a transformation function for the input type?
github-actions
Bot
added
awaiting committer review
Rationale for this change
inverse_permutationandscattercurrently accept only signed integer indices, requiring unnecessary casts for unsigned arrays. This PR adds unsigned support while reducing generated code size.What changes are included in this PR?
Are these changes tested?
Added unit tests for unsigned types, index bounds, sliced/chunked inputs, and validity bitmap boundaries.
Are there any user-facing changes?
Unsigned integer indices are now accepted. For example, inverse permutation of uint8
[2, null, 0, 1]returns uint8[2, 3, 0, null].No breaking API changes.
Was AI used for this PR?
The changes were thoroughly reviewed by a human with AI assistance.
PR code and description written by:
Reviewed before submission by:
inverse_permutationandscattershould accept unsigned integers #54920