The first architecture experiment keeps the packed and word-aligned storage formats
and moves selected operations onto common C kernels. Logical uint8/bool dtype
remains independent of physical bit width. No domain encoding is introduced.
Native Array.sum() ──────────────────┐
├── width/layout-specific C reduction
Array API sum(axis=None) ── C view ───┘
Array API conversion ────── C view ───── unpack only selected values
Array API assignment ───── C view ───── validate, then scatter in C
_view.h validates (shape, element_strides, element_offset) against the native
array before reading or writing. It supports scalar, empty, negative-stride,
noncontiguous, and repeated-element views, up to 64 dimensions. Footprint and
shape arithmetic use wide intermediates to reject overflow and invalid bounds.
An empty view may carry an offset outside storage, but never dereferences it.
These descriptors are currently parsed per bulk operation. Array API now uses a GC-aware C view base and a C shared storage cell, with direct scalar slots. Shape and stride tuples remain Python objects; this is not a complete native ND executor. Native Matrix/Ragged storage continues to use its existing flat array plus row metadata.
Scalar Array API reads address one packed element directly. Basic assignment snapshots and casts only the selected values before mutating storage, preserving overlap semantics and preventing partial writes on validation failure. Existing Array API views follow a width increase through the shared storage cell. Width growth repacks the root with at most a 1 KiB stack buffer, or expands directly into the new allocation for 8-bit output. It remains an O(n) path without an unpacked root-sized temporary allocation. Advanced indexing continues through the NumPy fallback.
The first implementation decoded 1,024-byte stack blocks and summed with NEON. This removed whole-array allocation, but its 5-bit packed sum still took about 11.9 µs for 65,536 elements. A second implementation folded adjacent bit fields inside a 64-bit word and improved that to about 9.2 µs (aligned: 3.2 µs).
For width b, each word is a sequence of base-2**b digits. Masks and shifts
combine adjacent digit pairs into wider slots, repeatedly, until their sum is
in one slot. The widths are compile-time constants so the compiler simplifies
masks, divisions and boundary cases. Unused high lanes are masked out.
The final dispatcher uses:
- 1 bit: existing optimized population-count kernel.
- 2–7 bits: word-level digit folding. Aligned layouts consume physical words.
- Packed 3 bit: complete logical lane groups, including cross-word extraction.
- Packed 5/6/7 bit: eight byte-aligned logical values per load where bounds permit, with safe prefix/tail handling. This experiment helped those widths but slowed 3-bit input, so it is not applied to 3-bit input.
- 8 bit: NEON byte reduction, or the portable C fallback.
No reduction path materializes a decoded array. Strided reduction specializes readers by width/layout, avoiding a function-pointer call and coordinate division per element. ND reduction traverses outer coordinates once per inner run.
The initial generic strided implementation regressed the 5-bit stride-3 workload from roughly 25 µs to 42 µs. Specialized readers restored it to roughly 21 µs; NumPy still wins that workload. Exact final measurements and reproduction commands are in the performance report.
Regression tests cover all eight widths and both layouts, shifted roots, word and block boundaries, random values, reversed/transposed views, repeated strides, empty/scalar descriptors, invalid bounds, atomic validation, shared-view widening, and overlapping assignment. The official Array API suite is retained unchanged apart from the existing narrow DLPack xfail and flaky-marker removal policy.
This stage does not change default layout, add a bitplane format, release the GIL, or promise NumPy zero-copy representation. The C view/storage and bounded-memory widening experiments are now implemented; see their measurements and remaining boundaries. Next useful experiments are cached numeric ND descriptors and strided/axis kernels. Blocked packing or bitplanes should be compared as separate storage experiments with random access, scanning, and conversion costs included; current measurements do not establish that either would be better.
Axis sums now write into a caller-owned uint64 result without fully unpacking the input. Column tiles and batches of short rows share the unpackers; other layouts use reduction runs. Small strides have fixed-mask kernels, with generic fallbacks for other offsets and steps. See measurements and scope.
Column tiles now accumulate in uint16 or uint32, using the bit-width maximum to bound each batch before flushing into uint64 output. Scratch space is at most 1 KiB decoded values plus 4 KiB accumulators, in addition to view descriptors. Row tiles use measured width/layout limits and preserve aligned word-folding paths at selected crossovers. See tile measurements.