[Python] Converting lists of string_view, binary_view, decimal32, decimal64 or fixed_size_binary to pandas raises NotImplementedError #55298

Activity

1. github-actions

   added

   Component: Python

   on Oct 28, 2026
2. hamarshehmhmd commented on Oct 28, 2026

   

   hamarshehmhmd

   on Oct 28, 2026

   More actions

   I can reproduce this on pyarrow 26.0.0 (pip wheel). All five value types fail inside a list with ArrowNotImplementedError: Not implemented type for Arrow list to pandas: ..., both through Array.to_pandas() and through Table.to_pandas(). Here is a workaround until the ListTypeSupported fix ships.

   Cast the nested value types to their supported equivalents first. The pandas result is identical: object arrays of str/bytes/Decimal, with the same nulls.

   import pyarrow as pa
   import pyarrow.types as pt
   
   def _compat(t):
       if pt.is_string_view(t):
           return pa.large_string()
       if pt.is_binary_view(t) or pt.is_fixed_size_binary(t):
           return pa.large_binary()
       if pt.is_decimal32(t) or pt.is_decimal64(t):
           return pa.decimal128(t.precision, t.scale)
       if pt.is_list(t) or pt.is_large_list(t):
           v = t.value_field
           return (pa.list_ if pt.is_list(t) else pa.large_list)(v.with_type(_compat(v.type)))
       if pt.is_struct(t):
           return pa.struct([f.with_type(_compat(f.type)) for f in t])
       return t
   
   def pandas_compatible(tbl: pa.Table) -> pa.Table:
       """Cast only the nested columns that to_pandas() can't handle yet."""
       schema = pa.schema(
           [f.with_type(_compat(f.type)) if pt.is_nested(f.type) else f for f in tbl.schema],
           metadata=tbl.schema.metadata,
       )
       return tbl.cast(schema)
   
   df = pandas_compatible(tbl).to_pandas()

   It only rewrites nested columns (lists, including lists inside structs), so top-level string_view columns still go through the normal conversion. I tested it on a table with list<string_view>, large_list<binary_view>, list<decimal32(5, 1)>, list<fixed_size_binary[2]> and struct<x: list<string_view>> columns. to_pandas() raises on the original table, and after the cast it returns the same values, nulls included, as the string/binary/decimal128 versions. For a single array, arr.cast(pa.list_(pa.string())).to_pandas() is enough.
3. github-actions

   assigned

   555aaditya

   on Oct 28, 2026