pandas-dev/pandas · error · ValueError

ArrowStringArray requires a PyArrow (chunked) array of large

Error message

ArrowStringArray requires a PyArrow (chunked) array of large_string type

What it means

Raised in ArrowStringArray.__init__ after the super().__init__ call if the backing pyarrow array's type is not pa.large_string(). ArrowStringArray normalizes string/string_view/dictionary-of-string inputs to large_string (line 151), but if a non-string pyarrow type (e.g. int32, binary, or a fixed-size binary) is passed, the cast is skipped and the post-condition check at line 159 fails with ValueError. This protects the invariant that the underlying buffer is large_string-typed, which the string kernels depend on.

Source

Thrown at pandas/core/arrays/string_arrow.py:160

            or (
                pa.types.is_dictionary(values.type)
                and (
                    pa.types.is_string(values.type.value_type)
                    or pa.types.is_large_string(values.type.value_type)
                    or _is_string_view(values.type.value_type)
                )
            )
        ):
            values = pc.cast(values, pa.large_string())

        super().__init__(values)

        if dtype is None:
            dtype = StringDtype(storage="pyarrow", na_value=libmissing.NA)
        self._dtype = dtype

        if not pa.types.is_large_string(self._pa_array.type):
            raise ValueError(
                "ArrowStringArray requires a PyArrow (chunked) array of "
                "large_string type"
            )

    def _from_pyarrow_array(self, pa_array):
        """
        Construct from a pyarrow Array/ChunkedArray result of an operation.

        Avoids full __init__ overhead (type checking, pc.cast, ArrowDtype
        construction, etc.).
        """
        assert isinstance(pa_array, (pa.Array, pa.ChunkedArray))
        if not pa.types.is_large_string(pa_array.type):
            pa_array = pa_array.cast(pa.large_string())
        obj = type(self).__new__(type(self))
        if isinstance(pa_array, pa.Array):
            pa_array = pa.chunked_array([pa_array])
        obj._pa_array = pa_array

View on GitHub (pinned to 71959b8cb9)

Solutions

1. Cast the pyarrow array to large_string before construction: `pa_arr = pa_arr.cast(pa.large_string())`.
2. If the source is binary, decode first: `pc.cast(pa_arr, pa.large_string(), safe=False)` after confirming it is valid UTF-8.
3. Prefer the public pd.array(data, dtype='string[pyarrow]') constructor, which handles conversion, instead of ArrowStringArray(pa_arr) directly.
4. Verify the input type with `pa_arr.type` before passing it in.

Example fix

# before
pa_arr = pa.array([b'a', b'b'], type=pa.binary())
ArrowStringArray(pa_arr)  # ValueError
# after
pa_arr = pa_arr.cast(pa.large_string())
ArrowStringArray(pa_arr)

import pyarrow as pa

def to_arrow_string_array(values):
    if not pa.types.is_large_string(values.type):
        if pa.types.is_string(values.type) or pa.types.is_binary(values.type):
            values = values.cast(pa.large_string())
        else:
            raise ValueError(f'Unsupported pyarrow type: {values.type}')
    return values

import pyarrow as pa
def is_large_string_array(arr) -> bool:
    return pa.types.is_large_string(getattr(arr, 'type', None))

try:
    from pandas.core.arrays.string_arrow import ArrowStringArray
    ArrowStringArray(pa_arr)
except ValueError as e:
    if 'large_string' in str(e):
        pa_arr = pa_arr.cast(pa.large_string())
    else:
        raise

When it happens

Trigger: Constructing ArrowStringArray directly from a pa.array of non-string type (e.g. pa.array([1,2,3]) or pa.array([b'a'], type=pa.binary())), or passing a ChunkedArray whose type survived the cast-skip branch. The check fires because pa.types.is_large_string(self._pa_array.type) is False.

Common situations: Manually wrapping a pyarrow array produced by compute kernels that changed type; passing binary() data expecting automatic utf8 decoding; interop code that assumes any pa.Array is acceptable.

Related errors

• DateOffset {other} is intra-day and cannot be applied to dat
• pyarrow>={PYARROW_MIN_VERSION} is required for PyArrow backe
• Values resolution does not match dtype.
• Value must be an instance of {type_repr}
• Value must be one of {pp_values}

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/e7d2aced5edb1329. Report an issue: GitHub.