pandas-dev/pandas · error · ValueError

ArrowStringArray requires a PyArrow (chunked) array of…

Error message

ArrowStringArray requires a PyArrow (chunked) array of large_string type

What it means

ArrowStringArray.__init__ requires its underlying pyarrow array to be large_string type; if after attempted casting the array is still not pa.large_string, it raises ValueError. Internally pandas promotes string to large_string for compatibility, but a non-castable incoming type (e.g. binary, dictionary-encoded, or fixed-size binary) hits this guard.

Solutions

1. Cast the array explicitly: pa.chunked_array(arr).cast(pa.large_string()).
2. Construct via pd.array(list_or_series, dtype='string[pyarrow]') so pandas handles promotion.
3. If the source is binary, decode first: arr.cast(pa.string()).cast(pa.large_string()).
4. Use ArrowDtype(pa.string()) instead of forcing ArrowStringArray for non-large_string data.

Example fix

// before
arr = pa.array([b'a', b'b'], type=pa.binary())
ArrowStringArray(arr)  # ValueError
// after
arr = arr.cast(pa.large_string())
ArrowStringArray(arr)

def to_arrow_string_array(values):
    import pyarrow as pa
    if not pa.types.is_large_string(values.type):
        values = values.cast(pa.large_string())
    return values

def is_large_string_array(arr) -> bool:
    import pyarrow as pa
    return pa.types.is_large_string(arr.type)

try:
    ArrowStringArray(arr)
except ValueError as e:
    if 'large_string' in str(e):
        ArrowStringArray(arr.cast(pa.large_string()))
    else:
        raise

When it happens

Trigger: Constructing ArrowStringArray with a pa.string() that failed promotion, or passing pa.binary(), pa.dictionary(...), or a chunked array of mismatched string types; calling StringDtype('pyarrow').__from_arrow__(arr) with an incompatible array.

Common situations: Loading parquet whose column is binary/utf8 with a non-default encoding; converting from a pyarrow Table column that was cast to dictionary type; manually building ArrowStringArray from a computed pa.array.

Related errors

• bad operand type for unary +
• Cannot perform reduction
• Invalid value for dtype 'str'. Value should be a string or…
• Invalid value ' ' for dtype 'str'. Value should be a string…
• Invalid value ' ' for dtype 'str'. Value should be a string…

AI-assisted analysis of pandas-dev/pandas@3b7651241d (2026-08-11). Data as JSON: /api/errors/e7d2aced5edb1329. Report an issue: GitHub.

Appendix: source

Thrown at pandas/core/arrays/string_arrow.py:166

            or (
                pa.types.is_dictionary(values.type)
                and (
                    pa.types.is_string(values.type.value_type)
                    or pa.types.is_large_string(values.type.value_type)
                    or _is_string_view(values.type.value_type)
                )
            )
        ):
            values = pc.cast(values, pa.large_string())

        super().__init__(values)

        if dtype is None:
            dtype = StringDtype(storage="pyarrow", na_value=libmissing.NA)
        self._dtype = dtype

        if not pa.types.is_large_string(self._pa_array.type):
            raise ValueError(
                "ArrowStringArray requires a PyArrow (chunked) array of "
                "large_string type"
            )

    def _from_pyarrow_array(self, pa_array):
        """
        Construct from a pyarrow Array/ChunkedArray result of an operation.

        Avoids full __init__ overhead (type checking, pc.cast, ArrowDtype
        construction, etc.).
        """
        assert isinstance(pa_array, (pa.Array, pa.ChunkedArray))
        if not pa.types.is_large_string(pa_array.type):
            pa_array = pa_array.cast(pa.large_string())
        obj = type(self).__new__(type(self))
        if isinstance(pa_array, pa.Array):
            pa_array = pa.chunked_array([pa_array])
        obj._pa_array = pa_array

View on GitHub (pinned to 3b7651241d)