pandas-dev/pandas · error · TypeError

Expected array of {self} type, got {array.type} instead

Error message

Expected array of {self} type, got {array.type} instead

What it means

Raised by NumericDtype.__from_arrow__ when constructing IntegerArray/FloatingArray from a pyarrow Array/ChunkedArray whose type, after round-trip to pandas dtype, is not an integer/unsigned/float kind (not in 'iuf'). pandas can convert itemsize but refuses genuinely incompatible types (e.g. strings) rather than producing invalid data (GH#31896 context).

Source

Thrown at pandas/core/arrays/numeric.py:91

        import pyarrow

        from pandas.core.arrays.arrow._arrow_utils import (
            pyarrow_array_to_numpy_and_mask,
        )

        array_class = self.construct_array_type()

        pyarrow_type = pyarrow.from_numpy_dtype(self.type)
        if not array.type.equals(pyarrow_type) and not pyarrow.types.is_null(
            array.type
        ):
            # test_from_arrow_type_error raise for string, but allow
            #  through itemsize conversion GH#31896
            rt_dtype = pandas_dtype(array.type.to_pandas_dtype())
            if rt_dtype.kind not in "iuf":
                # Could allow "c" or potentially disallow float<->int conversion,
                #  but at the moment we specifically test that uint<->int works
                raise TypeError(
                    f"Expected array of {self} type, got {array.type} instead"
                )

            array = array.cast(pyarrow_type)

        if isinstance(array, pyarrow.ChunkedArray):
            array = array.combine_chunks()

        data, mask = pyarrow_array_to_numpy_and_mask(array, dtype=self.numpy_dtype)
        if data.dtype.kind == "f" and is_nan_na():
            mask[np.isnan(data)] = False
        return array_class(data.copy(), ~mask, copy=False)

    @classmethod
    def _get_dtype_mapping(cls) -> Mapping[np.dtype, NumericDtype]:
        raise AbstractMethodError(cls)

    @classmethod

View on GitHub (pinned to 71959b8cb9)

Solutions

1. Cast the pyarrow array to a compatible type first: arr.cast(pa.int64()) before constructing the masked array.
2. Parse/convert the source data to numeric before building the arrow array.
3. Target the dtype matching the arrow type, or use object/string dtype explicitly.

Example fix

// before
pa.array(['1','2'], type=pa.string())  # -> from_arrow(Int64) raises

// after
pa.array(['1','2'], type=pa.string()).cast(pa.int32())  # then from_arrow(Int32)

import pyarrow as pa

def arrow_to_masked_numeric(array, target_dtype_cls):
    pa_type = pa.from_numpy_dtype(target_dtype_cls.type)
    if not (array.type.equals(pa_type) or pa.types.is_null(array.type)):
        rt = array.type.to_pandas_dtype()
        import numpy as np
        if np.dtype(rt).kind not in "iuf":
            raise TypeError(f"incompatible arrow type {array.type}")
        array = array.cast(pa_type)
    return target_dtype_cls.__from_arrow__(array)

def arrow_type_is_numeric_compat(array) -> bool:
    import pyarrow as pa, numpy as np
    try:
        return np.dtype(array.type.to_pandas_dtype()).kind in "iuf"
    except Exception:
        return False

try:
    arr = dtype_cls.__from_arrow__(pa_array)
except TypeError as e:
    if "Expected array of" in str(e):
        import pyarrow as pa
        arr = dtype_cls.__from_arrow__(pa_array.cast(pa.from_numpy_dtype(dtype_cls.type)))
    else:
        raise

When it happens

Trigger: Calling pa.Table.from_pandas / pd.array(arr, dtype='Int64') with a pyarrow string/binary/list array; converting a pyarrow table whose column is string into a nullable Int64 column directly via from_arrow.

Common situations: Mixing pyarrow and pandas dtypes; reading arrow data whose logical type does not match the target masked numeric dtype; assuming pyarrow will silently cast strings to ints.

Related errors

• Can only use the '.list' accessor with 'list[pyarrow]' dtype
• Can only use the '.struct' accessor with 'struct[pyarrow]' d
• operation '{name}' not supported for dtype '{self.dtype}'
• invalid dtype specified {dtype}
• values should be {descr} numpy array. Use the 'pd.array' fun

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/60f4515bc9f2e78b. Report an issue: GitHub.