pandas-dev/pandas · error · TypeError

Invalid value ' ' for dtype

Error message

Invalid value '{value!s}' for dtype '{self.dtype}'

What it means

Raised by ArrowExtensionArray.fillna when _box_pa (boxing `value` into a pyarrow scalar of the array's type) fails with pa.ArrowTypeError. This means the fill value is not castable to the array's dtype, e.g. filling a int64[pyarrow] array with a string, or filling a boolean array with a float like 1.5.

Solutions

  1. Cast the fill value to match the array dtype: int value for int arrays, pd.Timestamp for timestamp arrays, etc.
  2. Use pd.NA-aware construction so NA sentinels are recognized during parsing instead of fillna.
  3. For sentinel strings, replace them with pd.NA first: s.replace('', pd.NA).
  4. Validate fill value type against pa.types.* before calling fillna.

Example fix

# before
arr = pd.array([1, None, 3], dtype="int64[pyarrow]")
arr.fillna('missing')  # raises TypeError: Invalid value 'missing' for dtype 'int[pyarrow]'

# after
arr.fillna(0)
Defensive patterns

Strategy: validation

Validate before calling

import pyarrow as pa

def fill_value_for_dtype(value, arr):
    try:
        arr._box_pa(value, pa_type=arr._pa_array.type)
    except pa.ArrowTypeError as e:
        raise TypeError(f'Invalid value {value!r} for dtype {arr.dtype}') from e
    return value

Type guard

import pyarrow as pa

def is_castable_to(value, arr) -> bool:
    try:
        arr._box_pa(value, pa_type=arr._pa_array.type)
        return True
    except pa.ArrowTypeError:
        return False

Try / catch

try:
    result = arr.fillna(value)
except TypeError as e:
    if 'Invalid value' in str(e):
        # cast or pick a default matching the dtype
        result = arr.fillna(0)  # or pd.Timestamp(0), etc.
    else:
        raise

Prevention

When it happens

Trigger: arr.fillna('N/A') on an int64[pyarrow] array; boolean[pyarrow].fillna(1.5); filling a timestamp array with a plain string date; decimal[pyarrow].fillna(some_float) where the precision overflows.

Common situations: Loading data where NA sentinel strings ('', 'null') were not pre-converted; user-supplied default values typed incorrectly; filling temporal arrays with strings instead of Timestamp objects; decimal precision mismatches.

Related errors

AI-assisted analysis of pandas-dev/pandas@3b7651241d (2026-08-11). Data as JSON: /api/errors/92863d49a1049e80. Report an issue: GitHub.

Appendix: source

Thrown at pandas/core/arrays/arrow/array.py:1740

            )

        if limit is not None:
            return super().fillna(value=value, limit=limit, copy=copy)

        if isinstance(value, (np.ndarray, ExtensionArray)):
            # Similar to check_value_size, but we do not mask here since we may
            #  end up passing it to the super() method.
            if len(value) != len(self):
                raise ValueError(
                    f"Length of 'value' does not match. Got ({len(value)}) "
                    f" expected {len(self)}"
                )

        try:
            fill_value = self._box_pa(value, pa_type=self._pa_array.type)
        except pa.ArrowTypeError as err:
            msg = f"Invalid value '{value!s}' for dtype '{self.dtype}'"
            raise TypeError(msg) from err

        try:
            return self._from_pyarrow_array(
                _safe_fill_null(self._pa_array, fill_value=fill_value)
            )
        except pa.ArrowNotImplementedError:
            # ArrowNotImplementedError: Function 'coalesce' has no kernel
            #   matching input types (duration[ns], duration[ns])
            # TODO: remove try/except wrapper if/when pyarrow implements
            #   a kernel for duration types.
            pass

        return super().fillna(value=value, limit=limit, copy=copy)

    def isin(self, values: ArrayLike) -> npt.NDArray[np.bool_]:
        # short-circuit to return all False array.
        if not len(values):
            return np.zeros(len(self), dtype=bool)

View on GitHub (pinned to 3b7651241d)