pandas-dev/pandas · error · TypeError

Invalid value '{value!s}' for dtype '{self.dtype}'

Error message

Invalid value '{value!s}' for dtype '{self.dtype}'

What it means

Raised by ArrowExtensionArray.fillna when the fill value cannot be boxed into the array's pyarrow type (pyarrow raises ArrowTypeError). pandas re-raises it as a TypeError so callers get a clear 'invalid value for dtype' message instead of a low-level pyarrow error.

Source

Thrown at pandas/core/arrays/arrow/array.py:1715

            )

        if limit is not None:
            return super().fillna(value=value, limit=limit, copy=copy)

        if isinstance(value, (np.ndarray, ExtensionArray)):
            # Similar to check_value_size, but we do not mask here since we may
            #  end up passing it to the super() method.
            if len(value) != len(self):
                raise ValueError(
                    f"Length of 'value' does not match. Got ({len(value)}) "
                    f" expected {len(self)}"
                )

        try:
            fill_value = self._box_pa(value, pa_type=self._pa_array.type)
        except pa.ArrowTypeError as err:
            msg = f"Invalid value '{value!s}' for dtype '{self.dtype}'"
            raise TypeError(msg) from err

        try:
            return self._from_pyarrow_array(
                _safe_fill_null(self._pa_array, fill_value=fill_value)
            )
        except pa.ArrowNotImplementedError:
            # ArrowNotImplementedError: Function 'coalesce' has no kernel
            #   matching input types (duration[ns], duration[ns])
            # TODO: remove try/except wrapper if/when pyarrow implements
            #   a kernel for duration types.
            pass

        return super().fillna(value=value, limit=limit, copy=copy)

    def isin(self, values: ArrayLike) -> npt.NDArray[np.bool_]:
        # short-circuit to return all False array.
        if not len(values):
            return np.zeros(len(self), dtype=bool)

View on GitHub (pinned to 71959b8cb9)

Solutions

  1. Cast the fill value to the array's pyarrow type explicitly before calling fillna: `value = pa.scalar(value, type=arr.dtype.pyarrow_dtype)`.
  2. Use a value that matches the dtype's native Python representation (e.g. `pd.Timestamp` for timestamp arrays, `datetime.date` for date arrays).
  3. If the array dtype is wrong, convert it with `.astype(...)` before filling.

Example fix

// before
s = pd.Series([1, None], dtype="timestamp[us][pyarrow]")
s.fillna("2020-01-01")

// after
s.fillna(pd.Timestamp("2020-01-01"))
Defensive patterns

Strategy: validation

Validate before calling

import pyarrow as pa

def to_arrow_scalar(value, dtype):
    pa_type = dtype.pyarrow_dtype if hasattr(dtype, "pyarrow_dtype") else None
    try:
        return pa.scalar(value, type=pa_type) if pa_type else pa.scalar(value)
    except (pa.ArrowTypeError, pa.ArrowInvalid):
        raise TypeError(f"value {value!r} not valid for dtype {dtype}")

Type guard

def is_valid_for_dtype(value, dtype) -> bool:
    import pyarrow as pa
    pa_type = getattr(dtype, "pyarrow_dtype", None)
    try:
        pa.scalar(value, type=pa_type) if pa_type else pa.scalar(value)
        return True
    except (pa.ArrowTypeError, pa.ArrowInvalid):
        return False

Try / catch

try:
    arr.fillna(value)
except TypeError as e:
    if "Invalid value" in str(e) and "for dtype" in str(e):
        arr.fillna(arr.dtype.na_value)  # fall back to native NA
    else:
        raise

Prevention

When it happens

Trigger: Calling `fillna(value)` on an ArrowExtensionArray where `value` is not convertible to `self._pa_array.type` — e.g. filling a `timestamp[us][pyarrow]` array with a plain string, or a `int32[pyarrow]` array with a float like 1.5 that would truncate.

Common situations: Loading data from JSON/CSV where fill constants come in as strings, mixing Python types across dtype migrations (e.g. default ints vs floats), or passing `pd.NA`/`None` where a concrete scalar is required.

Related errors

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/92863d49a1049e80. Report an issue: GitHub.