pandas-dev/pandas · error · TypeError

Invalid value ' ' for dtype

Error message

Invalid value '{value!s}' for dtype '{self.dtype}'

What it means

Raised by NumpyExtensionArray._validate_setitem_value when np_can_hold_element raises LossySetitemError — i.e. the value cannot be stored in the array's numpy dtype without loss of information (e.g. putting 1.5 into an int64 array, or a large int into int8). The original LossySetitemError is chained as the cause. This guards item assignment (arr[i] = v) and fill operations so silent truncation cannot occur.

Solutions

1. Cast the value to the array's dtype before assignment: int(value), or value.astype(arr.dtype.numpy_dtype).
2. Widen the array dtype to fit the value: arr = arr.astype('Int64') (or Float64) before assignment.
3. Use pd.array(..., dtype=<wider>) at construction so the value is representable.

Example fix

# before
arr = pd.array([1, 2, 3], dtype='Int8')
arr[0] = 200  # raises (200 > 127)

# after
arr = arr.astype('Int16')
arr[0] = 200

import numpy as np

def safe_setitem(arr, idx, value):
    np_dtype = arr.dtype.numpy_dtype if hasattr(arr.dtype, 'numpy_dtype') else arr.dtype
    try:
        value = np.array(value).astype(np_dtype).item()
    except (TypeError, OverflowError, ValueError):
        arr = arr.astype('Float64')
    arr[idx] = value
    return arr

import numpy as np

def value_fits_dtype(value, np_dtype) -> bool:
    try:
        np.array(value).astype(np_dtype)
        return True
    except (TypeError, OverflowError, ValueError):
        return False

try:
    arr[i] = value
except TypeError as e:
    if 'Invalid value' in str(e):
        arr = arr.astype('Float64')
        arr[i] = value
    else:
        raise

When it happens

Trigger: arr = pd.array([1, 2, 3]); arr[0] = 1.5 — float into Int64. arr[0] = 10**20 into an Int8 column. Filling an integer NumpyExtensionArray with a float value via where/fillna that routes through _validate_setitem_value.

Common situations: Conditional assignment with a value whose type doesn't fit; merging/joining columns of narrower dtype; user expects silent numpy-style truncation but pandas now validates.

Related errors

• ArrowStringArray requires a PyArrow (chunked) array of…
• Cannot cast NaN value to Integer dtype.
• cannot convert float NaN to integer
• Cannot interpolate with
• Cannot modify read-only array

AI-assisted analysis of pandas-dev/pandas@3b7651241d (2026-08-11). Data as JSON: /api/errors/9f86da8cf1dce830. Report an issue: GitHub.

Appendix: source

Thrown at pandas/core/arrays/numpy_.py:185

        if copy and result is scalars:
            result = result.copy()
        return cls(result)

    def _validate_setitem_value(self, value):
        if isinstance(value, type(self)):
            value = value._ndarray

        # Match Block._standardize_fill_value behavior
        if self._ndarray.dtype.kind != "O" and is_valid_na_for_dtype(
            value, self._ndarray.dtype
        ):
            value = self.dtype.na_value

        try:
            return np_can_hold_element(self._ndarray.dtype, value)
        except LossySetitemError as err:
            raise TypeError(
                f"Invalid value '{value!s}' for dtype '{self.dtype}'"
            ) from err
        except NotImplementedError:
            # np_can_hold_element doesn't handle all dtypes (e.g. "U"),
            # fall back to no validation for those.
            return value

    def searchsorted(
        self,
        value: NumpyValueArrayLike | ExtensionArray,
        side: Literal["left", "right"] = "left",
        sorter: NumpySorter | None = None,
    ) -> npt.NDArray[np.intp] | np.intp:
        # Parent's searchsorted calls _validate_setitem_value, which is
        # too strict for search (e.g. rejects float into int). Delegate
        # directly to numpy which handles cross-dtype searches correctly.
        return self._ndarray.searchsorted(value, side=side, sorter=sorter)  # type: ignore[arg-type]

View on GitHub (pinned to 3b7651241d)