pandas-dev/pandas · error · ValueError

Cannot cast dtype to

Error message

Cannot cast {self.categories.dtype} dtype to {dtype}

What it means

Raised in `Categorical.astype` when casting the underlying category values to the requested `dtype` raises `TypeError` or `ValueError` downstream. The categories themselves cannot be coerced (e.g. non-numeric strings to `int`), so the per-element `take_nd` lookup cannot proceed.

Solutions

1. Inspect `cat.categories` and clean non-coercible entries before the cast.
2. Convert via `pd.to_numeric(cat, errors='coerce')` to force unparseable values to NaN.
3. Cast categories to string first if you want a uniform object/string output: `cat.astype('str')`.
4. Pre-filter categories to only the numeric-parseable subset.

Example fix

# before
cat = pd.Categorical(['1', '2', 'x'])
arr = cat.astype('int64')  # ValueError: Cannot cast object dtype to int64

# after
import pandas as pd
arr = pd.to_numeric(pd.Series(cat), errors='coerce').to_numpy()

import pandas as pd

def safe_numeric_cast(cat):
    try:
        return cat.astype('float64')
    except ValueError:
        return pd.to_numeric(pd.Series(cat), errors='coerce').to_numpy()

def categories_are_numeric(cat) -> bool:
    import pandas as pd
    return pd.to_numeric(pd.Series(cat.categories), errors='coerce').notna().all()

try:
    arr = cat.astype('int64')
except ValueError as e:
    if 'Cannot cast' in str(e):
        import pandas as pd
        arr = pd.to_numeric(pd.Series(cat), errors='coerce').to_numpy()
    else:
        raise

When it happens

Trigger: `pd.Categorical(['a', 'b']).astype('int64')`, `pd.Categorical(['1.5', 'x']).astype('float64')`, or any cast where `self.categories.astype(dtype)` fails.

Common situations: Assuming a categorical of stringified numbers will coerce cleanly to numeric; leftover sentinel strings (`'NA'`, `'.'`) blocking numeric casts; downstream type narrowing after `read_csv` inferring object columns.

Related errors

• Cannot convert float NaN to integer
• codes cannot contain NA values
• codes need to be array-like integers
• codes need to be between -1 and len(categories)-1
• items in new_categories are not the same as in old…

AI-assisted analysis of pandas-dev/pandas@3b7651241d (2026-08-11). Data as JSON: /api/errors/64581e3b8c850f8a. Report an issue: GitHub.

Appendix: source

Thrown at pandas/core/arrays/categorical.py:649

                result = np.array(self, dtype=dtype)

        else:
            # GH8628 (PERF): astype category codes instead of astyping array
            new_cats = self.categories._values

            try:
                new_cats = new_cats.astype(dtype=dtype, copy=copy)
                fill_value = self.categories._na_value
                if not is_valid_na_for_dtype(fill_value, dtype):
                    fill_value = lib.item_from_zerodim(
                        np.array(self.categories._na_value).astype(dtype)
                    )
            except (
                TypeError,  # downstream error msg for CategoricalIndex is misleading
                ValueError,
            ) as err:
                msg = f"Cannot cast {self.categories.dtype} dtype to {dtype}"
                raise ValueError(msg) from err

            result = take_nd(
                new_cats, ensure_platform_int(self._codes), fill_value=fill_value
            )

        return result

    @classmethod
    def _from_inferred_categories(
        cls, inferred_categories, inferred_codes, dtype, true_values=None
    ) -> Self:
        """
        Construct a Categorical from inferred values.

        For inferred categories (`dtype` is None) the categories are sorted.
        For explicit `dtype`, the `inferred_categories` are cast to the
        appropriate type.

View on GitHub (pinned to 3b7651241d)