pandas-dev/pandas · error · ValueError

Cannot cast {self.categories.dtype} dtype to {dtype}

Error message

Cannot cast {self.categories.dtype} dtype to {dtype}

What it means

Raised by Categorical.astype when the categories' underlying dtype cannot be cast to the requested target dtype (the inner `new_cats.astype(dtype)` raised TypeError or ValueError). The error is re-raised with a clearer categorical-specific message pointing at the categories dtype, not the codes.

Source

Thrown at pandas/core/arrays/categorical.py:649

                result = np.array(self, dtype=dtype)

        else:
            # GH8628 (PERF): astype category codes instead of astyping array
            new_cats = self.categories._values

            try:
                new_cats = new_cats.astype(dtype=dtype, copy=copy)
                fill_value = self.categories._na_value
                if not is_valid_na_for_dtype(fill_value, dtype):
                    fill_value = lib.item_from_zerodim(
                        np.array(self.categories._na_value).astype(dtype)
                    )
            except (
                TypeError,  # downstream error msg for CategoricalIndex is misleading
                ValueError,
            ) as err:
                msg = f"Cannot cast {self.categories.dtype} dtype to {dtype}"
                raise ValueError(msg) from err

            result = take_nd(
                new_cats, ensure_platform_int(self._codes), fill_value=fill_value
            )

        return result

    @classmethod
    def _from_inferred_categories(
        cls, inferred_categories, inferred_codes, dtype, true_values=None
    ) -> Self:
        """
        Construct a Categorical from inferred values.

        For inferred categories (`dtype` is None) the categories are sorted.
        For explicit `dtype`, the `inferred_categories` are cast to the
        appropriate type.

View on GitHub (pinned to 71959b8cb9)

Solutions

1. Use `cat.codes` to get integer position encodings instead of astype(int).
2. Map categories to numeric values explicitly: `cat.map({'a':0,'b':1}).astype(int)`.
3. Cast to a compatible dtype (e.g. `astype(str)` if categories are strings).
4. Pre-convert categories: rebuild the Categorical from numeric categories.

Example fix

# before
cat = pd.Categorical(['a','b','c'])
cat.astype(int)
# after
cat = pd.Categorical(['a','b','c'])
cat.codes  # integer positions, or use cat.map(mapping)

import numpy as np

def cat_cast_or_codes(cat, dtype):
    try:
        return cat.astype(dtype)
    except ValueError:
        if dtype.kind in 'iu':
            return cat.codes
        raise

def categories_castable_to(cat, dtype) -> bool:
    try:
        cat.categories._values.astype(dtype)
        return True
    except (TypeError, ValueError):
        return False

try:
    out = cat.astype('int')
except ValueError as e:
    if 'Cannot cast' in str(e):
        out = cat.codes
    else:
        raise

When it happens

Trigger: `cat.astype('int')` where categories are non-numeric strings (e.g. ['a','b']); or `cat.astype(np.float32)` where categories are objects that fail conversion. The fallback block at line 633+ catches the inner cast failure.

Common situations: Treating string labels as numbers; converting ordinal labels to numeric codes via astype instead of `.codes`; mismatched dtypes after read_csv type inference.

Related errors

• Cannot convert float NaN to integer
• Lengths must match.
• The categories must be provided in 'categories' or 'dtype'.
• new categories need to have the same number of items as the
• items in new_categories are not the same as in old categorie

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/64581e3b8c850f8a. Report an issue: GitHub.