pandas-dev/pandas · error · TypeError

Cannot setitem on a Categorical with a new category ({fill_v

Error message

Cannot setitem on a Categorical with a new category ({fill_value}), set the categories first

What it means

Raised by _validate_scalar during item assignment when the value being set is not a valid NA for the categories' dtype and is not among the existing categories. Categoricals have a closed category set; assigning a value outside it would require a new code that does not exist, so pandas tells you to register the category first.

Source

Thrown at pandas/core/arrays/categorical.py:1722

        Parameters
        ----------
        fill_value : object

        Returns
        -------
        fill_value : int

        Raises
        ------
        TypeError
        """

        if is_valid_na_for_dtype(fill_value, self.categories.dtype):
            fill_value = -1
        elif fill_value in self.categories:
            fill_value = self._unbox_scalar(fill_value)
        else:
            raise TypeError(
                "Cannot setitem on a Categorical with a new "
                f"category ({fill_value}), set the categories first"
            ) from None
        return fill_value

    @classmethod
    def _validate_codes_for_dtype(cls, codes, *, dtype: CategoricalDtype) -> np.ndarray:
        if isinstance(codes, ExtensionArray) and is_integer_dtype(codes.dtype):
            # Avoid the implicit conversion of Int to object
            if isna(codes).any():
                raise ValueError("codes cannot contain NA values")
            codes = codes.to_numpy(dtype=np.int64)
        else:
            codes = np.asarray(codes)
        if len(codes) and codes.dtype.kind not in "iu":
            raise ValueError("codes need to be array-like integers")

        if len(codes) and (codes.max() >= len(dtype.categories) or codes.min() < -1):

View on GitHub (pinned to 71959b8cb9)

Solutions

Add the category first: `cat = cat.add_categories(['z'])` then `cat[0] = 'z'`.
Assign pd.NA/np.nan if you intend to mark missing (must be a valid NA for the dtype).
Rebuild the categorical with the full expected category set up front.

Example fix

# before
cat = pd.Categorical(['a','b'], categories=['a','b'])
cat[0] = 'c'
# after
cat = pd.Categorical(['a','b'], categories=['a','b']).add_categories(['c'])
cat[0] = 'c'

Defensive patterns

Strategy: validation

Validate before calling

def safe_cat_setitem(cat, idx, value):
    import numpy as np
    import pandas as pd
    if not (value in cat.categories or (value is pd.NA) or (isinstance(value, float) and np.isnan(value))):
        raise ValueError(f"{value!r} not a category; add it first")
    cat[idx] = value
    return cat

Type guard

def is_valid_category_value(cat, value) -> bool:
    import pandas as pd
    import numpy as np
    return value in cat.categories or value is pd.NA or (isinstance(value, float) and np.isnan(value))

Try / catch

try:
    cat[0] = value
except TypeError as e:
    if 'new category' in str(e):
        cat = cat.add_categories([value])
        cat[0] = value
    else:
        raise

Prevention

Pre-register the full expected category set at construction.
Validate assignment values against cat.categories before setting.
Use add_categories before writing novel labels.

When it happens

Trigger: `cat[0] = 'z'` where 'z' is not in cat.categories and is not NaN-compatible. Also fires through `cat.fill_value` validation and DataFrame `.loc`/`.iloc` sets on a categorical column.

Common situations: Writing new labels into a categorical column read from a fixed vocabulary; appending rows with novel enum values to a categorical DataFrame column.

Related errors

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/c13739540d9b512b. Report an issue: GitHub.