pandas-dev/pandas · error · TypeError

Cannot setitem on a Categorical with a new category, set the

Error message

Cannot setitem on a Categorical with a new category, set the categories first

What it means

Raised by Categorical._validate_listlike when assigning a list-like value that contains elements not present in the existing categories (and not NaN). Categorical is a closed set: new labels cannot be introduced by assignment, only by explicitly expanding the category set. NaN is always allowed because missing values do not extend the category domain.

Source

Thrown at pandas/core/arrays/categorical.py:2483

                raise TypeError(
                    "Cannot set a Categorical with another, "
                    "without identical categories"
                )
            # dtype equality implies categories_match_up_to_permutation
            value = self._encode_with_my_categories(value)
            return value._codes

        from pandas import Index

        # tupleize_cols=False for e.g. test_fillna_iterable_category GH#41914
        to_add = Index._with_infer(value, tupleize_cols=False, copy=False).difference(
            self.categories
        )

        # no assignments of values not in categories, but it's always ok to set
        # something to np.nan
        if len(to_add) and not isna(to_add).all():
            raise TypeError(
                "Cannot setitem on a Categorical with a new "
                "category, set the categories first"
            )

        codes = self.categories.get_indexer(value)
        return codes.astype(self._ndarray.dtype, copy=False)

    def _reverse_indexer(self) -> dict[Hashable, npt.NDArray[np.intp]]:
        """
        Compute the inverse of a categorical, returning
        a dict of categories -> indexers.

        *This is an internal function*

        Returns
        -------
        Dict[Hashable, np.ndarray[np.intp]]
            dict of categories -> indexers

View on GitHub (pinned to 71959b8cb9)

Solutions

1. Extend the categories first: s = s.cat.add_categories(['new_label']) then assign.
2. Recreate the Categorical with pd.Categorical(data, categories=[...all expected labels...]) covering the full value domain.
3. Assign np.nan instead of a new label, or filter out unseen values before assignment.

Example fix

// before
s = pd.Series(pd.Categorical(['a','b'], categories=['a','b']))
s.iloc[0] = 'c'  # TypeError: Cannot setitem with a new category

// after
s = s.cat.add_categories(['c'])
s.iloc[0] = 'c'

def extend_for_new_labels(s, labels):
    import pandas as pd
    import numpy as np
    existing = set(s.cat.categories)
    new = {l for l in labels if not (l is None or (isinstance(l, float) and pd.isna(l)))} - existing
    if new:
        s = s.cat.add_categories(sorted(new))
    return s

import pandas as pd
from typing import Any

def all_values_in_categories(s: Any, values: Any) -> bool:
    if not isinstance(s.dtype, pd.CategoricalDtype):
        return True
    cats = set(s.cat.categories)
    return all(v in cats or pd.isna(v) for v in values)

try:
    s.iloc[i] = new_label
except TypeError as e:
    if 'new category' in str(e):
        s = s.cat.add_categories([new_label])
        s.iloc[i] = new_label
    else:
        raise

When it happens

Trigger: df.loc[i, 'cat_col'] = 'new_label' where 'new_label' is not a category; s[s>0] = [list containing a new value]; fillna with a value not in the category set.

Common situations: Incremental data loads introducing new labels into a column typed as category; user input that contains a value not seen during initial category inference; or default category inference on a training slice that misses values present at production time.

Related errors

• Cannot set a Categorical with another, without identical cat
• Cannot setitem on a Categorical with a new category ({fill_v
• index {key} is out of bounds for axis 0 with size {n}
• Length of indexer and values mismatch
• Lengths must match.

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/07711c7c5a3d799e. Report an issue: GitHub.