{"record":{"id":"07711c7c5a3d799e","repo":"pandas-dev/pandas","slug":"cannot-setitem-on-a-categorical-with-a-new-categor-07711c","errorCode":null,"errorMessage":"Cannot setitem on a Categorical with a new category, set the categories first","messagePattern":"Cannot setitem on a Categorical with a new category, set the categories first","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"pandas/core/arrays/categorical.py","lineNumber":2483,"sourceCode":"                raise TypeError(\n                    \"Cannot set a Categorical with another, \"\n                    \"without identical categories\"\n                )\n            # dtype equality implies categories_match_up_to_permutation\n            value = self._encode_with_my_categories(value)\n            return value._codes\n\n        from pandas import Index\n\n        # tupleize_cols=False for e.g. test_fillna_iterable_category GH#41914\n        to_add = Index._with_infer(value, tupleize_cols=False, copy=False).difference(\n            self.categories\n        )\n\n        # no assignments of values not in categories, but it's always ok to set\n        # something to np.nan\n        if len(to_add) and not isna(to_add).all():\n            raise TypeError(\n                \"Cannot setitem on a Categorical with a new \"\n                \"category, set the categories first\"\n            )\n\n        codes = self.categories.get_indexer(value)\n        return codes.astype(self._ndarray.dtype, copy=False)\n\n    def _reverse_indexer(self) -> dict[Hashable, npt.NDArray[np.intp]]:\n        \"\"\"\n        Compute the inverse of a categorical, returning\n        a dict of categories -> indexers.\n\n        *This is an internal function*\n\n        Returns\n        -------\n        Dict[Hashable, np.ndarray[np.intp]]\n            dict of categories -> indexers","sourceCodeStart":2465,"sourceCodeEnd":2501,"githubUrl":"https://github.com/pandas-dev/pandas/blob/71959b8cb9b2459c16e14b34f28b178ccfe14735/pandas/core/arrays/categorical.py#L2465-L2501","documentation":"Raised by Categorical._validate_listlike when assigning a list-like value that contains elements not present in the existing categories (and not NaN). Categorical is a closed set: new labels cannot be introduced by assignment, only by explicitly expanding the category set. NaN is always allowed because missing values do not extend the category domain.","triggerScenarios":"df.loc[i, 'cat_col'] = 'new_label' where 'new_label' is not a category; s[s>0] = [list containing a new value]; fillna with a value not in the category set.","commonSituations":"Incremental data loads introducing new labels into a column typed as category; user input that contains a value not seen during initial category inference; or default category inference on a training slice that misses values present at production time.","solutions":["Extend the categories first: s = s.cat.add_categories(['new_label']) then assign.","Recreate the Categorical with pd.Categorical(data, categories=[...all expected labels...]) covering the full value domain.","Assign np.nan instead of a new label, or filter out unseen values before assignment."],"exampleFix":"// before\ns = pd.Series(pd.Categorical(['a','b'], categories=['a','b']))\ns.iloc[0] = 'c'  # TypeError: Cannot setitem with a new category\n\n// after\ns = s.cat.add_categories(['c'])\ns.iloc[0] = 'c'","handlingStrategy":"validation","validationCode":"def extend_for_new_labels(s, labels):\n    import pandas as pd\n    import numpy as np\n    existing = set(s.cat.categories)\n    new = {l for l in labels if not (l is None or (isinstance(l, float) and pd.isna(l)))} - existing\n    if new:\n        s = s.cat.add_categories(sorted(new))\n    return s","typeGuard":"import pandas as pd\nfrom typing import Any\n\ndef all_values_in_categories(s: Any, values: Any) -> bool:\n    if not isinstance(s.dtype, pd.CategoricalDtype):\n        return True\n    cats = set(s.cat.categories)\n    return all(v in cats or pd.isna(v) for v in values)","tryCatchPattern":"try:\n    s.iloc[i] = new_label\nexcept TypeError as e:\n    if 'new category' in str(e):\n        s = s.cat.add_categories([new_label])\n        s.iloc[i] = new_label\n    else:\n        raise","preventionTips":["Pre-declare all expected categories when creating the Categorical.","Run an add_categories step before assigning externally-sourced labels."],"tags":["categorical","setitem","categories"],"analyzedSha":"71959b8cb9b2459c16e14b34f28b178ccfe14735","analyzedAt":"2026-08-07T01:30:20.476Z","schemaVersion":2},"datasetVersion":"2026-08-07T03:17:09.362Z"}