{"record":{"id":"c13739540d9b512b","repo":"pandas-dev/pandas","slug":"cannot-setitem-on-a-categorical-with-a-new-categor","errorCode":null,"errorMessage":"Cannot setitem on a Categorical with a new category ({fill_value}), set the categories first","messagePattern":"Cannot setitem on a Categorical with a new category \\((.+?)\\), set the categories first","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"pandas/core/arrays/categorical.py","lineNumber":1722,"sourceCode":"        Parameters\n        ----------\n        fill_value : object\n\n        Returns\n        -------\n        fill_value : int\n\n        Raises\n        ------\n        TypeError\n        \"\"\"\n\n        if is_valid_na_for_dtype(fill_value, self.categories.dtype):\n            fill_value = -1\n        elif fill_value in self.categories:\n            fill_value = self._unbox_scalar(fill_value)\n        else:\n            raise TypeError(\n                \"Cannot setitem on a Categorical with a new \"\n                f\"category ({fill_value}), set the categories first\"\n            ) from None\n        return fill_value\n\n    @classmethod\n    def _validate_codes_for_dtype(cls, codes, *, dtype: CategoricalDtype) -> np.ndarray:\n        if isinstance(codes, ExtensionArray) and is_integer_dtype(codes.dtype):\n            # Avoid the implicit conversion of Int to object\n            if isna(codes).any():\n                raise ValueError(\"codes cannot contain NA values\")\n            codes = codes.to_numpy(dtype=np.int64)\n        else:\n            codes = np.asarray(codes)\n        if len(codes) and codes.dtype.kind not in \"iu\":\n            raise ValueError(\"codes need to be array-like integers\")\n\n        if len(codes) and (codes.max() >= len(dtype.categories) or codes.min() < -1):","sourceCodeStart":1704,"sourceCodeEnd":1740,"githubUrl":"https://github.com/pandas-dev/pandas/blob/3b7651241d4da534b3559b60ef128e1c34f54116/pandas/core/arrays/categorical.py#L1704-L1740","documentation":"Raised by `_validate_setitem_value` (used by `__setitem__`, `fillna`, `where`, etc.) when assigning a fill value that is not a valid NA for the categories' dtype and is not present in the existing categories. Categoricals are closed under their category set — new labels cannot be introduced implicitly via item assignment; they must be added via `add_categories`/`set_categories` first.","triggerScenarios":"`cat[0] = 'z'` where `'z'` is not a current category; `cat.fillna('missing')` where `'missing'` is not in the categories; `cat.where(cond, 'sentinel')` with an unknown sentinel.","commonSituations":"Replacing missing values with a label that wasn't predeclared; assignment loops introducing new labels; downstream pipelines that expect free-form string assignment.","solutions":["Add the category before assigning: `cat = cat.add_categories(['z']); cat[0] = 'z'`.","Use `set_categories` to expand the label set in one call.","For `fillna`, ensure the fill value is in `cat.categories` or use a NaN-compatible value for the dtype.","If the new label is not meaningful, map it to NaN instead: `cat[0] = np.nan`."],"exampleFix":"# before\nimport numpy as np\ncat = pd.Categorical(['a', 'b', None], categories=['a', 'b'])\ncat = cat.fillna('unknown')  # TypeError\n\n# after\ncat = cat.add_categories(['unknown']).fillna('unknown')","handlingStrategy":"validation","validationCode":"def safe_cat_setitem(cat, idx, value):\n    import numpy as np\n    from pandas.api.types import is_valid_na_for_dtype\n    if not is_valid_na_for_dtype(value, cat.categories.dtype) and value not in cat.categories:\n        cat = cat.add_categories([value])\n    cat[idx] = value\n    return cat","typeGuard":"def is_assignable_to(cat, value) -> bool:\n    from pandas.api.types import is_valid_na_for_dtype\n    return is_valid_na_for_dtype(value, cat.categories.dtype) or value in cat.categories","tryCatchPattern":"try:\n    cat[i] = value\nexcept TypeError as e:\n    if 'new category' in str(e):\n        cat = cat.add_categories([value])\n        cat[i] = value\n    else:\n        raise","preventionTips":["Ensure any new label appears in `cat.categories` before assigning via `__setitem__`/`fillna`/`where`.","Pre-declare all expected labels when constructing the Categorical.","Map unexpected labels to `np.nan` if adding a category is not desired."],"tags":["categorical","setitem","fillna","new-category","typeerror"],"backgroundTag":null,"analyzedSha":"3b7651241d4da534b3559b60ef128e1c34f54116","analyzedAt":"2026-08-11T22:10:44.015Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}