{"record":{"id":"c13739540d9b512b","repo":"pandas-dev/pandas","slug":"cannot-setitem-on-a-categorical-with-a-new-categor","errorCode":null,"errorMessage":"Cannot setitem on a Categorical with a new category ({fill_value}), set the categories first","messagePattern":"Cannot setitem on a Categorical with a new category \\((.+?)\\), set the categories first","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"pandas/core/arrays/categorical.py","lineNumber":1722,"sourceCode":"        Parameters\n        ----------\n        fill_value : object\n\n        Returns\n        -------\n        fill_value : int\n\n        Raises\n        ------\n        TypeError\n        \"\"\"\n\n        if is_valid_na_for_dtype(fill_value, self.categories.dtype):\n            fill_value = -1\n        elif fill_value in self.categories:\n            fill_value = self._unbox_scalar(fill_value)\n        else:\n            raise TypeError(\n                \"Cannot setitem on a Categorical with a new \"\n                f\"category ({fill_value}), set the categories first\"\n            ) from None\n        return fill_value\n\n    @classmethod\n    def _validate_codes_for_dtype(cls, codes, *, dtype: CategoricalDtype) -> np.ndarray:\n        if isinstance(codes, ExtensionArray) and is_integer_dtype(codes.dtype):\n            # Avoid the implicit conversion of Int to object\n            if isna(codes).any():\n                raise ValueError(\"codes cannot contain NA values\")\n            codes = codes.to_numpy(dtype=np.int64)\n        else:\n            codes = np.asarray(codes)\n        if len(codes) and codes.dtype.kind not in \"iu\":\n            raise ValueError(\"codes need to be array-like integers\")\n\n        if len(codes) and (codes.max() >= len(dtype.categories) or codes.min() < -1):","sourceCodeStart":1704,"sourceCodeEnd":1740,"githubUrl":"https://github.com/pandas-dev/pandas/blob/71959b8cb9b2459c16e14b34f28b178ccfe14735/pandas/core/arrays/categorical.py#L1704-L1740","documentation":"Raised by _validate_scalar during item assignment when the value being set is not a valid NA for the categories' dtype and is not among the existing categories. Categoricals have a closed category set; assigning a value outside it would require a new code that does not exist, so pandas tells you to register the category first.","triggerScenarios":"`cat[0] = 'z'` where 'z' is not in cat.categories and is not NaN-compatible. Also fires through `cat.fill_value` validation and DataFrame `.loc`/`.iloc` sets on a categorical column.","commonSituations":"Writing new labels into a categorical column read from a fixed vocabulary; appending rows with novel enum values to a categorical DataFrame column.","solutions":["Add the category first: `cat = cat.add_categories(['z'])` then `cat[0] = 'z'`.","Assign pd.NA/np.nan if you intend to mark missing (must be a valid NA for the dtype).","Rebuild the categorical with the full expected category set up front."],"exampleFix":"# before\ncat = pd.Categorical(['a','b'], categories=['a','b'])\ncat[0] = 'c'\n# after\ncat = pd.Categorical(['a','b'], categories=['a','b']).add_categories(['c'])\ncat[0] = 'c'","handlingStrategy":"validation","validationCode":"def safe_cat_setitem(cat, idx, value):\n    import numpy as np\n    import pandas as pd\n    if not (value in cat.categories or (value is pd.NA) or (isinstance(value, float) and np.isnan(value))):\n        raise ValueError(f\"{value!r} not a category; add it first\")\n    cat[idx] = value\n    return cat","typeGuard":"def is_valid_category_value(cat, value) -> bool:\n    import pandas as pd\n    import numpy as np\n    return value in cat.categories or value is pd.NA or (isinstance(value, float) and np.isnan(value))","tryCatchPattern":"try:\n    cat[0] = value\nexcept TypeError as e:\n    if 'new category' in str(e):\n        cat = cat.add_categories([value])\n        cat[0] = value\n    else:\n        raise","preventionTips":["Pre-register the full expected category set at construction.","Validate assignment values against cat.categories before setting.","Use add_categories before writing novel labels."],"tags":["categorical","setitem","new-category","typeerror"],"analyzedSha":"71959b8cb9b2459c16e14b34f28b178ccfe14735","analyzedAt":"2026-08-07T01:30:20.476Z","schemaVersion":2},"datasetVersion":"2026-08-07T03:17:09.362Z"}