{"record":{"id":"581edfe0e5dfbc77","repo":"pandas-dev/pandas","slug":"codes-cannot-contain-na-values","errorCode":null,"errorMessage":"codes cannot contain NA values","messagePattern":"codes cannot contain NA values","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"pandas/core/arrays/categorical.py","lineNumber":1733,"sourceCode":"        \"\"\"\n\n        if is_valid_na_for_dtype(fill_value, self.categories.dtype):\n            fill_value = -1\n        elif fill_value in self.categories:\n            fill_value = self._unbox_scalar(fill_value)\n        else:\n            raise TypeError(\n                \"Cannot setitem on a Categorical with a new \"\n                f\"category ({fill_value}), set the categories first\"\n            ) from None\n        return fill_value\n\n    @classmethod\n    def _validate_codes_for_dtype(cls, codes, *, dtype: CategoricalDtype) -> np.ndarray:\n        if isinstance(codes, ExtensionArray) and is_integer_dtype(codes.dtype):\n            # Avoid the implicit conversion of Int to object\n            if isna(codes).any():\n                raise ValueError(\"codes cannot contain NA values\")\n            codes = codes.to_numpy(dtype=np.int64)\n        else:\n            codes = np.asarray(codes)\n        if len(codes) and codes.dtype.kind not in \"iu\":\n            raise ValueError(\"codes need to be array-like integers\")\n\n        if len(codes) and (codes.max() >= len(dtype.categories) or codes.min() < -1):\n            raise ValueError(\"codes need to be between -1 and len(categories)-1\")\n        return codes\n\n    # -------------------------------------------------------------\n\n    @ravel_compat\n    def __array__(\n        self, dtype: NpDtype | None = None, copy: bool | None = None\n    ) -> np.ndarray:\n        \"\"\"\n        The numpy array interface.","sourceCodeStart":1715,"sourceCodeEnd":1751,"githubUrl":"https://github.com/pandas-dev/pandas/blob/3b7651241d4da534b3559b60ef128e1c34f54116/pandas/core/arrays/categorical.py#L1715-L1751","documentation":"Raised by `_validate_codes_for_dtype` (invoked from `from_codes` with `validate=True`) when the supplied `codes` are an integer ExtensionArray (e.g. `pd.array([0, 1], dtype='Int64')`) that contains NA. Internal Categorical codes use `-1` for missing values in a plain int64 ndarray; they cannot store pandas NA, so the validator refuses NA-bearing integer codes.","triggerScenarios":"Passing a nullable `Int64` array/Series with `pd.NA` to `pd.Categorical.from_codes(..., validate=True)`, or codes read from a nullable column that were not cleaned.","commonSituations":"Interfacing with the pandas nullable integer extension types; reading codes from Parquet/Arrow that surfaces `pd.NA`; treating a survey dataset where non-responses appear as `pd.NA` in the codes column.","solutions":["Replace NA with -1 before calling: `codes = codes.fillna(-1).astype('int64')`.","Drop rows with NA codes: `codes = codes.dropna().astype('int64')`.","Validate `not codes.isna().any()` before passing to `from_codes`.","If you have a nullable Int array, convert via `.to_numpy(dtype='int64', na_value=-1)`."],"exampleFix":"# before\nimport pandas as pd\ncodes = pd.array([0, 1, pd.NA], dtype='Int64')\ncat = pd.Categorical.from_codes(codes, categories=['a', 'b'])  # ValueError\n\n# after\ncodes = codes.to_numpy(dtype='int64', na_value=-1)\ncat = pd.Categorical.from_codes(codes, categories=['a', 'b'])","handlingStrategy":"validation","validationCode":"import numpy as np\nimport pandas as pd\n\ndef clean_codes(codes):\n    codes = pd.array(codes) if not isinstance(codes, pd.ArrayLike) else codes\n    if codes.isna().any():\n        codes = codes.fillna(-1)\n    return codes.to_numpy(dtype='int64') if hasattr(codes, 'to_numpy') else np.asarray(codes, dtype='int64')","typeGuard":"def codes_have_no_na(codes) -> bool:\n    import pandas as pd\n    arr = pd.array(codes) if not hasattr(codes, 'isna') else codes\n    return not arr.isna().any()","tryCatchPattern":"try:\n    cat = pd.Categorical.from_codes(codes, categories=cats)\nexcept ValueError as e:\n    if 'cannot contain NA' in str(e):\n        import pandas as pd\n        cleaned = pd.array(codes).fillna(-1).to_numpy(dtype='int64')\n        cat = pd.Categorical.from_codes(cleaned, categories=cats)\n    else:\n        raise","preventionTips":["Replace `pd.NA` in codes with `-1` before calling `from_codes`.","When reading codes from nullable Int columns, use `.to_numpy(dtype='int64', na_value=-1)`.","Validate `not pd.array(codes).isna().any()` at the boundary."],"tags":["categorical","from-codes","codes","nullable","valueerror"],"backgroundTag":null,"analyzedSha":"3b7651241d4da534b3559b60ef128e1c34f54116","analyzedAt":"2026-08-11T22:10:44.015Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}