{"record":{"id":"581edfe0e5dfbc77","repo":"pandas-dev/pandas","slug":"codes-cannot-contain-na-values","errorCode":null,"errorMessage":"codes cannot contain NA values","messagePattern":"codes cannot contain NA values","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"pandas/core/arrays/categorical.py","lineNumber":1733,"sourceCode":"        \"\"\"\n\n        if is_valid_na_for_dtype(fill_value, self.categories.dtype):\n            fill_value = -1\n        elif fill_value in self.categories:\n            fill_value = self._unbox_scalar(fill_value)\n        else:\n            raise TypeError(\n                \"Cannot setitem on a Categorical with a new \"\n                f\"category ({fill_value}), set the categories first\"\n            ) from None\n        return fill_value\n\n    @classmethod\n    def _validate_codes_for_dtype(cls, codes, *, dtype: CategoricalDtype) -> np.ndarray:\n        if isinstance(codes, ExtensionArray) and is_integer_dtype(codes.dtype):\n            # Avoid the implicit conversion of Int to object\n            if isna(codes).any():\n                raise ValueError(\"codes cannot contain NA values\")\n            codes = codes.to_numpy(dtype=np.int64)\n        else:\n            codes = np.asarray(codes)\n        if len(codes) and codes.dtype.kind not in \"iu\":\n            raise ValueError(\"codes need to be array-like integers\")\n\n        if len(codes) and (codes.max() >= len(dtype.categories) or codes.min() < -1):\n            raise ValueError(\"codes need to be between -1 and len(categories)-1\")\n        return codes\n\n    # -------------------------------------------------------------\n\n    @ravel_compat\n    def __array__(\n        self, dtype: NpDtype | None = None, copy: bool | None = None\n    ) -> np.ndarray:\n        \"\"\"\n        The numpy array interface.","sourceCodeStart":1715,"sourceCodeEnd":1751,"githubUrl":"https://github.com/pandas-dev/pandas/blob/71959b8cb9b2459c16e14b34f28b178ccfe14735/pandas/core/arrays/categorical.py#L1715-L1751","documentation":"Raised by _validate_codes_for_dtype when the codes passed to from_codes are a pandas nullable integer ExtensionArray (e.g. Int64) that contains NA. Integer codes must be concrete integers because -1 is the only sentinel for missing; an NA code is ambiguous and cannot be stored in the underlying int ndarray.","triggerScenarios":"`pd.Categorical.from_codes(pd.array([0, None, 1], dtype='Int64'), categories=['a','b'])`. The check fires specifically for integer ExtensionArrays before converting to numpy.","commonSituations":"Passing codes computed from a nullable integer column without filling NA; reading codes from parquet/arrow that surfaces as Int64 with nulls.","solutions":["Fill NA codes before passing: `codes = codes.fillna(-1).astype('int64')` then from_codes (using -1 for missing).","Drop rows with NA codes if missingness is not meaningful.","Use a plain numpy int array (no NA) computed deterministically."],"exampleFix":"# before\ncodes = pd.array([0, None, 1], dtype='Int64')\npd.Categorical.from_codes(codes, categories=['a','b'])\n# after\ncodes = pd.array([0, None, 1], dtype='Int64').fillna(-1).astype('int64')\npd.Categorical.from_codes(codes, categories=['a','b'])","handlingStrategy":"validation","validationCode":"import numpy as np\n\ndef clean_codes_for_dtype(codes):\n    import pandas as pd\n    if isinstance(codes, pd.arrays.IntegerArray):\n        if codes.isna().any():\n            codes = codes.fillna(-1)\n        codes = codes.astype('int64')\n    return np.asarray(codes, dtype=np.int64)","typeGuard":"def codes_has_no_na(codes) -> bool:\n    import pandas as pd\n    if hasattr(codes, 'isna'):\n        return not bool(codes.isna().any())\n    return True","tryCatchPattern":"try:\n    cat = pd.Categorical.from_codes(codes, categories=cats)\nexcept ValueError as e:\n    if 'NA values' in str(e):\n        import numpy as np\n        codes = codes.fillna(-1).astype('int64')\n        cat = pd.Categorical.from_codes(codes, categories=cats)\n    else:\n        raise","preventionTips":["Fill NA codes with -1 (the missing sentinel) before from_codes.","Convert nullable Int64 codes to plain int64 explicitly.","Validate no NA in codes for integer ExtensionArray inputs."],"tags":["categorical","from-codes","nullable-integer","na","valueerror"],"analyzedSha":"71959b8cb9b2459c16e14b34f28b178ccfe14735","analyzedAt":"2026-08-07T01:30:20.476Z","schemaVersion":2},"datasetVersion":"2026-08-07T03:17:09.362Z"}