pandas-dev/pandas · error · ValueError

codes cannot contain NA values

Error message

codes cannot contain NA values

What it means

Raised by _validate_codes_for_dtype when the codes passed to from_codes are a pandas nullable integer ExtensionArray (e.g. Int64) that contains NA. Integer codes must be concrete integers because -1 is the only sentinel for missing; an NA code is ambiguous and cannot be stored in the underlying int ndarray.

Source

Thrown at pandas/core/arrays/categorical.py:1733

        """

        if is_valid_na_for_dtype(fill_value, self.categories.dtype):
            fill_value = -1
        elif fill_value in self.categories:
            fill_value = self._unbox_scalar(fill_value)
        else:
            raise TypeError(
                "Cannot setitem on a Categorical with a new "
                f"category ({fill_value}), set the categories first"
            ) from None
        return fill_value

    @classmethod
    def _validate_codes_for_dtype(cls, codes, *, dtype: CategoricalDtype) -> np.ndarray:
        if isinstance(codes, ExtensionArray) and is_integer_dtype(codes.dtype):
            # Avoid the implicit conversion of Int to object
            if isna(codes).any():
                raise ValueError("codes cannot contain NA values")
            codes = codes.to_numpy(dtype=np.int64)
        else:
            codes = np.asarray(codes)
        if len(codes) and codes.dtype.kind not in "iu":
            raise ValueError("codes need to be array-like integers")

        if len(codes) and (codes.max() >= len(dtype.categories) or codes.min() < -1):
            raise ValueError("codes need to be between -1 and len(categories)-1")
        return codes

    # -------------------------------------------------------------

    @ravel_compat
    def __array__(
        self, dtype: NpDtype | None = None, copy: bool | None = None
    ) -> np.ndarray:
        """
        The numpy array interface.

View on GitHub (pinned to 71959b8cb9)

Solutions

1. Fill NA codes before passing: `codes = codes.fillna(-1).astype('int64')` then from_codes (using -1 for missing).
2. Drop rows with NA codes if missingness is not meaningful.
3. Use a plain numpy int array (no NA) computed deterministically.

Example fix

# before
codes = pd.array([0, None, 1], dtype='Int64')
pd.Categorical.from_codes(codes, categories=['a','b'])
# after
codes = pd.array([0, None, 1], dtype='Int64').fillna(-1).astype('int64')
pd.Categorical.from_codes(codes, categories=['a','b'])

import numpy as np

def clean_codes_for_dtype(codes):
    import pandas as pd
    if isinstance(codes, pd.arrays.IntegerArray):
        if codes.isna().any():
            codes = codes.fillna(-1)
        codes = codes.astype('int64')
    return np.asarray(codes, dtype=np.int64)

def codes_has_no_na(codes) -> bool:
    import pandas as pd
    if hasattr(codes, 'isna'):
        return not bool(codes.isna().any())
    return True

try:
    cat = pd.Categorical.from_codes(codes, categories=cats)
except ValueError as e:
    if 'NA values' in str(e):
        import numpy as np
        codes = codes.fillna(-1).astype('int64')
        cat = pd.Categorical.from_codes(codes, categories=cats)
    else:
        raise

When it happens

Trigger: `pd.Categorical.from_codes(pd.array([0, None, 1], dtype='Int64'), categories=['a','b'])`. The check fires specifically for integer ExtensionArrays before converting to numpy.

Common situations: Passing codes computed from a nullable integer column without filling NA; reading codes from parquet/arrow that surfaces as Int64 with nulls.

Related errors

• The categories must be provided in 'categories' or 'dtype'.
• codes need to be array-like integers
• codes need to be between -1 and len(categories)-1
• Lengths must match.
• Cannot convert float NaN to integer

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/581edfe0e5dfbc77. Report an issue: GitHub.