pandas-dev/pandas · error · ValueError

codes cannot contain NA values

Error message

codes cannot contain NA values

What it means

Raised by `_validate_codes_for_dtype` (invoked from `from_codes` with `validate=True`) when the supplied `codes` are an integer ExtensionArray (e.g. `pd.array([0, 1], dtype='Int64')`) that contains NA. Internal Categorical codes use `-1` for missing values in a plain int64 ndarray; they cannot store pandas NA, so the validator refuses NA-bearing integer codes.

Solutions

1. Replace NA with -1 before calling: `codes = codes.fillna(-1).astype('int64')`.
2. Drop rows with NA codes: `codes = codes.dropna().astype('int64')`.
3. Validate `not codes.isna().any()` before passing to `from_codes`.
4. If you have a nullable Int array, convert via `.to_numpy(dtype='int64', na_value=-1)`.

Example fix

# before
import pandas as pd
codes = pd.array([0, 1, pd.NA], dtype='Int64')
cat = pd.Categorical.from_codes(codes, categories=['a', 'b'])  # ValueError

# after
codes = codes.to_numpy(dtype='int64', na_value=-1)
cat = pd.Categorical.from_codes(codes, categories=['a', 'b'])

import numpy as np
import pandas as pd

def clean_codes(codes):
    codes = pd.array(codes) if not isinstance(codes, pd.ArrayLike) else codes
    if codes.isna().any():
        codes = codes.fillna(-1)
    return codes.to_numpy(dtype='int64') if hasattr(codes, 'to_numpy') else np.asarray(codes, dtype='int64')

def codes_have_no_na(codes) -> bool:
    import pandas as pd
    arr = pd.array(codes) if not hasattr(codes, 'isna') else codes
    return not arr.isna().any()

try:
    cat = pd.Categorical.from_codes(codes, categories=cats)
except ValueError as e:
    if 'cannot contain NA' in str(e):
        import pandas as pd
        cleaned = pd.array(codes).fillna(-1).to_numpy(dtype='int64')
        cat = pd.Categorical.from_codes(cleaned, categories=cats)
    else:
        raise

When it happens

Trigger: Passing a nullable `Int64` array/Series with `pd.NA` to `pd.Categorical.from_codes(..., validate=True)`, or codes read from a nullable column that were not cleaned.

Common situations: Interfacing with the pandas nullable integer extension types; reading codes from Parquet/Arrow that surfaces `pd.NA`; treating a survey dataset where non-responses appear as `pd.NA` in the codes column.

Related errors

• codes need to be array-like integers
• codes need to be between -1 and len(categories)-1
• The categories must be provided in 'categories' or 'dtype'…
• Cannot cast dtype to
• Cannot convert float NaN to integer

AI-assisted analysis of pandas-dev/pandas@3b7651241d (2026-08-11). Data as JSON: /api/errors/581edfe0e5dfbc77. Report an issue: GitHub.

Appendix: source

Thrown at pandas/core/arrays/categorical.py:1733

        """

        if is_valid_na_for_dtype(fill_value, self.categories.dtype):
            fill_value = -1
        elif fill_value in self.categories:
            fill_value = self._unbox_scalar(fill_value)
        else:
            raise TypeError(
                "Cannot setitem on a Categorical with a new "
                f"category ({fill_value}), set the categories first"
            ) from None
        return fill_value

    @classmethod
    def _validate_codes_for_dtype(cls, codes, *, dtype: CategoricalDtype) -> np.ndarray:
        if isinstance(codes, ExtensionArray) and is_integer_dtype(codes.dtype):
            # Avoid the implicit conversion of Int to object
            if isna(codes).any():
                raise ValueError("codes cannot contain NA values")
            codes = codes.to_numpy(dtype=np.int64)
        else:
            codes = np.asarray(codes)
        if len(codes) and codes.dtype.kind not in "iu":
            raise ValueError("codes need to be array-like integers")

        if len(codes) and (codes.max() >= len(dtype.categories) or codes.min() < -1):
            raise ValueError("codes need to be between -1 and len(categories)-1")
        return codes

    # -------------------------------------------------------------

    @ravel_compat
    def __array__(
        self, dtype: NpDtype | None = None, copy: bool | None = None
    ) -> np.ndarray:
        """
        The numpy array interface.

View on GitHub (pinned to 3b7651241d)