pandas-dev/pandas · error · ValueError

codes need to be between -1 and len(categories)-1

Error message

codes need to be between -1 and len(categories)-1

What it means

Raised by `_validate_codes_for_dtype` when the codes array contains a value `>= len(categories)` or `< -1`. Valid codes are integers in `[-1, len(categories) - 1]`, where `-1` represents a missing value. Out-of-range codes would index outside the categories array (segfault risk per the docstring) so they are caught here.

Solutions

1. Ensure all codes are within `[-1, len(categories) - 1]`: clip or remap them before calling.
2. Pass `validate=False` only if you are certain the codes are correct (the docstring warns of segfault risk).
3. If categories were trimmed, recode using `recode_for_categories` or rebuild via the values constructor.
4. Inspect `codes.min()`, `codes.max()` against `len(categories)` as a guard.

Example fix

# before
import numpy as np
cat = pd.Categorical.from_codes(
    np.array([0, 2, 1]), categories=['a', 'b']  # ValueError: 2 >= 2
)

# after (expand categories or remap codes)
cat = pd.Categorical.from_codes(
    np.array([0, 2, 1]), categories=['a', 'b', 'c']
)

import numpy as np

def validate_codes_range(codes, n_categories):
    codes = np.asarray(codes)
    if codes.size and (codes.max() >= n_categories or codes.min() < -1):
        raise ValueError(f'codes out of range [-1, {n_categories - 1}]')
    return codes

def codes_in_range(codes, n_categories) -> bool:
    import numpy as np
    arr = np.asarray(codes)
    if arr.size == 0:
        return True
    return arr.min() >= -1 and arr.max() < n_categories

try:
    cat = pd.Categorical.from_codes(codes, categories=cats)
except ValueError as e:
    if 'between -1 and len(categories)' in str(e):
        # expand categories or remap codes to fit
        cat = pd.Categorical.from_codes(codes, categories=cats + extra)
    else:
        raise

When it happens

Trigger: `pd.Categorical.from_codes([0, 2], categories=['a', 'b'])` (max code 2 >= 2 categories), or any codes array with a negative value other than -1.

Common situations: Removing categories without recoding; constructing codes from a different/older category set; off-by-one indexing in custom label encoders; serialization round-trips where categories were dropped.

Related errors

• codes cannot contain NA values
• codes need to be array-like integers
• The categories must be provided in 'categories' or 'dtype'…
• Cannot cast dtype to
• Cannot convert float NaN to integer

AI-assisted analysis of pandas-dev/pandas@3b7651241d (2026-08-11). Data as JSON: /api/errors/27991018c95ebb2f. Report an issue: GitHub.

Appendix: source

Thrown at pandas/core/arrays/categorical.py:1741

                "Cannot setitem on a Categorical with a new "
                f"category ({fill_value}), set the categories first"
            ) from None
        return fill_value

    @classmethod
    def _validate_codes_for_dtype(cls, codes, *, dtype: CategoricalDtype) -> np.ndarray:
        if isinstance(codes, ExtensionArray) and is_integer_dtype(codes.dtype):
            # Avoid the implicit conversion of Int to object
            if isna(codes).any():
                raise ValueError("codes cannot contain NA values")
            codes = codes.to_numpy(dtype=np.int64)
        else:
            codes = np.asarray(codes)
        if len(codes) and codes.dtype.kind not in "iu":
            raise ValueError("codes need to be array-like integers")

        if len(codes) and (codes.max() >= len(dtype.categories) or codes.min() < -1):
            raise ValueError("codes need to be between -1 and len(categories)-1")
        return codes

    # -------------------------------------------------------------

    @ravel_compat
    def __array__(
        self, dtype: NpDtype | None = None, copy: bool | None = None
    ) -> np.ndarray:
        """
        The numpy array interface.

        Users should not call this directly. Rather, it is invoked by
        :func:`numpy.array` and :func:`numpy.asarray`.

        Parameters
        ----------
        dtype : np.dtype or None
            Specifies the dtype for the array.

View on GitHub (pinned to 3b7651241d)