pandas-dev/pandas · error · ValueError

codes need to be between -1 and len(categories)-1

Error message

codes need to be between -1 and len(categories)-1

What it means

Raised by _validate_codes_for_dtype when any code is < -1 or >= len(categories). Codes are 0-based positions into the categories array with -1 reserved for missing; out-of-range codes would index nonexistent categories and segfault or produce garbage, so validation (enabled by default in from_codes) refuses them.

Source

Thrown at pandas/core/arrays/categorical.py:1741

                "Cannot setitem on a Categorical with a new "
                f"category ({fill_value}), set the categories first"
            ) from None
        return fill_value

    @classmethod
    def _validate_codes_for_dtype(cls, codes, *, dtype: CategoricalDtype) -> np.ndarray:
        if isinstance(codes, ExtensionArray) and is_integer_dtype(codes.dtype):
            # Avoid the implicit conversion of Int to object
            if isna(codes).any():
                raise ValueError("codes cannot contain NA values")
            codes = codes.to_numpy(dtype=np.int64)
        else:
            codes = np.asarray(codes)
        if len(codes) and codes.dtype.kind not in "iu":
            raise ValueError("codes need to be array-like integers")

        if len(codes) and (codes.max() >= len(dtype.categories) or codes.min() < -1):
            raise ValueError("codes need to be between -1 and len(categories)-1")
        return codes

    # -------------------------------------------------------------

    @ravel_compat
    def __array__(
        self, dtype: NpDtype | None = None, copy: bool | None = None
    ) -> np.ndarray:
        """
        The numpy array interface.

        Users should not call this directly. Rather, it is invoked by
        :func:`numpy.array` and :func:`numpy.asarray`.

        Parameters
        ----------
        dtype : np.dtype or None
            Specifies the dtype for the array.

View on GitHub (pinned to 71959b8cb9)

Solutions

1. Ensure codes are within [-1, len(categories)-1]; clip or remap: `np.clip(codes, -1, len(categories)-1)`.
2. Regenerate codes from the current categories via `pd.Categorical(values, categories=[...]).codes`.
3. Pass `validate=False` to from_codes ONLY if you are certain the codes are correct (beware: invalid codes may segfault).
4. Recompute codes with `cat.categories.get_indexer(values)`.

Example fix

# before
pd.Categorical.from_codes([0, 2], categories=['a','b'])
# after
pd.Categorical.from_codes([0, 1], categories=['a','b'])

import numpy as np

def validate_code_range(codes, n_categories):
    codes = np.asarray(codes)
    if codes.size and (codes.min() < -1 or codes.max() >= n_categories):
        raise ValueError(f"codes out of range [-1, {n_categories - 1}]")
    return codes

def codes_in_range(codes, n_categories) -> bool:
    import numpy as np
    codes = np.asarray(codes)
    return codes.size == 0 or (codes.min() >= -1 and codes.max() < n_categories)

try:
    cat = pd.Categorical.from_codes(codes, categories=cats)
except ValueError as e:
    if 'between -1 and' in str(e):
        import numpy as np
        codes = np.clip(np.asarray(codes), -1, len(cats) - 1)
        cat = pd.Categorical.from_codes(codes, categories=cats)
    else:
        raise

When it happens

Trigger: `pd.Categorical.from_codes([0, 2], categories=['a','b'])` (max code 2 >= len 2), or codes containing -2. Validation runs when validate=True (the default).

Common situations: Codes derived from a different/older category list whose length shrank; off-by-one when hand-building codes; mismatched categories after filtering.

Related errors

• The categories must be provided in 'categories' or 'dtype'.
• codes cannot contain NA values
• codes need to be array-like integers
• Value must be an instance of {type_repr}
• Value must be one of {pp_values}

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/27991018c95ebb2f. Report an issue: GitHub.