{"record":{"id":"27991018c95ebb2f","repo":"pandas-dev/pandas","slug":"codes-need-to-be-between-1-and-len-categories-1","errorCode":null,"errorMessage":"codes need to be between -1 and len(categories)-1","messagePattern":"codes need to be between -1 and len\\(categories\\)-1","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"pandas/core/arrays/categorical.py","lineNumber":1741,"sourceCode":"                \"Cannot setitem on a Categorical with a new \"\n                f\"category ({fill_value}), set the categories first\"\n            ) from None\n        return fill_value\n\n    @classmethod\n    def _validate_codes_for_dtype(cls, codes, *, dtype: CategoricalDtype) -> np.ndarray:\n        if isinstance(codes, ExtensionArray) and is_integer_dtype(codes.dtype):\n            # Avoid the implicit conversion of Int to object\n            if isna(codes).any():\n                raise ValueError(\"codes cannot contain NA values\")\n            codes = codes.to_numpy(dtype=np.int64)\n        else:\n            codes = np.asarray(codes)\n        if len(codes) and codes.dtype.kind not in \"iu\":\n            raise ValueError(\"codes need to be array-like integers\")\n\n        if len(codes) and (codes.max() >= len(dtype.categories) or codes.min() < -1):\n            raise ValueError(\"codes need to be between -1 and len(categories)-1\")\n        return codes\n\n    # -------------------------------------------------------------\n\n    @ravel_compat\n    def __array__(\n        self, dtype: NpDtype | None = None, copy: bool | None = None\n    ) -> np.ndarray:\n        \"\"\"\n        The numpy array interface.\n\n        Users should not call this directly. Rather, it is invoked by\n        :func:`numpy.array` and :func:`numpy.asarray`.\n\n        Parameters\n        ----------\n        dtype : np.dtype or None\n            Specifies the dtype for the array.","sourceCodeStart":1723,"sourceCodeEnd":1759,"githubUrl":"https://github.com/pandas-dev/pandas/blob/3b7651241d4da534b3559b60ef128e1c34f54116/pandas/core/arrays/categorical.py#L1723-L1759","documentation":"Raised by `_validate_codes_for_dtype` when the codes array contains a value `>= len(categories)` or `< -1`. Valid codes are integers in `[-1, len(categories) - 1]`, where `-1` represents a missing value. Out-of-range codes would index outside the categories array (segfault risk per the docstring) so they are caught here.","triggerScenarios":"`pd.Categorical.from_codes([0, 2], categories=['a', 'b'])` (max code 2 >= 2 categories), or any codes array with a negative value other than -1.","commonSituations":"Removing categories without recoding; constructing codes from a different/older category set; off-by-one indexing in custom label encoders; serialization round-trips where categories were dropped.","solutions":["Ensure all codes are within `[-1, len(categories) - 1]`: clip or remap them before calling.","Pass `validate=False` only if you are certain the codes are correct (the docstring warns of segfault risk).","If categories were trimmed, recode using `recode_for_categories` or rebuild via the values constructor.","Inspect `codes.min()`, `codes.max()` against `len(categories)` as a guard."],"exampleFix":"# before\nimport numpy as np\ncat = pd.Categorical.from_codes(\n    np.array([0, 2, 1]), categories=['a', 'b']  # ValueError: 2 >= 2\n)\n\n# after (expand categories or remap codes)\ncat = pd.Categorical.from_codes(\n    np.array([0, 2, 1]), categories=['a', 'b', 'c']\n)","handlingStrategy":"validation","validationCode":"import numpy as np\n\ndef validate_codes_range(codes, n_categories):\n    codes = np.asarray(codes)\n    if codes.size and (codes.max() >= n_categories or codes.min() < -1):\n        raise ValueError(f'codes out of range [-1, {n_categories - 1}]')\n    return codes","typeGuard":"def codes_in_range(codes, n_categories) -> bool:\n    import numpy as np\n    arr = np.asarray(codes)\n    if arr.size == 0:\n        return True\n    return arr.min() >= -1 and arr.max() < n_categories","tryCatchPattern":"try:\n    cat = pd.Categorical.from_codes(codes, categories=cats)\nexcept ValueError as e:\n    if 'between -1 and len(categories)' in str(e):\n        # expand categories or remap codes to fit\n        cat = pd.Categorical.from_codes(codes, categories=cats + extra)\n    else:\n        raise","preventionTips":["Cross-check `codes.min() >= -1` and `codes.max() < len(categories)` before `from_codes`.","When trimming categories, recode existing values via `recode_for_categories` or rebuild from values.","Pass `validate=True` (the default) during development to surface range issues early."],"tags":["categorical","from-codes","codes","range","valueerror"],"backgroundTag":null,"analyzedSha":"3b7651241d4da534b3559b60ef128e1c34f54116","analyzedAt":"2026-08-11T22:10:44.015Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}