{"record":{"id":"27991018c95ebb2f","repo":"pandas-dev/pandas","slug":"codes-need-to-be-between-1-and-len-categories-1","errorCode":null,"errorMessage":"codes need to be between -1 and len(categories)-1","messagePattern":"codes need to be between -1 and len\\(categories\\)-1","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"pandas/core/arrays/categorical.py","lineNumber":1741,"sourceCode":"                \"Cannot setitem on a Categorical with a new \"\n                f\"category ({fill_value}), set the categories first\"\n            ) from None\n        return fill_value\n\n    @classmethod\n    def _validate_codes_for_dtype(cls, codes, *, dtype: CategoricalDtype) -> np.ndarray:\n        if isinstance(codes, ExtensionArray) and is_integer_dtype(codes.dtype):\n            # Avoid the implicit conversion of Int to object\n            if isna(codes).any():\n                raise ValueError(\"codes cannot contain NA values\")\n            codes = codes.to_numpy(dtype=np.int64)\n        else:\n            codes = np.asarray(codes)\n        if len(codes) and codes.dtype.kind not in \"iu\":\n            raise ValueError(\"codes need to be array-like integers\")\n\n        if len(codes) and (codes.max() >= len(dtype.categories) or codes.min() < -1):\n            raise ValueError(\"codes need to be between -1 and len(categories)-1\")\n        return codes\n\n    # -------------------------------------------------------------\n\n    @ravel_compat\n    def __array__(\n        self, dtype: NpDtype | None = None, copy: bool | None = None\n    ) -> np.ndarray:\n        \"\"\"\n        The numpy array interface.\n\n        Users should not call this directly. Rather, it is invoked by\n        :func:`numpy.array` and :func:`numpy.asarray`.\n\n        Parameters\n        ----------\n        dtype : np.dtype or None\n            Specifies the dtype for the array.","sourceCodeStart":1723,"sourceCodeEnd":1759,"githubUrl":"https://github.com/pandas-dev/pandas/blob/71959b8cb9b2459c16e14b34f28b178ccfe14735/pandas/core/arrays/categorical.py#L1723-L1759","documentation":"Raised by _validate_codes_for_dtype when any code is < -1 or >= len(categories). Codes are 0-based positions into the categories array with -1 reserved for missing; out-of-range codes would index nonexistent categories and segfault or produce garbage, so validation (enabled by default in from_codes) refuses them.","triggerScenarios":"`pd.Categorical.from_codes([0, 2], categories=['a','b'])` (max code 2 >= len 2), or codes containing -2. Validation runs when validate=True (the default).","commonSituations":"Codes derived from a different/older category list whose length shrank; off-by-one when hand-building codes; mismatched categories after filtering.","solutions":["Ensure codes are within [-1, len(categories)-1]; clip or remap: `np.clip(codes, -1, len(categories)-1)`.","Regenerate codes from the current categories via `pd.Categorical(values, categories=[...]).codes`.","Pass `validate=False` to from_codes ONLY if you are certain the codes are correct (beware: invalid codes may segfault).","Recompute codes with `cat.categories.get_indexer(values)`."],"exampleFix":"# before\npd.Categorical.from_codes([0, 2], categories=['a','b'])\n# after\npd.Categorical.from_codes([0, 1], categories=['a','b'])","handlingStrategy":"validation","validationCode":"import numpy as np\n\ndef validate_code_range(codes, n_categories):\n    codes = np.asarray(codes)\n    if codes.size and (codes.min() < -1 or codes.max() >= n_categories):\n        raise ValueError(f\"codes out of range [-1, {n_categories - 1}]\")\n    return codes","typeGuard":"def codes_in_range(codes, n_categories) -> bool:\n    import numpy as np\n    codes = np.asarray(codes)\n    return codes.size == 0 or (codes.min() >= -1 and codes.max() < n_categories)","tryCatchPattern":"try:\n    cat = pd.Categorical.from_codes(codes, categories=cats)\nexcept ValueError as e:\n    if 'between -1 and' in str(e):\n        import numpy as np\n        codes = np.clip(np.asarray(codes), -1, len(cats) - 1)\n        cat = pd.Categorical.from_codes(codes, categories=cats)\n    else:\n        raise","preventionTips":["Regenerate codes from the current categories rather than reusing stale ones.","Clip or remap codes after filtering categories.","Keep validate=True (default) in from_codes during development to catch drift early."],"tags":["categorical","from-codes","out-of-range","validation","valueerror"],"analyzedSha":"71959b8cb9b2459c16e14b34f28b178ccfe14735","analyzedAt":"2026-08-07T01:30:20.476Z","schemaVersion":2},"datasetVersion":"2026-08-07T03:17:09.362Z"}