pandas-dev/pandas · error · ValueError

new categories need to have the same number of items as the…

Error message

new categories need to have the same number of items as the old categories!

What it means

Raised by the `rename_categories`/`set_categories(fastpath=...)` path when the new categories collection has a different length than the existing categories. Renaming is a 1-to-1 relabeling of codes — the cardinality must stay identical so each existing code keeps a target label.

Solutions

1. Use `cat.set_categories(new_list)` instead — it intentionally allows a different cardinality (existing codes that no longer map become NaN).
2. Provide a replacement list whose length equals `len(cat.categories)`.
3. Use a dict mapping old→new with `rename_categories` for partial renames (keys must cover all current categories).
4. Verify `len(new_categories) == len(cat.categories)` before calling rename.

Example fix

# before
cat = pd.Categorical(['a', 'b', 'c'])
cat = cat.rename_categories(['x', 'y'])  # ValueError

# after (rename, full mapping)
cat = cat.rename_categories({'a': 'x', 'b': 'y', 'c': 'z'})
# after (replace categories entirely, size may change)
cat = cat.set_categories(['x', 'y'])

def safe_rename(cat, new_categories):
    new_categories = list(new_categories)
    if len(new_categories) != len(cat.categories):
        return cat.set_categories(new_categories)  # intentional size change
    return cat.rename_categories(new_categories)

def is_one_to_one_rename(cat, new_categories) -> bool:
    return len(list(new_categories)) == len(cat.categories)

try:
    cat = cat.rename_categories(new_list)
except ValueError as e:
    if 'same number of items' in str(e):
        cat = cat.set_categories(new_list)
    else:
        raise

When it happens

Trigger: Calling `cat.rename_categories(['x', 'y'])` on a Categorical with 3 (or 1) current categories; passing a dict to rename that omits some categories when length semantics apply.

Common situations: Renaming labels after a refactor where the set of categories also changed; off-by-one in replacement lists; confusing `rename_categories` with `set_categories` (the latter allows size changes).

Related errors

• Cannot cast dtype to
• Cannot convert float NaN to integer
• codes cannot contain NA values
• codes need to be array-like integers
• codes need to be between -1 and len(categories)-1

AI-assisted analysis of pandas-dev/pandas@3b7651241d (2026-08-11). Data as JSON: /api/errors/c0c43ffd8ef8cb39. Report an issue: GitHub.

Appendix: source

Thrown at pandas/core/arrays/categorical.py:987

        >>> c
        ['a', 'b']
        Categories (2, str): ['a', 'b']

        >>> c._set_categories(pd.Index(["a", "c"]))
        >>> c
        ['a', 'c']
        Categories (2, str): ['a', 'c']
        """
        if fastpath:
            new_dtype = CategoricalDtype._from_fastpath(categories, self.ordered)
        else:
            new_dtype = CategoricalDtype(categories, ordered=self.ordered)
        if (
            not fastpath
            and self.dtype.categories is not None
            and len(new_dtype.categories) != len(self.dtype.categories)
        ):
            raise ValueError(
                "new categories need to have the same number of "
                "items as the old categories!"
            )

        super().__init__(self._ndarray, new_dtype)

    def _set_dtype(self, dtype: CategoricalDtype, *, copy: bool) -> Self:
        """
        Internal method for directly updating the CategoricalDtype

        Parameters
        ----------
        dtype : CategoricalDtype

        Notes
        -----
        We don't do any validation here. It's assumed that the dtype is
        a (valid) instance of `CategoricalDtype`.

View on GitHub (pinned to 3b7651241d)