pandas-dev/pandas · error · ValueError

removals must all be in old categories: {not_included}

Error message

removals must all be in old categories: {not_included}

What it means

Raised by remove_categories when some entries in `removals` are not present in the current categories. Removal must target existing categories; the message lists the ones that could not be found (after dedup and dropna on the removals index).

Source

Thrown at pandas/core/arrays/categorical.py:1492

        [NaN, 'c', 'b', 'c', NaN]
        Categories (2, str): ['b', 'c']
        """
        from pandas import Index

        if not is_list_like(removals):
            removals = [removals]

        removals = Index(removals).unique().dropna()
        new_categories = (
            self.dtype.categories.difference(removals, sort=False)
            if self.dtype.ordered is True
            else self.dtype.categories.difference(removals)
        )
        not_included = removals.difference(self.dtype.categories)

        if len(not_included) != 0:
            not_included = set(not_included)
            raise ValueError(f"removals must all be in old categories: {not_included}")

        return self.set_categories(new_categories, ordered=self.ordered, rename=False)

    def remove_unused_categories(self) -> Self:
        """
        Remove categories which are not used.

        This method is useful when working with datasets
        that undergo dynamic changes where categories may no longer be
        relevant, allowing to maintain a clean, efficient data structure.

        Returns
        -------
        Categorical
            Categorical with unused categories dropped.

        See Also
        --------

View on GitHub (pinned to 71959b8cb9)

Solutions

1. Intersect with current categories: `cat.remove_categories([c for c in removals if c in cat.categories])`.
2. Check membership first: `to_remove = set(removals) & set(cat.categories)` then `cat.remove_categories(list(to_remove))`.
3. If you want to also drop rows, filter the data rather than removing categories.

Example fix

# before
cat = pd.Categorical(['a','b'], categories=['a','b'])
cat.remove_categories(['a','z'])
# after
cat = pd.Categorical(['a','b'], categories=['a','b'])
cat.remove_categories(['a'])

def safe_remove_categories(cat, removals):
    missing = set(removals) - set(cat.categories)
    if missing:
        raise ValueError(f"not categories: {missing}")
    return cat.remove_categories(removals)

def all_removable_categories(cat, removals) -> bool:
    return set(removals).issubset(set(cat.categories))

try:
    cat = cat.remove_categories(removals)
except ValueError as e:
    if 'must all be in old categories' in str(e):
        cat = cat.remove_categories([c for c in removals if c in cat.categories])
    else:
        raise

When it happens

Trigger: `cat.remove_categories(['z'])` where 'z' is not in cat.categories; or removing a category that was already removed in a prior step.

Common situations: Hardcoding removal lists from an external schema that has drifted from the data; chaining removes without re-checking remaining categories.

Related errors

• Lengths must match.
• Cannot convert float NaN to integer
• Cannot cast {self.categories.dtype} dtype to {dtype}
• The categories must be provided in 'categories' or 'dtype'.
• new categories need to have the same number of items as the

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/8376b5e2168cb5f4. Report an issue: GitHub.