{"record":{"id":"8376b5e2168cb5f4","repo":"pandas-dev/pandas","slug":"removals-must-all-be-in-old-categories-not-inclu","errorCode":null,"errorMessage":"removals must all be in old categories: {not_included}","messagePattern":"removals must all be in old categories: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"pandas/core/arrays/categorical.py","lineNumber":1492,"sourceCode":"        [NaN, 'c', 'b', 'c', NaN]\n        Categories (2, str): ['b', 'c']\n        \"\"\"\n        from pandas import Index\n\n        if not is_list_like(removals):\n            removals = [removals]\n\n        removals = Index(removals).unique().dropna()\n        new_categories = (\n            self.dtype.categories.difference(removals, sort=False)\n            if self.dtype.ordered is True\n            else self.dtype.categories.difference(removals)\n        )\n        not_included = removals.difference(self.dtype.categories)\n\n        if len(not_included) != 0:\n            not_included = set(not_included)\n            raise ValueError(f\"removals must all be in old categories: {not_included}\")\n\n        return self.set_categories(new_categories, ordered=self.ordered, rename=False)\n\n    def remove_unused_categories(self) -> Self:\n        \"\"\"\n        Remove categories which are not used.\n\n        This method is useful when working with datasets\n        that undergo dynamic changes where categories may no longer be\n        relevant, allowing to maintain a clean, efficient data structure.\n\n        Returns\n        -------\n        Categorical\n            Categorical with unused categories dropped.\n\n        See Also\n        --------","sourceCodeStart":1474,"sourceCodeEnd":1510,"githubUrl":"https://github.com/pandas-dev/pandas/blob/71959b8cb9b2459c16e14b34f28b178ccfe14735/pandas/core/arrays/categorical.py#L1474-L1510","documentation":"Raised by remove_categories when some entries in `removals` are not present in the current categories. Removal must target existing categories; the message lists the ones that could not be found (after dedup and dropna on the removals index).","triggerScenarios":"`cat.remove_categories(['z'])` where 'z' is not in cat.categories; or removing a category that was already removed in a prior step.","commonSituations":"Hardcoding removal lists from an external schema that has drifted from the data; chaining removes without re-checking remaining categories.","solutions":["Intersect with current categories: `cat.remove_categories([c for c in removals if c in cat.categories])`.","Check membership first: `to_remove = set(removals) & set(cat.categories)` then `cat.remove_categories(list(to_remove))`.","If you want to also drop rows, filter the data rather than removing categories."],"exampleFix":"# before\ncat = pd.Categorical(['a','b'], categories=['a','b'])\ncat.remove_categories(['a','z'])\n# after\ncat = pd.Categorical(['a','b'], categories=['a','b'])\ncat.remove_categories(['a'])","handlingStrategy":"validation","validationCode":"def safe_remove_categories(cat, removals):\n    missing = set(removals) - set(cat.categories)\n    if missing:\n        raise ValueError(f\"not categories: {missing}\")\n    return cat.remove_categories(removals)","typeGuard":"def all_removable_categories(cat, removals) -> bool:\n    return set(removals).issubset(set(cat.categories))","tryCatchPattern":"try:\n    cat = cat.remove_categories(removals)\nexcept ValueError as e:\n    if 'must all be in old categories' in str(e):\n        cat = cat.remove_categories([c for c in removals if c in cat.categories])\n    else:\n        raise","preventionTips":["Intersect removals with current categories before calling remove_categories.","Avoid hardcoding removal lists from external schemas; re-derive from data.","Re-check categories between chained removals."],"tags":["categorical","remove-categories","missing","valueerror"],"analyzedSha":"71959b8cb9b2459c16e14b34f28b178ccfe14735","analyzedAt":"2026-08-07T01:30:20.476Z","schemaVersion":2},"datasetVersion":"2026-08-07T03:17:09.362Z"}