pandas-dev/pandas · error · TypeError

Categoricals can only be compared if 'categories' are the sa

Error message

Categoricals can only be compared if 'categories' are the same.

What it means

Raised when comparing two Categorical objects whose categories do not match up to a permutation (checked via _categories_match_up_to_permutation). For ordered comparisons the categories must be identical in identity/order; for unordered they must be the same set. Pandas refuses to silently compare codes that reference different category meanings.

Source

Thrown at pandas/core/arrays/categorical.py:150

    @unpack_zerodim_and_defer(opname)
    def func(self, other):
        hashable = is_hashable(other)
        if is_list_like(other) and len(other) != len(self) and not hashable:
            # in hashable case we may have a tuple that is itself a category
            raise ValueError("Lengths must match.")

        if not self.ordered:
            if opname in ["__lt__", "__gt__", "__le__", "__ge__"]:
                raise TypeError(
                    "Unordered Categoricals can only compare equality or not"
                )
        if isinstance(other, Categorical):
            # Two Categoricals can only be compared if the categories are
            # the same (maybe up to ordering, depending on ordered)

            msg = "Categoricals can only be compared if 'categories' are the same."
            if not self._categories_match_up_to_permutation(other):
                raise TypeError(msg)

            if not self.ordered and not self.categories.equals(other.categories):
                # both unordered and different order
                other_codes = recode_for_categories(
                    other.codes, other.categories, self.categories, copy=False
                )
            else:
                other_codes = other._codes

            ret = op(self._codes, other_codes)
            mask = (self._codes == -1) | (other_codes == -1)
            if mask.any():
                ret[mask] = fill_value
            return ret

        if hashable:
            if other in self.categories:
                i = self._unbox_scalar(other)

View on GitHub (pinned to 71959b8cb9)

Solutions

Unify categories first: `cat2 = cat2.cat.set_categories(cat1.categories)` (and ordered flag) before comparing.
Compare the underlying values instead: `np.asarray(cat1) == np.asarray(cat2)`.
Build both categoricals from a shared CategoricalDtype so categories are guaranteed identical.

Example fix

# before
c1 = pd.Categorical(['a','b'], categories=['a','b'])
c2 = pd.Categorical(['a','b'], categories=['a','b','c'])
c1 == c2
# after
c1 = pd.Categorical(['a','b'], categories=['a','b'])
c2 = pd.Categorical(['a','b'], categories=['a','b','c']).cat.remove_unused_categories()
c1 == c2

Defensive patterns

Strategy: validation

Validate before calling

def align_categories(cat1, cat2):
    if not cat1._categories_match_up_to_permutation(cat2):
        cat2 = cat2.cat.set_categories(cat1.categories, ordered=cat1.ordered)
    return cat2

# usage: cat2 = align_categories(cat1, cat2); cat1 == cat2

Type guard

def categories_match(cat1, cat2) -> bool:
    try:
        return cat1._categories_match_up_to_permutation(cat2)
    except Exception:
        return False

Try / catch

try:
    res = cat1 < cat2
except TypeError as e:
    if "categories" in str(e):
        cat2 = cat2.cat.set_categories(cat1.categories)
        res = cat1 < cat2
    else:
        raise

Prevention

Build both categoricals from one shared CategoricalDtype instance.
After concat/merge, harmonize categories with union_categoricals or set_categories.
Drop unused categories before comparing to reduce mismatch surface.

When it happens

Trigger: Evaluating `cat1 < cat2`, `cat1 == cat2`, etc. where both operands are Categorical but `cat1.categories` and `cat2.categories` differ in members or count (e.g. one has an extra category, or different dtypes).

Common situations: Comparing categorical columns from two DataFrames merged from different sources; concatenating categoricals with union_categories then comparing the pieces; dtype drift after astype('category') on different value sets.

Related errors

AI-assisted analysis of pandas-dev/pandas@71959b8cb9 (2026-08-07). Data as JSON: /api/errors/3598517defe9c86c. Report an issue: GitHub.