comparing categoricals with different Categories is not supp

Error message

comparing categoricals with different Categories is not supported through AnyValue

What it means

PartialEq for AnyValue refuses to compare two Categorical values whose categories mappings are different Arc allocations (Arc::ptr_eq fails). The Hash implementation hashes the raw category index, so allowing cross-map equality would silently break hash/eq consistency; hence the explicit unimplemented!() guard.

Source

Thrown at crates/polars-core/src/datatypes/any_value.rs:1266

            (Float32(l), Float32(r)) => l.to_total_ord() == r.to_total_ord(),
            (Float64(l), Float64(r)) => l.to_total_ord() == r.to_total_ord(),
            (String(l), String(r)) => l == r,
            (Binary(l), Binary(r)) => l == r,
            #[cfg(feature = "dtype-time")]
            (Time(l), Time(r)) => *l == *r,
            #[cfg(all(feature = "dtype-datetime", feature = "dtype-date"))]
            (Date(l), Date(r)) => *l == *r,
            #[cfg(all(feature = "dtype-datetime", feature = "dtype-date"))]
            (Datetime(l, tul, tzl), Datetime(r, tur, tzr)) => {
                *l == *r && *tul == *tur && tzl == tzr
            },
            (List(l), List(r)) => l == r,
            #[cfg(feature = "dtype-categorical")]
            (Categorical(cat_l, map_l), Categorical(cat_r, map_r)) => {
                if !Arc::ptr_eq(map_l, map_r) {
                    // We can't support this because our Hash impl directly hashes the index. If you
                    // add support for this we must change the Hash impl.
                    unimplemented!(
                        "comparing categoricals with different Categories is not supported through AnyValue"
                    );
                }

                cat_l == cat_r
            },
            #[cfg(feature = "dtype-categorical")]
            (Enum(cat_l, map_l), Enum(cat_r, map_r)) => {
                if !Arc::ptr_eq(map_l, map_r) {
                    // We can't support this because our Hash impl directly hashes the index. If you
                    // add support for this we must change the Hash impl.
                    unimplemented!(
                        "comparing enums with different FrozenCategories is not supported through AnyValue"
                    );
                }

                cat_l == cat_r
            },

View on GitHub (pinned to df599052da)

Solutions

Cast both sides to String before comparing values: df1["cat"].cast(pl.String) == df2["cat"].cast(pl.String)
Make the columns share categories before comparing: join/concat with the same string cache (with pl.StringCache(): ... in Python) or remap to a single categories set
Compare physical indices only when you have verified both columns use the same mapping

Example fix

# before
same = df1["cat"].get(0) == df2["cat"].get(0)  # panics if mappings differ

# after
same = str(df1["cat"].cast(pl.String).get(0)) == str(df2["cat"].cast(pl.String).get(0))

Defensive patterns

Strategy: validation

Validate before calling

# Compare as strings unless both sides share one dtype object
def categorical_values_comparable(a: pl.Series, b: pl.Series) -> bool:
    return a.dtype == b.dtype  # same dtype instance implies shared mapping

Type guard

def same_categories(a: pl.Series, b: pl.Series) -> bool:
    return (
        isinstance(a.dtype, pl.Categorical)
        and isinstance(b.dtype, pl.Categorical)
        and a.dtype == b.dtype
    )

Try / catch

try:
    eq = av1 == av2
except Exception:
    eq = str(av1.cast(pl.String)) == str(av2.cast(pl.String))  # polars PanicException in Python

Prevention

Wrap categorical creation/joins in a single with pl.StringCache(): block so maps are shared
Cast categorical columns to String before cross-frame value comparisons
Concatenate categorical frames before comparing so one mapping is established

When it happens

Trigger: Comparing two categorical AnyValues that come from columns with different (non-shared) RevMappings: df1["cat"].get(0) == df2["cat"].get(0), equality-based dedup or is_in checks across two independently created categorical columns, or scalar comparisons inside generic user code.

Common situations: Two DataFrames each with their own Categorical column (no shared string cache), comparing values between them after concat/join, or a value from one frame compared against a freshly created categorical literal.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/042bf1203e38ff3e. Report an issue: GitHub.