pola-rs/polars · error

merge-join kernel not implemented for {:?}

Error message

merge-join kernel not implemented for {:?}

What it means

The streaming-engine merge-join kernel (used for sorted equi-joins / IEJoin-style plans) only implements key dtypes that are primitive numeric, Boolean, String, Binary, BinaryOffset, Enum, or Null. Any other key dtype - notably Categorical (unlike Enum), temporal dtypes not pre-cast to physical, and other nested/exotic types - hits the unimplemented!() fallback.

Source

Thrown at crates/polars-ops/src/frame/join/merge_join.rs:84

            type PhysCa = ChunkedArray<<$C as PolarsCategoricalType>::PolarsPhysical>;
            let build_keys_ca: &PhysCa = build_keys.as_ref().as_ref();
            dispatch!(build_keys_ca)
        }),
        DataType::Null => match_null_keys_impl(
            build_keys.len(),
            probe_keys.len(),
            gather_build,
            gather_probe,
            gather_probe_unmatched,
            build_emit_unmatched,
            descending,
            nulls_equal,
            limit_results,
            build_row_offset,
            probe_row_offset,
            probe_last_matched,
        ),
        dt => unimplemented!("merge-join kernel not implemented for {:?}", dt),
    }
}

#[allow(clippy::mut_range_bound, clippy::too_many_arguments)]
fn match_keys_impl<'a, T: PolarsDataType>(
    build_keys: &'a ChunkedArray<T>,
    probe_keys: &'a ChunkedArray<T>,
    gather_build: &mut Vec<IdxSize>,
    gather_probe: &mut Vec<IdxSize>,
    mut gather_probe_unmatched: Option<&mut Vec<IdxSize>>,
    build_emit_unmatched: bool,
    descending: bool,
    nulls_equal: bool,
    limit_results: usize,
    build_row_offset: &mut usize,
    probe_row_offset: &mut usize,
    probe_first_unmatched: &mut usize,
) where

View on GitHub (pinned to df599052da)

Solutions

  1. Cast join keys to a supported dtype before the join: .cast(pl.String) for categoricals, or to Int64 physical for temporals if needed
  2. Convert categorical keys to a shared Enum or String dtype on both sides before joining
  3. Run the query with the in-memory engine (engine='in-memory') as a workaround while keys are non-standard

Example fix

# before
out = df1.join(df2, on="cat_key")  # categorical keys + streaming merge-join -> panic

# after
out = df1.with_columns(pl.col("cat_key").cast(pl.String)).join(
    df2.with_columns(pl.col("cat_key").cast(pl.String)), on="cat_key"
)
Defensive patterns

Strategy: validation

Validate before calling

MERGE_JOIN_KEYS = (
    set(pl.INTEGER_DTYPES) | set(pl.FLOAT_DTYPES)
    | {pl.Boolean, pl.String, pl.Binary, pl.Null}
)
def merge_join_keys_ok(*keys: pl.Series) -> bool:
    return all(k.dtype in MERGE_JOIN_KEYS or isinstance(k.dtype, pl.Enum) for k in keys)

Type guard

def merge_join_key(s: pl.Series) -> pl.Series:
    if not (s.dtype.is_numeric() or s.dtype in (pl.Boolean, pl.String, pl.Binary, pl.Null)) \
       and not isinstance(s.dtype, pl.Enum):
        return s.cast(pl.String)
    return s

Prevention

When it happens

Trigger: Running a join under the streaming engine (engine='streaming', or default in recent versions) whose join keys have an unsupported dtype, e.g. joining on a Categorical key column that has not been unified to Enum/String/physical representation.

Common situations: Joins on string-cache categoricals or on keys whose dtype changed after concat/concat_relaxed; queries that only panic once the planner picks the merge-join path (sorted inputs, ordered joins).

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/f711f1a53eb0b06a. Report an issue: GitHub.