pola-rs/polars · error

horizontal_flatten not supported for data type {:?}

Error message

horizontal_flatten not supported for data type {:?}

What it means

Panic from the horizontal_flatten kernel in polars-compute, the low-level routine behind concat_arr (horizontal concatenation into a fixed-size-list / Array dtype). The match on the Arrow physical type only implements Null, Boolean, Primitive, LargeBinary, Struct, LargeList, FixedSizeList, BinaryView and Utf8View; any other physical layout reaches unimplemented!(). Typical leftovers are the legacy Utf8/LargeUtf8 string layouts, Map, Dictionary, or Extension types.

Source

Thrown at crates/polars-compute/src/horizontal_flatten/mod.rs:125

                        .downcast_ref::<BinaryViewArray>()
                        .unwrap()
                        .clone()
                })
                .collect::<Vec<_>>(),
            widths,
            output_height,
            dtype,
        )),
        Utf8View => Box::new(horizontal_flatten_unchecked_impl_generic(
            &arrays
                .iter()
                .map(|x| x.as_any().downcast_ref::<Utf8ViewArray>().unwrap().clone())
                .collect::<Vec<_>>(),
            widths,
            output_height,
            dtype,
        )),
        t => unimplemented!("horizontal_flatten not supported for data type {:?}", t),
    }
}

unsafe fn horizontal_flatten_unchecked_impl_generic<T>(
    arrays: &[T],
    widths: &[usize],
    output_height: usize,
    dtype: &ArrowDataType,
) -> T
where
    T: StaticArray,
{
    assert!(!arrays.is_empty());
    assert_eq!(widths.len(), arrays.len());

    debug_assert!(widths.iter().all(|x| *x > 0));
    debug_assert!(
        arrays

View on GitHub (pinned to df599052da)

Solutions

  1. Cast the inner columns to a supported dtype before concatenating (String and Binary are fine because they map to Utf8View/BinaryView in current polars)
  2. Check the physical type of inputs first: use the modern String dtype instead of legacy Utf8/LargeUtf8 layouts
  3. Upgrade polars - newer kernels cover more physical types
  4. If you control the data, rebuild the arrays so Map/Dictionary/Extension are exploded into Struct/Utf8View before concat_arr

Example fix

# before
pl.concat_arr([
    df.select(pl.col("legacy_str").cast(pl.Utf8)).to_series(),  # legacy layout
    df["vals"]
])

# after
pl.concat_arr([
    df["legacy_str"].cast(pl.String),  # Utf8View physical type, supported
    df["vals"]
])
Defensive patterns

Strategy: validation

Validate before calling

# Python: verify inner physical layout before concat_arr-style ops
SUPPORTED_INNER = {pl.Null, pl.Boolean, pl.String, pl.Binary}
SUPPORTED_INNER |= {dt for dt in pl.INTEGER_DTYPES + pl.FLOAT_DTYPES}
inner = df["col"].dtype
if isinstance(inner, pl.List):
    inner = inner.inner
if isinstance(inner, pl.Array):
    inner = inner.inner
assert inner in SUPPORTED_INNER or inner.is_numeric(), f"unsupported inner dtype {inner}"

Type guard

def concat_arr_safe(cols: list[pl.Series]) -> bool:
    inner = cols[0].dtype
    ok = inner.is_numeric() or inner in (pl.Boolean, pl.String, pl.Binary, pl.Null)
    ok = ok and all(c.dtype.base_type() == inner.base_type() for c in cols)
    if isinstance(inner, (pl.List, pl.Struct, pl.Array)):
        ok = ok and concat_arr_safe([pl.Series([v]) for v in []]) or True
    return ok

Try / catch

try:
    out = pl.concat_arr(cols)
except pl.exceptions.PanicException:
    # unsupported physical layout: cast inputs and retry
    out = pl.concat_arr([c.cast(pl.String) if c.dtype == pl.Utf8 else c for c in cols])

Prevention

When it happens

Trigger: Calling a concat_arr-based API (e.g. pl.concat_arr / expressions that build DataType::Array columns) where the inner values array materializes with a physical type not in the supported set - for example after casting a String column to legacy Utf8, or with Map/Dictionary/Extension inner dtypes.

Common situations: Custom casting pipelines that force old Arrow string layouts, interop code that produces Map or Dictionary arrays, or older polars versions where String was not yet Utf8View-based.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/fd49c4d687793339. Report an issue: GitHub.