pola-rs/polars · error

not implemented

Error message

not implemented

What it means

polars-arrow's concatenate kernel (compute::concatenate::concatenate / concatenate_unchecked) dispatches on the array's physical type, and every supported type has a dedicated routine. The Union arm is a bare unimplemented!() (crates/polars-arrow/src/compute/concatenate.rs:108), so concatenating union arrays panics with 'not implemented' instead of returning a PolarsResult error. Empty input and a single non-empty array return early, so the panic needs at least two non-empty union arrays.

Source

Thrown at crates/polars-arrow/src/compute/concatenate.rs:108

        Null => Ok(Box::new(concatenate_null(arrays))),
        Boolean => Ok(Box::new(concatenate_bool(arrays))),
        Primitive(ptype) => {
            with_match_primitive_type_full!(ptype, |$T| {
                Ok(Box::new(concatenate_primitive::<$T, _>(arrays)))
            })
        },
        Binary => Ok(Box::new(concatenate_binary::<i32, _>(arrays)?)),
        LargeBinary => Ok(Box::new(concatenate_binary::<i64, _>(arrays)?)),
        Utf8 => Ok(Box::new(concatenate_utf8::<i32, _>(arrays)?)),
        LargeUtf8 => Ok(Box::new(concatenate_utf8::<i64, _>(arrays)?)),
        BinaryView => Ok(Box::new(concatenate_view::<[u8], _>(arrays))),
        Utf8View => Ok(Box::new(concatenate_view::<str, _>(arrays))),
        List => Ok(Box::new(concatenate_list::<i32, _>(arrays)?)),
        LargeList => Ok(Box::new(concatenate_list::<i64, _>(arrays)?)),
        FixedSizeBinary => Ok(Box::new(concatenate_fixed_size_binary(arrays)?)),
        FixedSizeList => Ok(Box::new(concatenate_fixed_size_list(arrays)?)),
        Struct => Ok(Box::new(concatenate_struct(arrays)?)),
        Union => unimplemented!(),
        Map => unimplemented!(),
        Dictionary(_) => unimplemented!(),
    }
}

fn concatenate_null<A: AsRef<dyn Array>>(arrays: &[A]) -> NullArray {
    let dtype = arrays[0].as_ref().dtype().clone();
    let total_len = arrays.iter().map(|arr| arr.as_ref().len()).sum();
    NullArray::new(dtype, total_len)
}

fn concatenate_bool<A: AsRef<dyn Array>>(arrays: &[A]) -> BooleanArray {
    let dtype = arrays[0].as_ref().dtype().clone();
    let (total_len, null_count) = len_null_count(arrays);
    let validity = concatenate_validities_with_len_null_count(arrays, total_len, null_count);

    let mut bitmap = BitmapBuilder::with_capacity(total_len);
    for arr in arrays {

View on GitHub (pinned to df599052da)

Solutions

1. Project away or drop union columns before concatenating
2. Cast the union column to a struct or string representation first, then concatenate
3. Branch on dtype().to_physical_type() and return a PolarsResult error for Union instead of letting the panic escape
4. Implement concatenate_union upstream or fall back to the arrow-rs concat kernel, which supports unions

Example fix

// before
let out = concatenate(&[&a, &b])?; // panics: Union => unimplemented!()

// after
use polars_arrow::datatypes::PhysicalType;
if a.dtype().to_physical_type() == PhysicalType::Union {
    polars_bail!(InvalidOperation: "concatenate of union arrays is not supported");
}
let out = concatenate(&[&a, &b])?;

use polars_arrow::datatypes::PhysicalType;
if arrays.iter().any(|a| a.dtype().to_physical_type() == PhysicalType::Union) {
    polars_bail!(InvalidOperation: "union columns cannot be concatenated");
}

fn is_concatenable(dtype: &ArrowDataType) -> bool {
    !matches!(
        dtype.to_physical_type(),
        PhysicalType::Union | PhysicalType::Map | PhysicalType::Dictionary(_)
    )
}

let res = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| concatenate(&arrays)));
let out = match res {
    Ok(v) => v?,
    Err(_) => polars_bail!(ComputeError: "concatenate panicked: unsupported dtype (Union/Map/Dictionary)"),
};

When it happens

Trigger: Calling concatenate(&[&a, &b]) or concatenate_unchecked with two or more non-empty arrays whose dtype is ArrowDataType::Union(_) — e.g. polars vstack/diag_concat/rechunk over batches containing a union column.

Common situations: Interop with engines that emit union columns (pyarrow, DataFusion, Spark-on-Arrow); reading Arrow IPC/Feather files with unions then concatenating record batches; tests that hand-build union arrays.

Related errors

• not implemented
• not implemented
• horizontal_flatten not supported for data type {:?}
• invalid or out-of-range datetime
• out-of-range date

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/a76069ec54b211a3. Report an issue: GitHub.