pola-rs/polars · error

Union struct must be created with the corresponding Union Da

Error message

Union struct must be created with the corresponding Union DataType

What it means

UnionArray::new_null builds a null union only from ArrowDataType::Union(..): it materializes one null child per field, sets types = 0 for every row and, for dense mode, offsets 0..length. Any other dtype panics with this message.

Source

Thrown at crates/polars-arrow/src/array/union/mod.rs:192

        if let ArrowDataType::Union(u) = &dtype {
            let fields = u
                .fields
                .iter()
                .map(|x| new_null_array(x.dtype().clone(), length))
                .collect();

            let offsets = if u.mode.is_sparse() {
                None
            } else {
                Some((0..length as i32).collect::<Vec<_>>().into())
            };

            // all from the same field
            let types = vec![0i8; length].into();

            Self::new(dtype, types, fields, offsets)
        } else {
            panic!("Union struct must be created with the corresponding Union DataType")
        }
    }

    /// Creates a new empty [`UnionArray`].
    pub fn new_empty(dtype: ArrowDataType) -> Self {
        if let ArrowDataType::Union(u) = dtype.to_storage() {
            let fields = u
                .fields
                .iter()
                .map(|x| new_empty_array(x.dtype().clone()))
                .collect();

            let offsets = if u.mode.is_sparse() {
                None
            } else {
                Some(Buffer::default())
            };

View on GitHub (pinned to 9b5d73fd00)

Solutions

1. Use polars_arrow::array::new_null_array(dtype, length), which handles Union correctly
2. Ensure the dtype passed is literally ArrowDataType::Union(fields, mode) before calling the concrete constructor
3. Add a Union arm to generic builders so the concrete constructor is never reached with a foreign dtype

Example fix

// before
let nulls = UnionArray::new_null(dtype, len);

// after
let nulls = polars_arrow::array::new_null_array(dtype, len);

fn union_only(dtype: &ArrowDataType) -> Result<&ArrowDataType, String> {
    match dtype.to_storage() {
        ArrowDataType::Union(_) => Ok(dtype),
        other => Err(format!("expected Union dtype, got {other:?}")),
    }
}

fn is_union_dtype(dtype: &ArrowDataType) -> bool {
    matches!(dtype.to_storage(), ArrowDataType::Union(_))
}

When it happens

Trigger: UnionArray::new_null(dtype, n) where dtype is not the Union variant — e.g. a Struct dtype, or a union column represented by a different variant after a schema round-trip.

Common situations: Generic null-column factories whose match forgot the union arm; IPC/readers constructing null padding rows for union columns for the first time.

Related errors

• StructArray must be initialized with DataType::Struct
• cannot set validity of a union array
• MutableUtf8ValuesArray can only be initialized with DataType
• not implemented
• The schema declaration does not match the deserialization

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/89249507193ac650. Report an issue: GitHub.