pola-rs/polars · error

Could not 'unwrap_required'. 'ZipValidity' iterator has null

Error message

Could not 'unwrap_required'. 'ZipValidity' iterator has nulls.

What it means

ZipValidity is an enum: Required(iter) wraps iteration over an array with no validity; Optional(iter) yields Option<T> and can produce nulls. unwrap_required() is the fast path that asserts the Required variant; calling it on an Optional value panics because nulls cannot be represented in the unwrapped iterator.

Source

Thrown at crates/polars-arrow/src/bitmap/utils/zip_validity.rs:206

}

unsafe impl<T, I, V> TrustedLen for ZipValidity<T, I, V>
where
    I: TrustedLen<Item = T>,
    V: TrustedLen<Item = bool>,
{
}

impl<T, I, V> ZipValidity<T, I, V>
where
    I: Iterator<Item = T>,
    V: Iterator<Item = bool>,
{
    /// Unwrap into an iterator that has no null values.
    pub fn unwrap_required(self) -> I {
        match self {
            ZipValidity::Required(i) => i,
            _ => panic!("Could not 'unwrap_required'. 'ZipValidity' iterator has nulls."),
        }
    }

    /// Unwrap into an iterator that has null values.
    pub fn unwrap_optional(self) -> ZipValidityIter<T, I, V> {
        match self {
            ZipValidity::Optional(i) => i,
            _ => panic!("Could not 'unwrap_optional'. 'ZipValidity' iterator has no nulls."),
        }
    }
}

View on GitHub (pinned to 9b5d73fd00)

Solutions

1. Gate the fast path: if arr.validity().is_none() { ...unwrap_required()... } else { fall back to Option handling }
2. Handle nulls explicitly: keep the Optional iterator and map/flatten the Option items
3. Centralize the guard in a helper so every call site of unwrap_required is protected

Example fix

// before
let vals: Vec<&str> = iter.unwrap_required().collect();

// after
let vals: Vec<&str> = if arr.validity().is_none() {
    iter.unwrap_required().collect()
} else {
    iter.flatten().collect()
};

fn is_dense(arr: &dyn polars_arrow::array::Array) -> bool {
    arr.validity().is_none()
}
// only take the unwrap_required fast path when is_dense(&arr)

fn can_unwrap_required<T, I, V>(zv: &polars_arrow::bitmap::utils::ZipValidity<T, I, V>) -> bool {
    matches!(zv, polars_arrow::bitmap::utils::ZipValidity::Required(_))
}

When it happens

Trigger: arr_iter.unwrap_required() on the ZipValidity produced for an array whose validity().is_some(); the classic caller is a dense fast path that skipped its validity().is_none() guard.

Common situations: Performance-tuned loops assuming a column is fully dense; after a filter, left join, or strided slice introduces the first nulls into a previously dense column; data-dependent panics that only fire on some inputs.

Related errors

• Could not 'unwrap_optional'. 'ZipValidity' iterator has no n
• activate 'dtype-array'
• StructArray must be initialized with DataType::Struct
• Union struct must be created with the corresponding Union Da
• cannot set validity of a union array

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/8e9c0b672308657d. Report an issue: GitHub.