{"record":{"id":"ac5b23bc6116749f","repo":"pola-rs/polars","slug":"cannot-set-validity-of-a-union-array","errorCode":null,"errorMessage":"cannot set validity of a union array","messagePattern":"cannot set validity of a union array","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/polars-arrow/src/array/union/mod.rs","lineNumber":349,"sourceCode":"    pub unsafe fn value_unchecked(&self, index: usize) -> Box<dyn Scalar> {\n        debug_assert!(index < self.len());\n        let (type_, index) = self.index_unchecked(index);\n        // SAFETY: assumption of the struct\n        debug_assert!(type_ < self.fields.len());\n        let field = self.fields.get_unchecked(type_).as_ref();\n        new_scalar(field, index)\n    }\n}\n\nimpl Array for UnionArray {\n    impl_common_array!();\n\n    fn validity(&self) -> Option<&Bitmap> {\n        None\n    }\n\n    fn with_validity(&self, _: Option<Bitmap>) -> Box<dyn Array> {\n        panic!(\"cannot set validity of a union array\")\n    }\n}\n\nimpl UnionArray {\n    fn try_get_all(dtype: &ArrowDataType) -> PolarsResult<UnionComponents<'_>> {\n        match dtype.to_storage() {\n            ArrowDataType::Union(u) => Ok((&u.fields, u.ids.as_ref().map(|x| x.as_ref()), u.mode)),\n            _ => polars_bail!(ComputeError:\n                \"The UnionArray requires a logical type of DataType::Union\",\n            ),\n        }\n    }\n\n    fn get_all(dtype: &ArrowDataType) -> (&[Field], Option<&[i32]>, UnionMode) {\n        Self::try_get_all(dtype).unwrap()\n    }\n\n    /// Returns all fields from [`ArrowDataType::Union`].","sourceCodeStart":331,"sourceCodeEnd":367,"githubUrl":"https://github.com/pola-rs/polars/blob/9b5d73fd00236295624374b075d16b1fe6ec6df9/crates/polars-arrow/src/array/union/mod.rs#L331-L367","documentation":"UnionArray's Array impl hard-codes with_validity to panic and validity() to return None: the Arrow union layout has no top-level validity bitmap, so attaching one is rejected rather than silently dropped. Nullability lives in the child fields.","triggerScenarios":"Generic code doing arr.with_validity(Some(bitmap)) on Box<dyn Array>/&dyn Array when the concrete array is a UnionArray — typical in filter/zip/concat-style kernels that recompute a validity and re-apply it uniformly to the output.","commonSituations":"Reusing a kernel across a heterogeneous schema that includes a union column; porting code from arrays that do support validity; wrapping foreign arrow union data into polars pipelines.","solutions":["Branch first: if matches!(arr.dtype().to_physical_type(), PhysicalType::Union), skip validity handling for that array","Model top-level nullability by wrapping the union in a StructArray that carries the validity, or make the relevant union child field nullable","In generic pipelines, apply validity to child arrays (zip_validity-style) instead of blind with_validity on the top level"],"exampleFix":"// before\nlet out = arr.with_validity(Some(validity));\n\n// after\nlet out = if matches!(arr.dtype().to_physical_type(), PhysicalType::Union) {\n    arr.to_boxed() // unions cannot carry top-level validity\n} else {\n    arr.with_validity(Some(validity))\n};","handlingStrategy":"type-guard","validationCode":"fn can_set_validity(arr: &dyn polars_arrow::array::Array) -> bool {\n    !matches!(arr.dtype().to_physical_type(), polars_arrow::datatypes::PhysicalType::Union)\n}","typeGuard":"fn is_union_array(arr: &dyn polars_arrow::array::Array) -> bool {\n    matches!(arr.dtype().to_physical_type(), polars_arrow::datatypes::PhysicalType::Union)\n}","tryCatchPattern":null,"preventionTips":["Branch on physical type before with_validity in generic kernels","Encode top-level nullability for unions via a wrapping StructArray or nullable child fields","Assume with_validity is opt-in per layout, not a universal Array operation"],"tags":["rust","polars","arrow","union","validity"],"backgroundTag":"validity-not-supported","analyzedSha":"9b5d73fd00236295624374b075d16b1fe6ec6df9","analyzedAt":"2026-08-19T12:15:06.350Z","contentChangedAt":"2026-08-19T12:15:06.350Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}