{"record":{"id":"63795a777278ec44","repo":"pola-rs/polars","slug":"length-to-fit-in-usize","errorCode":null,"errorMessage":"length to fit in `usize`","messagePattern":"length to fit in `usize`","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/polars-arrow/src/ffi/array.rs","lineNumber":328,"sourceCode":"        Ok(Buffer::from(v))\n    }\n}\n\n/// returns the buffer `i` of `array` interpreted as a [`Bitmap`].\n/// # Safety\n/// This function is safe iff:\n/// * the buffer at position `index` is valid for the declared length\n/// * the buffers' pointer is not mutable for the lifetime of `owner`\nunsafe fn create_bitmap(\n    array: &ArrowArray,\n    dtype: &ArrowDataType,\n    owner: InternalArrowArray,\n    index: usize,\n    // if this is the validity bitmap\n    // we can use the null count directly\n    is_validity: bool,\n) -> PolarsResult<Bitmap> {\n    let len: usize = array.length.try_into().expect(\"length to fit in `usize`\");\n    if len == 0 {\n        // Zero-length arrays might have invalid pointers for zero-length slices in Rust,\n        // so this is more than just an optimization.\n        return Ok(Bitmap::new());\n    }\n    let ptr = get_buffer_ptr(array, dtype, index)?;\n\n    // Pointer of u8 has alignment 1, so we don't have to check alignment.\n    let offset: usize = array.offset.try_into().expect(\"offset to fit in `usize`\");\n    let bytes_len = bytes_for(offset + len);\n    let slice = core::slice::from_raw_parts(ptr, bytes_len);\n    let storage = SharedStorage::from_slice_with_owner(slice, owner);\n\n    let null_count = if is_validity {\n        Some(array.null_count())\n    } else {\n        None\n    };","sourceCodeStart":310,"sourceCodeEnd":346,"githubUrl":"https://github.com/pola-rs/polars/blob/9b5d73fd00236295624374b075d16b1fe6ec6df9/crates/polars-arrow/src/ffi/array.rs#L310-L346","documentation":"When importing an array over the Arrow C Data Interface (crates/polars-arrow/src/ffi/array.rs), `ArrowArray.length` is an `i64` per the spec. `create_bitmap` does `array.length.try_into().expect(\"length to fit in `usize`\")`. On 64-bit targets this only fails for negative lengths; on 32-bit targets also for lengths above u32::MAX. A negative/garbage length from the foreign producer is treated as an unrecoverable invariant violation and panics.","triggerScenarios":"`from_ffi`/`try_from_ffi` on an `ArrowArray` whose `length` field is negative (uninitialized or corrupted C struct) or, on 32-bit builds, larger than usize. The panic fires while building the validity/offset bitmap for any array with non-zero length, including validity-only paths.","commonSituations":"FFI interop with C/C++/Golang/Java producers: passing an ArrowArray struct that wasn't fully initialized (e.g. zeroing only some fields, or exporting via a stale/mismatched ABI version), or memory corruption/missing owner lifetime on the producer side. Rare in correct pyarrow-based flows, more common in hand-written C bridges.","solutions":["Fix the producer: initialize every ArrowArray field per the C data interface spec (length >= 0, sane offset, release callback set).","Validate the struct in the producer/exporter before handing it to Rust.","Wrap `from_ffi` in `std::panic::catch_unwind` at the interop boundary and report which batch failed instead of crashing the process.","If on a 32-bit target with legitimately huge arrays, move to 64-bit — lengths that don't fit usize can't be handled at all."],"exampleFix":"// before (Rust consumer)\nlet array = unsafe { from_ffi(imported, owner) }?; // panics if producer sent negative length\n\n// after\nlet result = std::panic::catch_unwind(|| unsafe { from_ffi(imported, owner) });\nlet array = result.map_err(|_| polars_err!(ComputeError: \"foreign producer exported an ArrowArray with invalid length\"))??;","handlingStrategy":"try-catch","validationCode":"// ArrowArray fields are pub(super); validate on the producer side before export:\n// C/other-language producer: assert(array->length >= 0 && array->offset >= 0);\n// and (on 32-bit) assert((uint64_t)array->length <= UINT32_MAX);","typeGuard":null,"tryCatchPattern":"let array = std::panic::catch_unwind(|| unsafe { polars_arrow::ffi::from_ffi(imported, owner) })\n    .map_err(|_| polars_err!(ComputeError: \"foreign ArrowArray has invalid length; check producer\"))?;","preventionTips":["Fully initialize every ArrowArray field on export (length, offset, null_count, buffers, release).","Pin the same major version of the C Data Interface on both sides of the boundary.","Add producer-side assertions on length/offset before invoking the consumer.","Keep catch_unwind only at the interop boundary, and log which batch triggered it."],"tags":["rust","polars","arrow","ffi","c-data-interface","usize","overflow","panic"],"backgroundTag":"c-data-interface-invalid-metadata","analyzedSha":"9b5d73fd00236295624374b075d16b1fe6ec6df9","analyzedAt":"2026-08-19T12:15:06.350Z","contentChangedAt":"2026-08-19T12:15:06.350Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}