{"record":{"id":"bc9af2becd522bfe","repo":"pola-rs/polars","slug":"offset-to-fit-in-usize","errorCode":null,"errorMessage":"offset to fit in `usize`","messagePattern":"offset to fit in `usize`","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/polars-arrow/src/ffi/array.rs","lineNumber":337,"sourceCode":"unsafe fn create_bitmap(\n    array: &ArrowArray,\n    dtype: &ArrowDataType,\n    owner: InternalArrowArray,\n    index: usize,\n    // if this is the validity bitmap\n    // we can use the null count directly\n    is_validity: bool,\n) -> PolarsResult<Bitmap> {\n    let len: usize = array.length.try_into().expect(\"length to fit in `usize`\");\n    if len == 0 {\n        // Zero-length arrays might have invalid pointers for zero-length slices in Rust,\n        // so this is more than just an optimization.\n        return Ok(Bitmap::new());\n    }\n    let ptr = get_buffer_ptr(array, dtype, index)?;\n\n    // Pointer of u8 has alignment 1, so we don't have to check alignment.\n    let offset: usize = array.offset.try_into().expect(\"offset to fit in `usize`\");\n    let bytes_len = bytes_for(offset + len);\n    let slice = core::slice::from_raw_parts(ptr, bytes_len);\n    let storage = SharedStorage::from_slice_with_owner(slice, owner);\n\n    let null_count = if is_validity {\n        Some(array.null_count())\n    } else {\n        None\n    };\n    Ok(Bitmap::from_inner_unchecked(\n        storage, offset, len, null_count,\n    ))\n}\n\nfn buffer_offset(array: &ArrowArray, dtype: &ArrowDataType, i: usize) -> usize {\n    use PhysicalType::*;\n    match (dtype.to_physical_type(), i) {\n        (LargeUtf8, 2) | (LargeBinary, 2) | (Utf8, 2) | (Binary, 2) => 0,","sourceCodeStart":319,"sourceCodeEnd":355,"githubUrl":"https://github.com/pola-rs/polars/blob/9b5d73fd00236295624374b075d16b1fe6ec6df9/crates/polars-arrow/src/ffi/array.rs#L319-L355","documentation":"In crates/polars-arrow/src/ffi/array.rs `create_bitmap` converts `array.offset` (an `i64` from the C Data Interface) with `.try_into().expect(\"offset to fit in `usize`\")`. A negative offset — or an offset exceeding usize width on 32-bit targets — cannot become a Rust slice offset, so the import panics. Per the Arrow spec offset must be >= 0; the expect enforces that contract.","triggerScenarios":"`from_ffi`/`try_from_ffi` on an ArrowArray with a negative `offset` field, typically an uninitialized or corrupted struct from a foreign producer; or a sliced array exported by buggy code that computed the offset as a signed subtraction underflow.","commonSituations":"Hand-written C/C++ bridges that export sliced buffers (offset = buffer_start - array_start can underflow); producers using a different major version of the C data interface with mismatched struct layout, so Rust reads garbage for offset; partial memset of the struct.","solutions":["Fix the exporter so `offset` is always a non-negative i64 within the buffer.","Pre-validate in the producer: assert offset >= 0 and offset + length <= buffer capacity before release.","Catch panics with `catch_unwind` around `from_ffi` at the interop boundary and surface a descriptive error.","Log the raw ArrowArray fields (length/offset/null_count) in the exporter when debugging to confirm which side violates the contract."],"exampleFix":"// before\nlet array = unsafe { from_ffi(imported, owner) }?; // panics: offset to fit in `usize`\n\n// after\nlet array = std::panic::catch_unwind(|| unsafe { from_ffi(imported, owner) })\n    .map_err(|_| polars_err!(ComputeError: \"invalid ArrowArray.offset from foreign producer\"))??;","handlingStrategy":"try-catch","validationCode":"// Producer-side pre-check (Rust exporter of your own ArrowArray):\n// assert!(offset >= 0 && length >= 0 && offset + length <= total_slots);\n// Consumer-side: fields are pub(super), so pre-validation isn't possible — isolate with catch_unwind.","typeGuard":null,"tryCatchPattern":"let array = std::panic::catch_unwind(|| unsafe { polars_arrow::ffi::from_ffi(imported, owner) })\n    .map_err(|_| polars_err!(ComputeError: \"foreign ArrowArray has invalid offset; check slice math\"))?;","preventionTips":["Compute offsets with unsigned arithmetic on the producer to avoid signed underflow when slicing.","Never pass partially-initialized ArrowArray structs across FFI.","Round-trip test every FFI producer against a reference consumer (pyarrow) before shipping."],"tags":["rust","polars","arrow","ffi","c-data-interface","offset","usize","panic"],"backgroundTag":"c-data-interface-invalid-metadata","analyzedSha":"9b5d73fd00236295624374b075d16b1fe6ec6df9","analyzedAt":"2026-08-19T12:15:06.350Z","contentChangedAt":"2026-08-19T12:15:06.350Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}