pola-rs/polars · error

offset to fit in `usize`

Error message

offset to fit in `usize`

What it means

In crates/polars-arrow/src/ffi/array.rs `create_bitmap` converts `array.offset` (an `i64` from the C Data Interface) with `.try_into().expect("offset to fit in `usize`")`. A negative offset — or an offset exceeding usize width on 32-bit targets — cannot become a Rust slice offset, so the import panics. Per the Arrow spec offset must be >= 0; the expect enforces that contract.

Source

Thrown at crates/polars-arrow/src/ffi/array.rs:337

unsafe fn create_bitmap(
    array: &ArrowArray,
    dtype: &ArrowDataType,
    owner: InternalArrowArray,
    index: usize,
    // if this is the validity bitmap
    // we can use the null count directly
    is_validity: bool,
) -> PolarsResult<Bitmap> {
    let len: usize = array.length.try_into().expect("length to fit in `usize`");
    if len == 0 {
        // Zero-length arrays might have invalid pointers for zero-length slices in Rust,
        // so this is more than just an optimization.
        return Ok(Bitmap::new());
    }
    let ptr = get_buffer_ptr(array, dtype, index)?;

    // Pointer of u8 has alignment 1, so we don't have to check alignment.
    let offset: usize = array.offset.try_into().expect("offset to fit in `usize`");
    let bytes_len = bytes_for(offset + len);
    let slice = core::slice::from_raw_parts(ptr, bytes_len);
    let storage = SharedStorage::from_slice_with_owner(slice, owner);

    let null_count = if is_validity {
        Some(array.null_count())
    } else {
        None
    };
    Ok(Bitmap::from_inner_unchecked(
        storage, offset, len, null_count,
    ))
}

fn buffer_offset(array: &ArrowArray, dtype: &ArrowDataType, i: usize) -> usize {
    use PhysicalType::*;
    match (dtype.to_physical_type(), i) {
        (LargeUtf8, 2) | (LargeBinary, 2) | (Utf8, 2) | (Binary, 2) => 0,

View on GitHub (pinned to 9b5d73fd00)

Solutions

1. Fix the exporter so `offset` is always a non-negative i64 within the buffer.
2. Pre-validate in the producer: assert offset >= 0 and offset + length <= buffer capacity before release.
3. Catch panics with `catch_unwind` around `from_ffi` at the interop boundary and surface a descriptive error.
4. Log the raw ArrowArray fields (length/offset/null_count) in the exporter when debugging to confirm which side violates the contract.

Example fix

// before
let array = unsafe { from_ffi(imported, owner) }?; // panics: offset to fit in `usize`

// after
let array = std::panic::catch_unwind(|| unsafe { from_ffi(imported, owner) })
    .map_err(|_| polars_err!(ComputeError: "invalid ArrowArray.offset from foreign producer"))??;

// Producer-side pre-check (Rust exporter of your own ArrowArray):
// assert!(offset >= 0 && length >= 0 && offset + length <= total_slots);
// Consumer-side: fields are pub(super), so pre-validation isn't possible — isolate with catch_unwind.

let array = std::panic::catch_unwind(|| unsafe { polars_arrow::ffi::from_ffi(imported, owner) })
    .map_err(|_| polars_err!(ComputeError: "foreign ArrowArray has invalid offset; check slice math"))?;

When it happens

Trigger: `from_ffi`/`try_from_ffi` on an ArrowArray with a negative `offset` field, typically an uninitialized or corrupted struct from a foreign producer; or a sliced array exported by buggy code that computed the offset as a signed subtraction underflow.

Common situations: Hand-written C/C++ bridges that export sliced buffers (offset = buffer_start - array_start can underflow); producers using a different major version of the C data interface with mismatched struct layout, so Rust reads garbage for offset; partial memset of the struct.

Related errors

• length to fit in `usize`
• Offset to fit in `usize`
• The external API has a non-utf8 as format
• not implemented
• The schema declaration does not match the deserialization

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/bc9af2becd522bfe. Report an issue: GitHub.