pola-rs/polars · error

i64 offsets do not fit in i32 offsets

Error message

i64 offsets do not fit in i32 offsets

What it means

expect in create_list when narrowing i64 row-offsets to i32 for an IPC ListArray: the total offsets exceed i32 range, so the array cannot be represented as a (non-large) list. Signals data larger than the 32-bit offset layout allows; read as LargeList.

Source

Thrown at crates/polars-parquet/src/arrow/read/deserialize/mod.rs:64

}

/// Creates a new [`ListArray`] or [`FixedSizeListArray`].
pub fn create_list(
    dtype: ArrowDataType,
    nested: &mut NestedState,
    values: Box<dyn Array>,
) -> Box<dyn Array> {
    let (length, mut offsets, validity) = nested.pop().unwrap();
    let validity = validity.and_then(freeze_validity);
    match dtype.to_storage() {
        ArrowDataType::List(_) => {
            offsets.push(values.len() as i64);

            let offsets = offsets.iter().map(|x| *x as i32).collect::<Vec<_>>();

            let offsets: Offsets<i32> = offsets
                .try_into()
                .expect("i64 offsets do not fit in i32 offsets");

            Box::new(ListArray::<i32>::new(
                dtype,
                offsets.into(),
                values,
                validity,
            ))
        },
        ArrowDataType::LargeList(_) => {
            offsets.push(values.len() as i64);

            Box::new(ListArray::<i64>::new(
                dtype,
                offsets.try_into().expect("List too large"),
                values,
                validity,
            ))
        },

View on GitHub (pinned to 9b5d73fd00)

Solutions

1. The large (i64) list offsets exceed i32 range; split the data into smaller chunks or use large-list types end-to-end.
2. Filter/reduce the column so total element count fits in i32 offsets.

When it happens

Trigger: This panic/expect fires when execution reaches an unguarded state described by: "i64 offsets do not fit in i32 offsets". Typical triggers: unsupported dtype or feature-gated code path reached at runtime, invalid user input or environment variable value, calling API methods in the wrong order or on mismatched types, or data (lengths, offsets, ranges) violating the function's preconditions.

Common situations: Encountered when input data or configuration does not meet the preconditions of the throwing code path ("i64 offsets do not fit in i32 offsets"). Common cases: missing Cargo feature flags, mismatched dtypes/lengths between arrays or series, out-of-range temporal values, malformed environment variables, or operations on unsupported/complex nested types.

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/ccf5e80b0a1511c4. Report an issue: GitHub.