pola-rs/polars · error

List too large

Error message

List too large

What it means

expect when building an IPC LargeList whose Offsets buffer cannot be constructed (offset overflow beyond i64 limits). Practically unreachable for real data; indicates corrupt or absurd offset values during deserialization.

Source

Thrown at crates/polars-parquet/src/arrow/read/deserialize/mod.rs:78

            let offsets = offsets.iter().map(|x| *x as i32).collect::<Vec<_>>();

            let offsets: Offsets<i32> = offsets
                .try_into()
                .expect("i64 offsets do not fit in i32 offsets");

            Box::new(ListArray::<i32>::new(
                dtype,
                offsets.into(),
                values,
                validity,
            ))
        },
        ArrowDataType::LargeList(_) => {
            offsets.push(values.len() as i64);

            Box::new(ListArray::<i64>::new(
                dtype,
                offsets.try_into().expect("List too large"),
                values,
                validity,
            ))
        },
        ArrowDataType::FixedSizeList(_, _) => {
            Box::new(FixedSizeListArray::new(dtype, length, values, validity))
        },
        _ => unreachable!(),
    }
}

/// Creates a new [`MapArray`].
pub fn create_map(
    dtype: ArrowDataType,
    nested: &mut NestedState,
    values: Box<dyn Array>,
) -> Box<dyn Array> {
    let (_, mut offsets, validity) = nested.pop().unwrap();

View on GitHub (pinned to 9b5d73fd00)

Solutions

1. The list column exceeds the maximum supported size; chunk the data or use LargeList dtype.
2. Reduce the number of elements per list or split the column.

When it happens

Trigger: This panic/expect fires when execution reaches an unguarded state described by: "List too large". Typical triggers: unsupported dtype or feature-gated code path reached at runtime, invalid user input or environment variable value, calling API methods in the wrong order or on mismatched types, or data (lengths, offsets, ranges) violating the function's preconditions.

Common situations: Encountered when input data or configuration does not meet the preconditions of the throwing code path ("List too large"). Common cases: missing Cargo feature flags, mismatched dtypes/lengths between arrays or series, out-of-range temporal values, malformed environment variables, or operations on unsupported/complex nested types.

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/eac045005f67a427. Report an issue: GitHub.