pola-rs/polars · error

The length of the values must be equal to the last offset va

Error message

The length of the values must be equal to the last offset value

What it means

MutableUtf8ValuesArray::new_unchecked validates (despite the name) that the offsets buffer ends exactly at values.len() via try_check_offsets_bounds; a last offset smaller or larger than the values length returns Err and the .expect turns it into this panic.

Source

Thrown at crates/polars-arrow/src/array/utf8/mutable_values.rs:110

    ///
    /// # Panic
    /// This function does not panic iff:
    /// * `offsets.last()` is greater than `values.len()`
    /// * The `dtype`'s [`crate::datatypes::PhysicalType`] is equal to either `Utf8` or `LargeUtf8`.
    ///
    /// # Safety
    /// This function is safe iff:
    /// * the offsets are monotonically increasing
    /// * The `values` between two consecutive `offsets` are not valid utf8
    /// # Implementation
    /// This function is `O(1)`
    pub unsafe fn new_unchecked(
        dtype: ArrowDataType,
        offsets: Offsets<O>,
        values: Vec<u8>,
    ) -> Self {
        try_check_offsets_bounds(&offsets, values.len())
            .expect("The length of the values must be equal to the last offset value");

        if dtype.to_physical_type() != Self::default_dtype().to_physical_type() {
            panic!(
                "MutableUtf8ValuesArray can only be initialized with DataType::Utf8 or DataType::LargeUtf8"
            )
        }

        Self {
            dtype,
            offsets,
            values,
        }
    }

    /// Returns the default [`ArrowDataType`] of this container: [`ArrowDataType::Utf8`] or [`ArrowDataType::LargeUtf8`]
    /// depending on the generic [`Offset`].
    pub fn default_dtype() -> ArrowDataType {
        Utf8Array::<O>::default_dtype()

View on GitHub (pinned to 9b5d73fd00)

Solutions

1. Assert offsets.last_offset() == values.len() before constructing, and truncate or extend values to match
2. Rebuild the array by pushing each slice (push keeps offsets and values consistent)
3. When slicing, keep the offsets relative and resize the values buffer to offsets.last_offset()

Example fix

// before
let arr = unsafe { MutableUtf8ValuesArray::new_unchecked(dtype, offsets, values) };

// after: make the values buffer end exactly at the last offset
let last = offsets.last_offset();
let values = values[..last.min(values.len())].to_vec();
// pad if values.len() < last instead of truncating, then:
let arr = unsafe { MutableUtf8ValuesArray::new_unchecked(dtype, offsets, values) };

fn offsets_match<O: polars_arrow::offset::Offset>(offsets: &polars_arrow::offset::Offsets<O>, values: &[u8]) -> bool {
    offsets.last_offset() == values.len()
}
// assert!(offsets_match(&offsets, &values)); before new_unchecked

When it happens

Trigger: Constructing new_unchecked with offsets and values buffers from different chunk lengths: last offset < values.len() (trailing garbage bytes) or > values.len() (truncated values buffer); slicing the values buffer without rebasing the final offset.

Common situations: FFI/imported buffers from arrow-rs or IPC data; hand-rolled concatenation that copies offsets from one array and values from another; off-by-one when dropping a prefix of the values bytes.

Related errors

• MutableUtf8ValuesArray can only be initialized with DataType
• The external API has a non-utf8 as format
• activate 'dtype-array'
• StructArray must be initialized with DataType::Struct
• Union struct must be created with the corresponding Union Da

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/1cb2e7e1e921d7bc. Report an issue: GitHub.