What causes "not implemented" in pola-rs/polars?

Guard by dtype before calling: only invoke get_values_size() for LargeList/LargeBinary/LargeUtf8/FixedSizeList/Utf8View/BinaryView; use arr.len() for fixed-width arrays.

pola-rs/polars · critical

not implemented

Error message

not implemented

What it means

Panic (unimplemented!()) in the ValueSize::get_values_size impl for ArrayRef in polars-arrow. This dispatch downcasts only LargeUtf8, FixedSizeList, LargeList, LargeBinary, Utf8View and BinaryView arrays (computing the offset-aware 'visible' values size); any other dtype inside the ArrayRef - primitives, Boolean, Null, or legacy Utf8/List/Binary - falls to the catch-all arm and panics. The method is meant for variable-length/nested payloads where raw len() does not reflect sliced offsets, so calling it on fixed-width arrays is unsupported.

Source

Thrown at crates/polars-arrow/src/array/values.rs:88

                .downcast_ref::<ListArray<i64>>()
                .unwrap()
                .get_values_size(),
            ArrowDataType::LargeBinary => self
                .as_any()
                .downcast_ref::<BinaryArray<i64>>()
                .unwrap()
                .get_values_size(),
            ArrowDataType::Utf8View => self
                .as_any()
                .downcast_ref::<Utf8ViewArray>()
                .unwrap()
                .total_bytes_len(),
            ArrowDataType::BinaryView => self
                .as_any()
                .downcast_ref::<BinaryViewArray>()
                .unwrap()
                .total_bytes_len(),
            _ => unimplemented!(),
        }
    }
}

View on GitHub (pinned to df599052da)

Solutions

Guard by dtype before calling: only invoke get_values_size() for LargeList/LargeBinary/LargeUtf8/FixedSizeList/Utf8View/BinaryView; use arr.len() for fixed-width arrays.
Normalize incoming Arrow data to Polars-native dtypes (Utf8View, BinaryView, LargeList, LargeBinary) before it enters polars code paths.
If reached from polars internals, inspect the column's actual dtype (schema print) and cast/rebuild it; report a minimal repro upstream if it is an internal invariant violation.
Upgrade polars - dispatch coverage in this match grows over releases.

Example fix

// before (Rust)
let n = arr.get_values_size(); // panics for Int32/Boolean/Utf8<ArrayRef>

// after
let n = match arr.dtype().to_physical_type() {
    PhysicalType::LargeList
    | PhysicalType::FixedSizeList
    | PhysicalType::LargeBinary
    | PhysicalType::LargeUtf8
    | PhysicalType::BinaryView
    | PhysicalType::Utf8View => arr.get_values_size(),
    _ => arr.len(),
};

Defensive patterns

Strategy: validation

Validate before calling

// Rust
use polars_arrow::array::ValueSize;
let n = if matches!(arr.dtype(), ArrowDataType::LargeList(_) | ArrowDataType::FixedSizeList(_, _) | ArrowDataType::LargeBinary | ArrowDataType::LargeUtf8 | ArrowDataType::Utf8View | ArrowDataType::BinaryView) {
    arr.get_values_size()
} else {
    arr.len()
};

Type guard

// Rust
fn supports_values_size(dt: &ArrowDataType) -> bool {
    matches!(dt, ArrowDataType::LargeList(_) | ArrowDataType::FixedSizeList(_, _) | ArrowDataType::LargeBinary | ArrowDataType::LargeUtf8 | ArrowDataType::Utf8View | ArrowDataType::BinaryView)
}

Try / catch

// Rust - contain the panic at a boundary if the dtype is untrusted
let size = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| arr.get_values_size()))
    .unwrap_or_else(|_| arr.len());

Prevention

get_values_size() is only for variable-length/nested arrays; use arr.len() for primitives and Boolean.
Ensure legacy Utf8/List (32-bit) arrays are converted to view/large variants before entering polars code.
When writing generic ArrayRef code, match on dtype before calling offset-aware size methods.

When it happens

Trigger: Calling `arr_ref.get_values_size()` on an ArrayRef wrapping an Int32/Float64/Boolean array, a Utf8Array<i32>, or a ListArray<i32>. Typically reached indirectly from polars internals (e.g. sizing values buffers for nested list gathering/extension) where an unexpected dtype flows into the ArrayRef-typed code path - most often a legacy 32-bit Utf8/List produced by external Arrow interop instead of the view/large variants Polars uses.

Common situations: Rust code handling generic ArrayRefs from Arrow IPC/FFI with legacy Utf8 or List types; mixed-dtype dispatch where only variable-length types should reach the call but a primitive slips through; upgrading polars versions that changed which dtypes are normalized to view types.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/e129ec4e5b568def. Report an issue: GitHub.