pola-rs/polars · error

not implemented

Error message

not implemented

What it means

ArrowDataType::to_physical_type() translates a logical dtype into the PhysicalType that all kernels dispatch on. The Unknown variant — a placeholder for types that could not be resolved, e.g. imported foreign Arrow data with an unrecognized type code — hits unimplemented!() (crates/polars-arrow/src/datatypes/mod.rs:330). Because virtually every kernel in the crate starts with to_physical_type(), any operation on an Unknown-typed array panics somewhere deep in the stack.

Source

Thrown at crates/polars-arrow/src/datatypes/mod.rs:330

            Interval(IntervalUnit::MonthDayMillis) => {
                PhysicalType::Primitive(PrimitiveType::MonthDayMillis)
            },
            Binary => PhysicalType::Binary,
            FixedSizeBinary(_) => PhysicalType::FixedSizeBinary,
            LargeBinary => PhysicalType::LargeBinary,
            Utf8 => PhysicalType::Utf8,
            LargeUtf8 => PhysicalType::LargeUtf8,
            BinaryView => PhysicalType::BinaryView,
            Utf8View => PhysicalType::Utf8View,
            List(_) => PhysicalType::List,
            FixedSizeList(_, _) => PhysicalType::FixedSizeList,
            LargeList(_) => PhysicalType::LargeList,
            Struct(_) => PhysicalType::Struct,
            Union(_) => PhysicalType::Union,
            Map(_, _) => PhysicalType::Map,
            Dictionary(key, _, _) => PhysicalType::Dictionary(*key),
            Extension(ext) => ext.inner.to_physical_type(),
            Unknown => unimplemented!(),
        }
    }

    // The datatype underlying this (possibly logical) arrow data type.
    pub fn underlying_physical_type(&self) -> ArrowDataType {
        use ArrowDataType::*;
        match self {
            Null | Boolean | Int8 | Int16 | Int32 | Int64 | Int128 | UInt8 | UInt16 | UInt32
            | UInt64 | UInt128 | Float16 | Float32 | Float64 | Binary | LargeBinary | Utf8
            | LargeUtf8 | BinaryView | Utf8View | FixedSizeBinary(_) | Unknown => self.clone(),

            Decimal32(_, _) | Date32 | Time32(_) | Interval(IntervalUnit::YearMonth) => Int32,
            Decimal64(_, _)
            | Date64
            | Timestamp(_, _)
            | Time64(_)
            | Duration(_)
            | Interval(IntervalUnit::DayTime) => Int64,

View on GitHub (pinned to df599052da)

Solutions

  1. Inspect the schema right after reading and drop or re-type Unknown columns before any compute
  2. Fix the producing side to write types this polars-arrow version understands
  3. Upgrade polars/polars-arrow so the foreign type becomes known on import
  4. Fail fast at ingestion with a named-column error instead of letting kernels panic

Example fix

// before
let out = concatenate(&[&a, &b])?; // panics inside to_physical_type() on Unknown

// after
for f in &schema.fields {
    if matches!(f.dtype(), ArrowDataType::Unknown) {
        polars_bail!(ComputeError: "column '{}' has Unknown dtype; re-export or drop it", f.name);
    }
}
let out = concatenate(&[&a, &b])?;
Defensive patterns

Strategy: validation

Validate before calling

for f in &schema.fields {
    polars_ensure!(
        !matches!(f.dtype(), ArrowDataType::Unknown),
        ComputeError: "column '{}' has Unknown dtype (unrecognized type on import); drop or re-type it", f.name
    );
}

Type guard

fn contains_unknown(dtype: &ArrowDataType) -> bool {
    match dtype {
        ArrowDataType::Unknown => true,
        ArrowDataType::List(f) | ArrowDataType::LargeList(f) | ArrowDataType::FixedSizeList(f, _) => contains_unknown(f.dtype()),
        ArrowDataType::Struct(fs) => fs.iter().any(|f| contains_unknown(f.dtype())),
        ArrowDataType::Dictionary(_, v, _) => contains_unknown(v),
        ArrowDataType::Extension(e) => contains_unknown(&e.inner),
        _ => false,
    }
}

Prevention

When it happens

Trigger: Reading IPC/Feather or FFI-imported data whose flatbuffer type is not recognized produces Unknown columns; then any compute (concatenate, take, cast, comparison) on that column calls to_physical_type() and panics.

Common situations: Version skew: a producer uses a newer or nonstandard Arrow type than this polars-arrow knows; corrupted type metadata; relay services that read and forward foreign Arrow without schema validation.

Related errors

AI-assisted analysis of pola-rs/polars@df599052da (2026-08-16). Data as JSON: /api/errors/ee43767fd42ef160. Report an issue: GitHub.