pola-rs/polars · error

Deserialization from JSON not implemented for

Error message

Deserialization from JSON not implemented for {adt:?}

What it means

The final match arm of _deserialize(): every ArrowDataType without a JSON decoding path lands here with unimplemented!, naming the dtype in the message. Handled types cover primitives, temporals, Decimal (feature-gated), Utf8 variants, LargeList, LargeBinary and Struct - so Map, Union, FixedSizeList, dictionary-encoded and other exotic types abort with this message.

Solutions

1. Rewrite unsupported fields in the provided schema to supported dtypes (Map -> List(Struct{k,v}), Dictionary -> its value type, FixedSizeList -> List)
2. Or read with a reduced schema containing only supported columns and cast afterwards
3. Watch upstream for newly supported dtypes in polars-json

Example fix

// before
let field = ArrowDataType::Map(key_value_field, false); // -> unimplemented!("...{adt:?}")

// after
let field = ArrowDataType::LargeList(Box::new(key_value_field.into())); // map as list of {key, value}

fn json_supported(dt: &ArrowDataType) -> bool {
    use ArrowDataType::*;
    !matches!(
        dt,
        Map(_, _) | Union(_, _) | FixedSizeList(_, _) | Dictionary(_, _)
    ) // and recurse into List/Struct children
}
fn schema_json_ready(schema: &ArrowSchema) -> Result<(), String> {
    for f in schema.fields() {
        if !json_supported(f.data_type()) {
            return Err(format!("field {} has dtype {:?} unsupported by polars-json", f.name(), f.data_type()));
        }
    }
    Ok(())
}

fn is_json_supported_dtype(dt: &ArrowDataType) -> bool {
    use ArrowDataType::*;
    !matches!(dt, Map(_, _) | Union(_, _) | FixedSizeList(_, _) | Dictionary(_, _) | NullArrow)
}

The message embeds the offending {adt:?}, so catch_unwind + panic-message parsing can identify the field - but schema pre-validation gives the same answer deterministically.

When it happens

Trigger: read_json/ndjson (or direct deserialize calls) with a schema containing ArrowDataType::Map, Union, FixedSizeList, Dictionary or similar - often schemas converted from arrow-rs RecordBatches, Flight, IPC or Avro-derived models.

Common situations: Reusing an Arrow Flight/IPC schema for JSON ingestion; schemas auto-generated from other systems that include Map or Dictionary types.

Related errors

• unexpected dtype when deserializing ndjson
• List offset is too large :/
• read an Array from a non-Array data type
• The external API has a non-utf8 as format
• activate dtype-categorical to convert dictionary arrays

AI-assisted analysis of pola-rs/polars@9b5d73fd00 (2026-08-19). Data as JSON: /api/errors/6e61c1225f79a10e. Report an issue: GitHub.

Appendix: source

Thrown at crates/polars-json/src/json/deserialize.rs:535

        ArrowDataType::Decimal(_, _) => Ok(Box::new(deserialize_decimal(rows, dtype)?)),
        ArrowDataType::LargeUtf8 => {
            fill_generic_array_from::<_, _, Utf8Array<i64>>(deserialize_utf8_into, rows)
        },
        ArrowDataType::Utf8View => {
            fill_generic_array_from::<_, _, Utf8ViewArray>(deserialize_utf8view_into, rows)
        },
        ArrowDataType::LargeList(_) => Ok(Box::new(deserialize_list(
            rows,
            dtype,
            allow_extra_fields_in_struct,
        )?)),
        ArrowDataType::LargeBinary => Ok(Box::new(deserialize_binary(rows)?)),
        ArrowDataType::Struct(_) => Ok(Box::new(deserialize_struct(
            rows,
            dtype,
            allow_extra_fields_in_struct,
        )?)),
        adt => unimplemented!("Deserialization from JSON not implemented for {adt:?}"),
    }
}

pub fn deserialize(
    json: &BorrowedValue,
    dtype: ArrowDataType,
    allow_extra_fields_in_struct: bool,
) -> PolarsResult<Box<dyn Array>> {
    match json {
        BorrowedValue::Array(rows) => match dtype {
            ArrowDataType::LargeList(inner) => {
                _deserialize(rows, inner.dtype, allow_extra_fields_in_struct)
            },
            _ => todo!("read an Array from a non-Array data type"),
        },
        _ => _deserialize(&[json], dtype, allow_extra_fields_in_struct),
    }
}

View on GitHub (pinned to 9b5d73fd00)