{"record":{"id":"95f1771807c523da","repo":"databendlabs/databend","slug":"invalid-map-type","errorCode":null,"errorMessage":"invalid map type","messagePattern":"invalid map type","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/query/storages/common/blocks/src/parquet_writer/bulk.rs","lineNumber":159,"sourceCode":"        | ArrowDataType::LargeUtf8\n        | ArrowDataType::BinaryView\n        | ArrowDataType::Utf8View => out.push(LeafEncoderKind::ByteArray),\n        ArrowDataType::List(f)\n        | ArrowDataType::LargeList(f)\n        | ArrowDataType::FixedSizeList(f, _)\n        | ArrowDataType::ListView(f)\n        | ArrowDataType::LargeListView(f) => classify_data_type(f.data_type(), out),\n        ArrowDataType::Struct(fields) => {\n            for field in fields {\n                classify_data_type(field.data_type(), out);\n            }\n        }\n        ArrowDataType::Map(f, _) => match f.data_type() {\n            ArrowDataType::Struct(fields) => {\n                classify_data_type(fields[0].data_type(), out);\n                classify_data_type(fields[1].data_type(), out);\n            }\n            _ => unreachable!(\"invalid map type\"),\n        },\n        ArrowDataType::Dictionary(_, value_type) => match value_type.as_ref() {\n            ArrowDataType::Utf8\n            | ArrowDataType::LargeUtf8\n            | ArrowDataType::Binary\n            | ArrowDataType::LargeBinary\n            | ArrowDataType::Utf8View\n            | ArrowDataType::BinaryView\n            | ArrowDataType::FixedSizeBinary(_) => out.push(LeafEncoderKind::ByteArray),\n            _ => out.push(LeafEncoderKind::Column),\n        },\n        // Primitives, FixedSizeBinary, Boolean, Null, etc.\n        _ => out.push(LeafEncoderKind::Column),\n    }\n}\n\n/// Low-level **leaf-oriented** single-row-group Parquet writer.\n///","sourceCodeStart":141,"sourceCodeEnd":177,"githubUrl":"https://github.com/databendlabs/databend/blob/288d84d76e20a2f8f7173bda9691eb6ece301aa9/src/query/storages/common/blocks/src/parquet_writer/bulk.rs#L141-L177","documentation":"When classifying a parquet Arrow schema for bulk writing, a Map field must always have a Struct child (the key/value entry struct). If the Arrow Map's child field is not a Struct, the code panics with the explicit message 'invalid map type', meaning the schema fed to the writer is malformed.","triggerScenarios":"Writing (or converting from an external source) a schema whose ArrowDataType::Map child entry is not a Struct — e.g. hand-built Arrow schemas, third-party parquet files with nonstandard map encodings, or wrong field ordering passed to the bulk writer.","commonSituations":"Ingesting parquet produced by other systems with nonstandard Map representations; constructing Arrow Map arrays with a non-struct entries field in a custom data-loading pipeline.","solutions":["Validate the Arrow schema before writing: assert every Map field's child is a Struct with key/value fields","Re-encode the source data using standard Arrow Map layout (struct entries with 'key' and 'value' fields)","If reading third-party parquet, cast/rebuild the map column to a standard Map type before the bulk writer"],"exampleFix":"// before\nArrowDataType::Map(f, _) => match f.data_type() {\n    ArrowDataType::Struct(fields) => { ... }\n    _ => unreachable!(\"invalid map type\"),\n},\n// after\nArrowDataType::Map(f, _) => match f.data_type() {\n    ArrowDataType::Struct(fields) => { ... }\n    other => return Err(ErrorCode::BadDataParquetValueType(\n        format!(\"invalid map type: {:?}\", other))),\n},","handlingStrategy":"validation","validationCode":"// validate Arrow Map schema before writing\nfunction isStandardArrowMap(field) {\n  const t = field.type;\n  if (!t || t typeId !== 12 /* Map */) return true; // not a map\n  const child = field.children[0];\n  return child.type.typeId === 8 /* Struct */ &&\n    child.children.length === 2;\n}","typeGuard":"function isStructEntriesMap(arrowField) {\n  return arrowField.type instanceof ArrowMap &&\n    arrowField.children[0].type instanceof ArrowStruct;\n}","tryCatchPattern":"try {\n  bulkWrite(blocks);\n} catch (e) {\n  if (/invalid map type/.test(e.message)) {\n    // normalize map schema (rebuild as struct-entries Map) and retry\n  }\n}","preventionTips":["Validate Arrow schemas (map child must be a Struct with key/value fields) before bulk writes","Re-encode parquet from third-party producers into standard Arrow Map layout","Pin consistent Arrow library versions across producer and writer"],"tags":["rust","parquet","arrow","map","schema"],"backgroundTag":"schema-validation-failed","analyzedSha":"288d84d76e20a2f8f7173bda9691eb6ece301aa9","analyzedAt":"2026-09-11T11:29:36.208Z","contentChangedAt":"2026-09-11T11:29:36.208Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}