databendlabs/databend · error

invalid map type

Error message

invalid map type

What it means

When classifying a parquet Arrow schema for bulk writing, a Map field must always have a Struct child (the key/value entry struct). If the Arrow Map's child field is not a Struct, the code panics with the explicit message 'invalid map type', meaning the schema fed to the writer is malformed.

Solutions

1. Validate the Arrow schema before writing: assert every Map field's child is a Struct with key/value fields
2. Re-encode the source data using standard Arrow Map layout (struct entries with 'key' and 'value' fields)
3. If reading third-party parquet, cast/rebuild the map column to a standard Map type before the bulk writer

Example fix

// before
ArrowDataType::Map(f, _) => match f.data_type() {
    ArrowDataType::Struct(fields) => { ... }
    _ => unreachable!("invalid map type"),
},
// after
ArrowDataType::Map(f, _) => match f.data_type() {
    ArrowDataType::Struct(fields) => { ... }
    other => return Err(ErrorCode::BadDataParquetValueType(
        format!("invalid map type: {:?}", other))),
},

// validate Arrow Map schema before writing
function isStandardArrowMap(field) {
  const t = field.type;
  if (!t || t typeId !== 12 /* Map */) return true; // not a map
  const child = field.children[0];
  return child.type.typeId === 8 /* Struct */ &&
    child.children.length === 2;
}

function isStructEntriesMap(arrowField) {
  return arrowField.type instanceof ArrowMap &&
    arrowField.children[0].type instanceof ArrowStruct;
}

try {
  bulkWrite(blocks);
} catch (e) {
  if (/invalid map type/.test(e.message)) {
    // normalize map schema (rebuild as struct-entries Map) and retry
  }
}

When it happens

Trigger: Writing (or converting from an external source) a schema whose ArrowDataType::Map child entry is not a Struct — e.g. hand-built Arrow schemas, third-party parquet files with nonstandard map encodings, or wrong field ordering passed to the bulk writer.

Common situations: Ingesting parquet produced by other systems with nonstandard Map representations; constructing Arrow Map arrays with a non-struct entries field in a custom data-loading pipeline.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• internal error: entered unreachable code
• DataType::Map should contain a struct field child
• internal error: entered unreachable code
• internal error: entered unreachable code
• failed to infer billing_usage_daily schema

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/95f1771807c523da. Report an issue: GitHub.

Appendix: source

Thrown at src/query/storages/common/blocks/src/parquet_writer/bulk.rs:159

        | ArrowDataType::LargeUtf8
        | ArrowDataType::BinaryView
        | ArrowDataType::Utf8View => out.push(LeafEncoderKind::ByteArray),
        ArrowDataType::List(f)
        | ArrowDataType::LargeList(f)
        | ArrowDataType::FixedSizeList(f, _)
        | ArrowDataType::ListView(f)
        | ArrowDataType::LargeListView(f) => classify_data_type(f.data_type(), out),
        ArrowDataType::Struct(fields) => {
            for field in fields {
                classify_data_type(field.data_type(), out);
            }
        }
        ArrowDataType::Map(f, _) => match f.data_type() {
            ArrowDataType::Struct(fields) => {
                classify_data_type(fields[0].data_type(), out);
                classify_data_type(fields[1].data_type(), out);
            }
            _ => unreachable!("invalid map type"),
        },
        ArrowDataType::Dictionary(_, value_type) => match value_type.as_ref() {
            ArrowDataType::Utf8
            | ArrowDataType::LargeUtf8
            | ArrowDataType::Binary
            | ArrowDataType::LargeBinary
            | ArrowDataType::Utf8View
            | ArrowDataType::BinaryView
            | ArrowDataType::FixedSizeBinary(_) => out.push(LeafEncoderKind::ByteArray),
            _ => out.push(LeafEncoderKind::Column),
        },
        // Primitives, FixedSizeBinary, Boolean, Null, etc.
        _ => out.push(LeafEncoderKind::Column),
    }
}

/// Low-level **leaf-oriented** single-row-group Parquet writer.
///

View on GitHub (pinned to 288d84d76e)