Map schema should have both key and value ids set or both…

Error message

Map schema {} should have both key and value ids set or both unset

What it means

Avro PruneColumns warns when projecting an Avro array-of-record map representation where only one of the Avro 'key'/'value' fields resolves to a field ID via the name mapping. Iceberg requires both IDs present or both absent; a partial mapping means column pruning cannot safely project the map, so the field is pruned (returns null).

Solutions

Regenerate the name mapping so both 'key' and 'value' entries have field-ids (e.g. via MetricsUtil or createNameMapping).
Fix the existing mapping JSON to include matching key_id and value_id entries for the map.
Remove the stale mapping so reads fall back to full projection, or rewrite files with current metadata.

Example fix

// before: mapping only has key
[{"field-id": 2, "names": ["m"]}, {"field-id": 3, "names": ["key"]}]
// after
[{"field-id": 2, "names": ["m"]}, {"field-id": 3, "names": ["key"]}, {"field-id": 4, "names": ["value"]}]

Defensive patterns

Strategy: validation

Validate before calling

// validate name mapping covers map key and value
NameMapping mapping = ...;
boolean complete = mapping != null; // regenerate via:
// NameMappingParser.fromJson(Mapping.createNameMapping(table.schema()))

Prevention

Always generate name mappings from the table schema instead of writing them by hand.
Regenerate mappings after any schema evolution.
Include explicit key/value entries with field-ids for every map field.

When it happens

Trigger: Reading an Avro manifest/data file with a name mapping (schema.name-mapping.default) that maps the map's 'key' field but not 'value' (or vice versa) during schema pruning.

Common situations: Hand-written or partially generated name mappings; mappings created before a schema evolution added IDs; mappings missing nested entries for map key/value.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/91e1463d30565940. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/avro/PruneColumns.java:155

      }
      return union;
    }

    return null;
  }

  @Override
  @SuppressWarnings("checkstyle:CyclomaticComplexity")
  public Schema array(Schema array, Schema element) {
    if (array.getLogicalType() instanceof LogicalMap) {
      Schema keyValue = array.getElementType();
      Integer keyId =
          AvroSchemaUtil.getFieldId(keyValue.getField("key"), nameMapping, fieldNames());
      Integer valueId =
          AvroSchemaUtil.getFieldId(keyValue.getField("value"), nameMapping, fieldNames());
      if (keyId == null || valueId == null) {
        if (keyId != null || valueId != null) {
          LOG.warn("Map schema {} should have both key and value ids set or both unset", array);
        }
        return null;
      }

      // if either key or value is selected, the whole map must be projected
      if (selectedIds.contains(keyId) || selectedIds.contains(valueId)) {
        return complexMapWithIds(array, keyId, valueId);
      } else if (element != null) {
        Schema.Field keyProjectionField = element.getField("key");
        Schema valueProjection = element.getField("value").schema();
        // it is possible that key is not selected, and
        // key schemas can be different if new field ids were assigned to them
        if (keyProjectionField != null
            && !Objects.equals(keyValue.getField("key").schema(), keyProjectionField.schema())) {
          Preconditions.checkState(
              SchemaNormalization.parsingFingerprint64(keyValue.getField("key").schema())
                  == SchemaNormalization.parsingFingerprint64(keyProjectionField.schema()),
              "Map keys should not be projected");

View on GitHub (pinned to 86d9c8fc54)