apache/iceberg · error · IllegalArgumentException

Expected two ORC type descriptions for a map, got

Error message

Expected two ORC type descriptions for a map, got: ${orcTypes}

What it means

ORC maps are encoded with exactly two child type descriptions: one for keys and one for values. The MapWriter constructor validates this when building Spark-to-ORC writers and throws IllegalArgumentException if the supplied type list does not contain exactly two entries.

Solutions

Ensure the ORC map type has exactly two children: map<keyType,valueType>.
Derive the TypeDescription from the Iceberg schema (TypeDescription.fromString or SparkSchemaUtil) rather than building it manually.
Re-check the table schema after any schema evolution operation that touched map columns.

Example fix

// before
MapWriter(kw, vw, Collections.singletonList(mapType))
// after
MapWriter(kw, vw, Arrays.asList(keyType, valueType))

Defensive patterns

Strategy: validation

Validate before calling

if (orcTypes.size() != 2) {
  throw new IllegalArgumentException("map schema must have exactly key+value children: " + orcTypes);
}

Type guard

boolean isValidMapSchema(TypeDescription t) {
  return t.getCategory() == TypeDescription.Category.MAP && t.getChildren().size() == 2;
}

Try / catch

try {
  writer = SparkOrcValueWriters.map(kw, vw, orcTypes);
} catch (IllegalArgumentException e) {
  throw new IllegalStateException("Bad ORC map schema: " + e.getMessage(), e);
}

Prevention

Use TypeDescription.fromString or Iceberg's ORC schema conversion for map columns.
Re-derive file schemas after any schema evolution touching maps.

When it happens

Trigger: Writing a Spark map column to an Iceberg ORC table when the ORC TypeDescription list for the map column has fewer or more than 2 elements (e.g. a corrupt or hand-built schema).

Common situations: Hand-constructed ORC schemas missing key/value child types; schema evolution leaving map columns malformed; third-party tools writing ORC schemas that Iceberg cannot map.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/33c4e954b60282c7. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/data/SparkOrcValueWriters.java:178

    @Override
    public Stream<FieldMetrics<?>> metrics() {
      return writer.metrics();
    }
  }

  private static class MapWriter<K, V> implements OrcValueWriter<MapData> {
    private final OrcValueWriter<K> keyWriter;
    private final OrcValueWriter<V> valueWriter;
    private final SparkOrcWriter.FieldGetter<K> keyFieldGetter;
    private final SparkOrcWriter.FieldGetter<V> valueFieldGetter;

    @SuppressWarnings("unchecked")
    MapWriter(
        OrcValueWriter<K> keyWriter,
        OrcValueWriter<V> valueWriter,
        List<TypeDescription> orcTypes) {
      if (orcTypes.size() != 2) {
        throw new IllegalArgumentException(
            "Expected two ORC type descriptions for a map, got: " + orcTypes);
      }
      this.keyWriter = keyWriter;
      this.valueWriter = valueWriter;
      this.keyFieldGetter =
          (SparkOrcWriter.FieldGetter<K>) SparkOrcWriter.createFieldGetter(orcTypes.get(0));
      this.valueFieldGetter =
          (SparkOrcWriter.FieldGetter<V>) SparkOrcWriter.createFieldGetter(orcTypes.get(1));
    }

    @Override
    public void nonNullWrite(int rowId, MapData map, ColumnVector output) {
      ArrayData key = map.keyArray();
      ArrayData value = map.valueArray();
      MapColumnVector cv = (MapColumnVector) output;
      // record the length and start of the list elements
      cv.lengths[rowId] = value.numElements();
      cv.offsets[rowId] = cv.childCount;

View on GitHub (pinned to 86d9c8fc54)