apache/iceberg · error · IllegalArgumentException

Cannot create writer for non-option union

Error message

Cannot create writer for non-option union: %s

What it means

Validation in BaseWriteBuilder.union: an Avro union schema for an optional field must have exactly two branches (null + type) to map to Iceberg's optional model. The %s is the union schema; the check fails when the union has more (or fewer) than two types, i.e. the schema is not a nullable option.

Solutions

1. Rewrite the Avro schema so nullable fields are two-branch unions of null and the value type
2. Flatten multi-branch unions into separate optional fields before writing
3. Validate the schema shape before handing it to the Avro writer builder

When it happens

Trigger: Thrown at core/src/main/java/org/apache/iceberg/avro/BaseWriteBuilder.java:48 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/1d3f1637f0766ffa. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/avro/BaseWriteBuilder.java:48

  protected abstract ValueWriter<?> fixedWriter(int length);

  @Override
  public ValueWriter<?> record(Schema record, List<String> names, List<ValueWriter<?>> fields) {
    return createRecordWriter(fields);
  }

  @Override
  public ValueWriter<?> union(Schema union, List<ValueWriter<?>> options) {
    Preconditions.checkArgument(
        options.size() == 2, "Cannot create writer for non-option union: %s", union);
    if (union.getTypes().get(0).getType() == Schema.Type.NULL) {
      return ValueWriters.option(0, options.get(1));
    } else if (union.getTypes().get(1).getType() == Schema.Type.NULL) {
      return ValueWriters.option(1, options.get(0));
    } else {
      throw new IllegalArgumentException(
          String.format("Cannot create writer for non-option union: %s", union));
    }
  }

  @Override
  public ValueWriter<?> array(Schema array, ValueWriter<?> elementWriter) {
    if (array.getLogicalType() instanceof LogicalMap) {
      ValueWriters.StructWriter<?> keyValueWriter = (ValueWriters.StructWriter<?>) elementWriter;
      return ValueWriters.arrayMap(keyValueWriter.writer(0), keyValueWriter.writer(1));
    }

    return ValueWriters.array(elementWriter);
  }

  @Override
  public ValueWriter<?> map(Schema map, ValueWriter<?> valueWriter) {
    return ValueWriters.map(ValueWriters.strings(), valueWriter);
  }

View on GitHub (pinned to 86d9c8fc54)