apache/iceberg · error · IllegalArgumentException

Missing required field

Error message

Missing required field: %s

What it means

VectorizedReaderBuilder.defaultReader picks a reader for a required field without matching Parquet data. If the field is required (not optional), has no constant, and no initial default, there is no way to produce values and the builder throws this IllegalArgumentException. It enforces Iceberg's requirement that required fields be readable.

Solutions

  1. Make the field optional or add an initial/default value via schema update so defaultReader can fall back
  2. Rewrite old data files to include the required column
  3. Use Spark's schema evolution with a write.default-column-project or constants, or set the field default in the Iceberg schema
  4. Read without vectorization if a custom path fills the field

Example fix

// before
Types.NestedField.required(1, "new_col", Types.StringType.get())
// after
Types.NestedField.optional(1, "new_col", Types.StringType.get())
// or
schema.addColumn("new_col", StringType.get()).withDefault("n/a")
Defensive patterns

Strategy: validation

Validate before calling

Types.NestedField f = schema.findField("new_col");
boolean safe = f.isOptional() || f.initialDefault() != null
    || schema.constants().containsKey(f.fieldId());
if (!safe) { /* make optional, set default, or rewrite files */ }

Try / catch

try {
  reader = IcebergParquetReaders.buildReader(task, schema, nameMapping, expected, filter, caseSensitive);
} catch (IllegalArgumentException e) {
  if (e.getMessage().startsWith("Missing required field")) { /* fix schema: optional/default */ }
  throw e;
}

Prevention

When it happens

Trigger: Building vectorized readers for a schema where a required field has no column data in the file, no addColumn constant, and no initialDefault — the final throw in defaultReader.

Common situations: Schema evolution adding a required field to old files lacking that column; hand-edited Avro/Parquet schemas marking fields required without defaults; identity-partition/constant column misconfiguration.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/aa3dc0bf24ed0db2. Report an issue: GitHub.

Appendix: source

Thrown at arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorizedReaderBuilder.java:137

    for (Types.NestedField field : icebergFields) {
      VectorizedReader<?> reader =
          VectorizedArrowReader.replaceWithMetadataReader(
              field, readersById.get(field.fieldId()), idToConstant, setArrowValidityVector);
      reorderedFields.add(defaultReader(field, reader));
    }
    return vectorizedReader(reorderedFields);
  }

  private VectorizedReader<?> defaultReader(Types.NestedField field, VectorizedReader<?> reader) {
    if (reader != null) {
      return reader;
    } else if (field.initialDefault() != null) {
      return constantReader(field, convert.apply(field.type(), field.initialDefault()));
    } else if (field.isOptional()) {
      return VectorizedArrowReader.nulls();
    }

    throw new IllegalArgumentException(String.format("Missing required field: %s", field.name()));
  }

  private <T> ConstantVectorReader<T> constantReader(Types.NestedField field, T constant) {
    return new ConstantVectorReader<>(field, constant);
  }

  protected VectorizedReader<?> vectorizedReader(List<VectorizedReader<?>> reorderedFields) {
    return readerFactory.apply(reorderedFields);
  }

  @Override
  public VectorizedReader<?> struct(
      Types.StructType expected, GroupType groupType, List<VectorizedReader<?>> fieldReaders) {
    if (expected != null) {
      throw new UnsupportedOperationException(
          "Vectorized reads are not supported yet for struct fields");
    }
    return null;

View on GitHub (pinned to 86d9c8fc54)