apache/iceberg · error · IllegalArgumentException

Missing required field

Error message

Missing required field: %s

What it means

During read-plan construction, addMissingFileReadersToPlan adds readers for fields present in the expected schema but absent from the file: optional fields get constant-null readers, but a REQUIRED field missing from the file cannot be synthesized, so this IllegalArgumentException is thrown naming the field.

Solutions

  1. Add the field as optional in the table schema (schema evolution) so null can be filled in for old files
  2. Rewrite affected data files to include the required column (e.g. via a rewrite action)
  3. Exclude the missing field from the read projection if it is not needed

Example fix

// before
schemaUpdate.addColumn("added_col", Types.StringType.get()); // default required? ensure optional
// after (Iceberg columns are optional by default; do not force required)
schemaUpdate.addColumn("added_col", Types.StringType.get()); // stays optional
Defensive patterns

Strategy: validation

Validate before calling

for (Types.NestedField f : table.schema().columns()) {
  if (f.isRequired() && !fileSchemaColumns.contains(f.name()) && readSchema.columns().contains(f)) {
    throw new IllegalStateException("required field missing from file: " + f.name());
  }
}

Try / catch

try { ... read ... } catch (IllegalArgumentException e) { if (e.getMessage().startsWith("Missing required field")) { /* alter schema to optional or rewrite files */ } else throw e; }

Prevention

When it happens

Trigger: Reading a data file that does not contain a column the Iceberg schema marks as required; the read projection includes that required field.

Common situations: Schema evolution where a field was added as required after files were written; manually editing a table schema to make an optional field required; external writers omitting columns.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/f6fd528c500590ab. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/avro/ValueReaders.java:325

    Object constant = idToConstant.get(fieldId);
    Types.NestedField field = expected.field(fieldId);

    if (constant != null) {
      if (fieldId == MetadataColumns.ROW_ID.fieldId()) {
        return ValueReaders.rowIds((Long) constant, null);
      }
      return ValueReaders.constant(constant);
    } else if (field.initialDefault() != null) {
      return ValueReaders.constant(convert.apply(field.type(), field.initialDefault()));
    } else if (fieldId == MetadataColumns.IS_DELETED.fieldId()) {
      return ValueReaders.constant(false);
    } else if (fieldId == MetadataColumns.ROW_POSITION.fieldId()) {
      return ValueReaders.positions();
    } else if (field.isOptional()) {
      return ValueReaders.constant(null);
    }

    throw new IllegalArgumentException(String.format("Missing required field: %s", field.name()));
  }

  public static ValueReader<Long> rowIds(Long baseRowId, ValueReader<?> idReader) {
    if (baseRowId != null) {
      return new RowIdReader(baseRowId, (ValueReader<Long>) idReader);
    } else {
      return ValueReaders.constant(null);
    }
  }

  public static ValueReader<Long> lastUpdated(
      Long baseRowId, Long fileSeqNumber, ValueReader<?> seqReader) {
    if (fileSeqNumber != null && baseRowId != null) {
      return new LastUpdatedSeqReader(fileSeqNumber, (ValueReader<Long>) seqReader);
    } else {
      return ValueReaders.constant(null);
    }
  }

View on GitHub (pinned to 86d9c8fc54)