apache/iceberg · error · IllegalArgumentException

Missing required field

Error message

Missing required field: %s

What it means

Thrown by ParquetValueReaders.defaultReader() when a required (non-optional, non-repeating) struct field in the Parquet schema has no corresponding column reader: it is neither a constant field, an optional field, nor otherwise resolvable. The reader cannot produce values for a required field it has no source for, so it fails fast with the field name.

Solutions

  1. Ensure the required field is present in the Parquet file schema or included in the projection
  2. Make the field optional in the table schema and backfill values (Iceberg requires adding columns as optional)
  3. Use typed constant/default values so the reader maps the missing field to initialDefault

Example fix

// before: adding a required column to an existing table (old files lack it)
schema = new Schema(Types.NestedField.required(1, "new_col", Types.IntegerType.get()));
// after: add as optional, then backfill
schema = new Schema(Types.NestedField.optional(1, "new_col", Types.IntegerType.get()));
Defensive patterns

Strategy: validation

Validate before calling

for (Types.NestedField field : struct.fields()) {
  if (field.isRequired() && !fileSchemaContainsField(field.name())) {
    throw new IllegalArgumentException("File schema lacks required field: " + field.name());
  }
}

Try / catch

try {
  reader = ParquetValueReaders.structs(...);
} catch (IllegalArgumentException e) {
  if (e.getMessage().startsWith("Missing required field")) {
    // fall back to projection including the missing column, or fail with a clear schema-evolution message
  } else throw e;
}

Prevention

When it happens

Trigger: Building a struct reader via ParquetValueReaders.structs(...) where the expected Iceberg struct type declares a required field but the Parquet file's projected schema omits it or provides it in a form defaultReader cannot map (not constant, not optional, not a primitive/group column).

Common situations: Schema evolution: a field was added to the Iceberg table as required after the file was written; projection/pruning dropping a required column; mismatch between the expected type and the file schema when reading files written by older writers.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/fac9c2abd075f635. Report an issue: GitHub.

Appendix: source

Thrown at parquet/src/main/java/org/apache/iceberg/parquet/ParquetValueReaders.java:286

        ? reader
        : new PresenceReader<>(reader, presence, fileSchema.getMaxRepetitionLevel(structPath));
  }

  private static ParquetValueReader<?> defaultReader(
      Types.NestedField field,
      ParquetValueReader<?> reader,
      int constantDefinitionLevel,
      BiFunction<org.apache.iceberg.types.Type, Object, Object> convertConstant) {
    if (reader != null) {
      return reader;
    } else if (field.initialDefault() != null) {
      Object value = convertConstant.apply(field.type(), field.initialDefault());
      return constant(value, constantDefinitionLevel);
    } else if (field.isOptional()) {
      return nulls();
    }

    throw new IllegalArgumentException(String.format("Missing required field: %s", field.name()));
  }

  /**
   * Returns the column whose definition level shows whether the struct is present on each row, or
   * null if a field reader already reads one or the struct can never be null.
   */
  private static ColumnDescriptor presenceColumn(
      MessageType fileSchema,
      String[] structPath,
      int structDefinitionLevel,
      List<Types.NestedField> expectedFields,
      Map<Integer, ParquetValueReader<?>> readersById) {
    boolean hasRealFieldReader =
        expectedFields.stream().anyMatch(field -> readersById.containsKey(field.fieldId()));
    if (hasRealFieldReader || structDefinitionLevel <= 0) {
      return null;
    }

View on GitHub (pinned to 86d9c8fc54)