{"record":{"id":"f6fd528c500590ab","repo":"apache/iceberg","slug":"missing-required-field-s-f6fd52","errorCode":null,"errorMessage":"Missing required field: %s","messagePattern":"Missing required field: (.+?)","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"core/src/main/java/org/apache/iceberg/avro/ValueReaders.java","lineNumber":325,"sourceCode":"    Object constant = idToConstant.get(fieldId);\n    Types.NestedField field = expected.field(fieldId);\n\n    if (constant != null) {\n      if (fieldId == MetadataColumns.ROW_ID.fieldId()) {\n        return ValueReaders.rowIds((Long) constant, null);\n      }\n      return ValueReaders.constant(constant);\n    } else if (field.initialDefault() != null) {\n      return ValueReaders.constant(convert.apply(field.type(), field.initialDefault()));\n    } else if (fieldId == MetadataColumns.IS_DELETED.fieldId()) {\n      return ValueReaders.constant(false);\n    } else if (fieldId == MetadataColumns.ROW_POSITION.fieldId()) {\n      return ValueReaders.positions();\n    } else if (field.isOptional()) {\n      return ValueReaders.constant(null);\n    }\n\n    throw new IllegalArgumentException(String.format(\"Missing required field: %s\", field.name()));\n  }\n\n  public static ValueReader<Long> rowIds(Long baseRowId, ValueReader<?> idReader) {\n    if (baseRowId != null) {\n      return new RowIdReader(baseRowId, (ValueReader<Long>) idReader);\n    } else {\n      return ValueReaders.constant(null);\n    }\n  }\n\n  public static ValueReader<Long> lastUpdated(\n      Long baseRowId, Long fileSeqNumber, ValueReader<?> seqReader) {\n    if (fileSeqNumber != null && baseRowId != null) {\n      return new LastUpdatedSeqReader(fileSeqNumber, (ValueReader<Long>) seqReader);\n    } else {\n      return ValueReaders.constant(null);\n    }\n  }","sourceCodeStart":307,"sourceCodeEnd":343,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/core/src/main/java/org/apache/iceberg/avro/ValueReaders.java#L307-L343","documentation":"During read-plan construction, addMissingFileReadersToPlan adds readers for fields present in the expected schema but absent from the file: optional fields get constant-null readers, but a REQUIRED field missing from the file cannot be synthesized, so this IllegalArgumentException is thrown naming the field.","triggerScenarios":"Reading a data file that does not contain a column the Iceberg schema marks as required; the read projection includes that required field.","commonSituations":"Schema evolution where a field was added as required after files were written; manually editing a table schema to make an optional field required; external writers omitting columns.","solutions":["Add the field as optional in the table schema (schema evolution) so null can be filled in for old files","Rewrite affected data files to include the required column (e.g. via a rewrite action)","Exclude the missing field from the read projection if it is not needed"],"exampleFix":"// before\nschemaUpdate.addColumn(\"added_col\", Types.StringType.get()); // default required? ensure optional\n// after (Iceberg columns are optional by default; do not force required)\nschemaUpdate.addColumn(\"added_col\", Types.StringType.get()); // stays optional","handlingStrategy":"validation","validationCode":"for (Types.NestedField f : table.schema().columns()) {\n  if (f.isRequired() && !fileSchemaColumns.contains(f.name()) && readSchema.columns().contains(f)) {\n    throw new IllegalStateException(\"required field missing from file: \" + f.name());\n  }\n}","typeGuard":null,"tryCatchPattern":"try { ... read ... } catch (IllegalArgumentException e) { if (e.getMessage().startsWith(\"Missing required field\")) { /* alter schema to optional or rewrite files */ } else throw e; }","preventionTips":["Add evolving columns as optional, never required","Run table metadata checks before schema hardening","Rewrite files after making a column required","Keep projections aligned with what old files actually contain"],"tags":["avro","schema-evolution","required-field"],"backgroundTag":"missing-required-argument","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}