{"record":{"id":"aa3dc0bf24ed0db2","repo":"apache/iceberg","slug":"missing-required-field-s","errorCode":null,"errorMessage":"Missing required field: %s","messagePattern":"Missing required field: (.+?)","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorizedReaderBuilder.java","lineNumber":137,"sourceCode":"    for (Types.NestedField field : icebergFields) {\n      VectorizedReader<?> reader =\n          VectorizedArrowReader.replaceWithMetadataReader(\n              field, readersById.get(field.fieldId()), idToConstant, setArrowValidityVector);\n      reorderedFields.add(defaultReader(field, reader));\n    }\n    return vectorizedReader(reorderedFields);\n  }\n\n  private VectorizedReader<?> defaultReader(Types.NestedField field, VectorizedReader<?> reader) {\n    if (reader != null) {\n      return reader;\n    } else if (field.initialDefault() != null) {\n      return constantReader(field, convert.apply(field.type(), field.initialDefault()));\n    } else if (field.isOptional()) {\n      return VectorizedArrowReader.nulls();\n    }\n\n    throw new IllegalArgumentException(String.format(\"Missing required field: %s\", field.name()));\n  }\n\n  private <T> ConstantVectorReader<T> constantReader(Types.NestedField field, T constant) {\n    return new ConstantVectorReader<>(field, constant);\n  }\n\n  protected VectorizedReader<?> vectorizedReader(List<VectorizedReader<?>> reorderedFields) {\n    return readerFactory.apply(reorderedFields);\n  }\n\n  @Override\n  public VectorizedReader<?> struct(\n      Types.StructType expected, GroupType groupType, List<VectorizedReader<?>> fieldReaders) {\n    if (expected != null) {\n      throw new UnsupportedOperationException(\n          \"Vectorized reads are not supported yet for struct fields\");\n    }\n    return null;","sourceCodeStart":119,"sourceCodeEnd":155,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorizedReaderBuilder.java#L119-L155","documentation":"VectorizedReaderBuilder.defaultReader picks a reader for a required field without matching Parquet data. If the field is required (not optional), has no constant, and no initial default, there is no way to produce values and the builder throws this IllegalArgumentException. It enforces Iceberg's requirement that required fields be readable.","triggerScenarios":"Building vectorized readers for a schema where a required field has no column data in the file, no addColumn constant, and no initialDefault — the final throw in defaultReader.","commonSituations":"Schema evolution adding a required field to old files lacking that column; hand-edited Avro/Parquet schemas marking fields required without defaults; identity-partition/constant column misconfiguration.","solutions":["Make the field optional or add an initial/default value via schema update so defaultReader can fall back","Rewrite old data files to include the required column","Use Spark's schema evolution with a write.default-column-project or constants, or set the field default in the Iceberg schema","Read without vectorization if a custom path fills the field"],"exampleFix":"// before\nTypes.NestedField.required(1, \"new_col\", Types.StringType.get())\n// after\nTypes.NestedField.optional(1, \"new_col\", Types.StringType.get())\n// or\nschema.addColumn(\"new_col\", StringType.get()).withDefault(\"n/a\")","handlingStrategy":"validation","validationCode":"Types.NestedField f = schema.findField(\"new_col\");\nboolean safe = f.isOptional() || f.initialDefault() != null\n    || schema.constants().containsKey(f.fieldId());\nif (!safe) { /* make optional, set default, or rewrite files */ }","typeGuard":null,"tryCatchPattern":"try {\n  reader = IcebergParquetReaders.buildReader(task, schema, nameMapping, expected, filter, caseSensitive);\n} catch (IllegalArgumentException e) {\n  if (e.getMessage().startsWith(\"Missing required field\")) { /* fix schema: optional/default */ }\n  throw e;\n}","preventionTips":["Add new required fields as optional with defaults via schema evolution","Use Iceberg's addColumn with initialDefault when backfilling","Never mark fields required without defaults in evolving schemas","Rewrite files so required columns are physically present"],"tags":["schema-evolution","required-field","vectorized-reads"],"backgroundTag":"missing-required-argument","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}