{"record":{"id":"d056f3c97da2b321","repo":"apache/beam","slug":"received-null-value-for-required-field-fieldname","errorCode":null,"errorMessage":"Received null value for required field '{fieldName}'.","messagePattern":"Received null value for required field '(.+?)'\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/IcebergUtils.java","lineNumber":545,"sourceCode":"  /** Converts an Iceberg {@link Record} to a Beam {@link Row}. */\n  public static Row icebergRecordToBeamRow(Schema schema, Record record) {\n    Row.Builder rowBuilder = Row.withSchema(schema);\n    for (Schema.Field field : schema.getFields()) {\n      @Nullable Object icebergValue = record.getField(field.getName());\n      addIcebergValue(rowBuilder, field, icebergValue);\n    }\n    return rowBuilder.build();\n  }\n\n  private static void addIcebergValue(\n      Row.Builder rowBuilder, Schema.Field field, @Nullable Object icebergValue) {\n    boolean isNullable = field.getType().getNullable();\n    if (icebergValue == null) {\n      if (isNullable) {\n        rowBuilder.addValue(null);\n        return;\n      }\n      throw new RuntimeException(\n          String.format(\"Received null value for required field '%s'.\", field.getName()));\n    }\n    switch (field.getType().getTypeName()) {\n      case BYTE:\n      case INT16:\n      case INT32:\n      case INT64:\n      case DECIMAL: // Iceberg and Beam both use BigDecimal\n      case FLOAT: // Iceberg and Beam both use float\n      case DOUBLE: // Iceberg and Beam both use double\n      case STRING: // Iceberg and Beam both use String\n      case BOOLEAN: // Iceberg and Beam both use boolean\n        rowBuilder.addValue(icebergValue);\n        break;\n      case ARRAY:\n        checkState(\n            icebergValue instanceof List,\n            \"Expected List type for field '%s' but received %s\",","sourceCodeStart":527,"sourceCodeEnd":563,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/IcebergUtils.java#L527-L563","documentation":"addIcebergValue converts Iceberg record values into a Beam Row builder. When the source Iceberg value is null but the corresponding Beam field type is non-nullable, a Row cannot be built, so a RuntimeException naming the field is thrown.","triggerScenarios":"structToRow or icebergRecordToBeamRow processes a required (non-optional) Beam field whose Iceberg value is null — e.g. the Iceberg table column was made required after old rows were written with nulls, or the Beam schema was declared with nullable=false while the underlying data is nullable.","commonSituations":"Schema evolution in Iceberg where a column changed from optional to required; Beam schema derived from a subset of columns with wrong nullability; reading legacy Iceberg data written before a NOT NULL constraint was added.","solutions":["Align nullability: declare the Beam field as nullable (FieldType.withNullable(true)) to tolerate nulls","Fix the data: filter or backfill Iceberg rows containing nulls in required columns before converting","If the column is truly required, validate the Iceberg table schema (icebergTable.schema().findField(name).isOptional()) before reading","Wrap record conversion and fall back to a default value for the field when null is encountered"],"exampleFix":"// before\nFieldType ft = FieldType.STRING; // non-nullable by default\n// after\nFieldType ft = FieldType.STRING.withNullable(true);","handlingStrategy":"validation","validationCode":"if (v == null && !field.getType().getNullable()) {\n  throw new IllegalArgumentException(\"Field '\" + field.getName() + \"' is required but value is null\");\n}","typeGuard":"boolean safeForRequired(Object v, Schema.Field f) {\n  return f.getType().getNullable() || v != null;\n}","tryCatchPattern":"try {\n  row = icebergRecordToBeamRow(schema, record);\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Received null value\")) {\n    LOG.warn(\"Null in required field, routing record to dead-letter\");\n  } else { throw e; }\n}","preventionTips":["Mark optional columns with withNullable(true) in the Beam schema","Validate Iceberg data for nulls before conversion","Keep Iceberg and Beam schema nullability in sync during evolution"],"tags":["java","iceberg","beam","null","schema"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}