{"record":{"id":"7f12b8e5b3015625","repo":"apache/beam","slug":"field-is-not-nullable","errorCode":null,"errorMessage":"Field is not nullable.","messagePattern":"Field is not nullable\\.","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java","lineNumber":748,"sourceCode":"\n  /** Convert a Beam Row to a BigQuery TableRow. */\n  public static TableRow toTableRow(Row row) {\n    TableRow output = new TableRow();\n    for (int i = 0; i < row.getFieldCount(); i++) {\n      @Nullable Object value = row.getValue(i);\n      Field schemaField = row.getSchema().getField(i);\n      @SuppressWarnings(\"nullness\") // TableRow.set is not annotated, but accepts nulls\n      TableRow updated =\n          output.set(schemaField.getName(), fromBeamField(schemaField.getType(), value));\n      output = updated;\n    }\n    return output;\n  }\n\n  private static @Nullable Object fromBeamField(FieldType fieldType, @Nullable Object fieldValue) {\n    if (fieldValue == null) {\n      if (!fieldType.getNullable()) {\n        throw new IllegalArgumentException(\"Field is not nullable.\");\n      }\n      return null;\n    }\n\n    switch (fieldType.getTypeName()) {\n      case ARRAY:\n      case ITERABLE:\n        FieldType elementType =\n            Preconditions.checkArgumentNotNull(fieldType.getCollectionElementType());\n        Iterable<?> items = (Iterable<?>) fieldValue;\n        List<@Nullable Object> convertedItems =\n            Lists.newArrayListWithCapacity(Iterables.size(items));\n        for (Object item : items) {\n          convertedItems.add(fromBeamField(elementType, item));\n        }\n        return convertedItems;\n\n      case MAP:","sourceCodeStart":730,"sourceCodeEnd":766,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java#L730-L766","documentation":"Thrown by BigQueryUtils.fromBeamField when converting a Beam row field back toward a BigQuery/Avro representation and the value is null while the beam Schema FieldType is declared non-nullable. The Beam schema contract forbids nulls in required fields, so the library fails fast instead of silently writing a null into a REQUIRED column. This is a schema/data mismatch, not an infrastructure failure.","triggerScenarios":"Calling BeamRow -> Avro/BigQuery conversion (e.g. BigQueryUtils.toAvro or row-to-JSON paths) via fromBeamField with a null fieldValue on a FieldType whose getNullable() is false.","commonSituations":"Building TableRow/Avro records from Beam rows where a REQUIRED BigQuery column has no value; rows produced by a DoFn that leaves required fields unset; a schema evolved from NULLABLE to REQUIRED while old data still contains nulls.","solutions":["Declare the field NULLABLE in the Beam Schema (Schema.Field.of(name, FieldType.X.withNullable(true))) to match the data.","Populate the field before conversion (e.g. withRow fallthrough/default value in the DoFn that builds the row).","Filter out rows with nulls in required fields before the conversion step.","If the BigQuery table column should accept nulls, change the BigQuery schema to NULLABLE and mirror it in the Beam schema."],"exampleFix":"// before\nSchema.Field f = Schema.Field.of(\"id\", FieldType.INT64); // required\nrow.getInt(\"id\") == null -> throws\n\n// after\nSchema.Field f = Schema.Field.of(\"id\", FieldType.INT64.withNullable(true));\n// or ensure a value: row = new RowBuilder().addInt(\"id\", id == null ? -1 : id).build();","handlingStrategy":"validation","validationCode":"// Java: validate row against schema before conversion\nfor (Schema.Field f : row.getSchema().getFields()) {\n  if (!f.getType().getNullable() && row.getValue(f.getName()) == null) {\n    throw new IllegalArgumentException(\"Non-nullable field has null value: \" + f.getName());\n  }\n}","typeGuard":"// Java\nstatic boolean isUsableRequiredValue(Row row, String name) {\n  Schema.Field f = row.getSchema().getField(name);\n  return f.getType().getNullable() || row.getValue(name) != null;\n}","tryCatchPattern":"try {\n  avroRecord = BigQueryUtils.toAvro(schema, row);\n} catch (IllegalArgumentException e) {\n  // log and route row to dead-letter PCollection\n}","preventionTips":["Always mirror BigQuery column modes (REQUIRED/NULLABLE/REPEATED) exactly in the Beam Schema.","Set defaults for required fields when constructing rows in DoFns.","Add a unit test converting representative rows through BigQueryUtils before running the pipeline."],"tags":["java","apache-beam","bigquery","nullability","schema"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}