{"record":{"id":"a2df699d9ace2d4f","repo":"apache/beam","slug":"can-t-convert-null-to-non-nullable-field","errorCode":null,"errorMessage":"Can't convert 'null' to non-nullable field","messagePattern":"Can't convert 'null' to non-nullable field","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java","lineNumber":1591,"sourceCode":"\n      case ENUM:\n        // enums are either Java enums, or GenericEnumSymbol,\n        // they don't share common interface, but override toString()\n        return convertEnumStrict(value, fieldType);\n\n      case ARRAY:\n        return convertArrayStrict(\n            (List<Object>) value, type.type.getElementType(), fieldType, genericData);\n\n      case MAP:\n        return convertMapStrict(\n            (Map<CharSequence, Object>) value, type.type.getValueType(), fieldType, genericData);\n\n      case UNION:\n        return convertUnionStrict(value, type.type, fieldType, genericData);\n\n      case NULL:\n        throw new IllegalArgumentException(\"Can't convert 'null' to non-nullable field\");\n\n      default:\n        throw new AssertionError(\"Unexpected AVRO Schema.Type: \" + type.type.getType());\n    }\n  }\n\n  /**\n   * Strict conversion from AVRO to Beam, strict because it doesn't do widening or narrowing during\n   * conversion.\n   *\n   * @param value {@link GenericRecord} or any nested value\n   * @param avroSchema schema for value\n   * @param fieldType target beam field type\n   * @return value converted for {@link Row}\n   */\n  @SuppressWarnings(\"unchecked\")\n  public static @PolyNull Object convertAvroFieldStrict(\n      @PolyNull Object value,","sourceCodeStart":1573,"sourceCodeEnd":1609,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java#L1573-L1609","documentation":"AvroUtils.convertAvroFieldStrict performs strict Avro-to-Beam Schema value conversion when materializing GenericRecords into Beam Rows. When the Avro schema branch being converted is Schema.Type.NULL (i.e., the value is literally null and the target Beam FieldType is not nullable), the converter cannot produce a non-null value for a non-nullable Beam field and throws this IllegalArgumentException. It exists because strict conversion refuses to silently coerce null into a field the Beam schema declares as non-nullable.","triggerScenarios":"Calling convertAvroFieldStrict(value, avroSchema, fieldType) (or toBeamRowStrict via AvroRecorder/AvroIO reads) where the resolved union branch or schema type is NULL but the target Beam FieldType is non-nullable — i.e., an Avro schema declares \"null\" only in a union (e.g. [\"null\",\"string\"]) yet the Beam schema field was built non-nullable, or a GenericRecord holds a null for a field whose Beam FieldType lacks the NULLABLE flag.","commonSituations":"Reading Avro files whose schema has nullable union fields (the default Avro idiom) into a Beam schema defined by hand without nullable fields; a schema drift where the Avro file contains nulls that the Beam schema does not allow; building Row objects in tests with null values for non-nullable fields.","solutions":["Make the corresponding Beam Schema field nullable (FieldType nullable: fieldType.withNullable(true) or \"field\": FieldType.STRING.withNullable(true)) so null values can be represented.","Sanitize/replace null values before conversion (e.g., via a MapElements that substitutes defaults) so non-nullable fields never receive null.","Verify the Beam schema was generated to match the Avro schema (use AvroUtils.toBeamSchema(avroSchema) instead of a hand-written schema) so nullability matches."],"exampleFix":"// before\nSchema beamSchema = Schema.of(Field.of(\"name\", FieldType.STRING));\nRow row = toBeamRowStrict(avroRecord, beamSchema); // throws when name is null\n\n// after\nSchema beamSchema = Schema.of(Field.of(\"name\", FieldType.STRING.withNullable(true)));","handlingStrategy":"validation","validationCode":"Schema beamSchema = AvroUtils.toBeamSchema(avroSchema);\nfor (Field f : beamSchema.getFields()) {\n  Object v = record.get(f.getName());\n  if (v == null && !f.getType().getNullable()) {\n    throw new IllegalStateException(\"field \" + f.getName() + \" is null but non-nullable\");\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  Row row = AvroUtils.toBeamRowStrict(record, beamSchema);\n} catch (IllegalArgumentException e) {\n  if (e.getMessage().contains(\"non-nullable\")) {\n    // handle null-in-non-nullable field (default or skip)\n  } else { throw e; }\n}","preventionTips":["Derive the Beam schema from the Avro schema with AvroUtils.toBeamSchema instead of writing it by hand.","Mark any Avro union containing \"null\" as withNullable(true) in the Beam schema.","Validate sample records against the schema before launching the pipeline."],"tags":["avro","beam","nullability","schema-conversion","illegal-argument"],"backgroundTag":"null-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}