{"record":{"id":"5a9a91196c676dbf","repo":"apache/beam","slug":"fieldtype-fieldtype-and-avro-schema-avroschema-don-t-have","errorCode":null,"errorMessage":"FieldType ${fieldType} and AVRO schema ${avroSchema} don't have matching nullability","messagePattern":"FieldType (.+?) and AVRO schema (.+?) don't have matching nullability","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java","lineNumber":1283,"sourceCode":"    }\n    return fieldType.getNullable() ? ReflectData.makeNullable(baseType) : baseType;\n  }\n\n  private static final Map<org.apache.avro.Schema, Function<Number, ? extends Number>>\n      NUMERIC_CONVERTERS =\n          ImmutableMap.of(\n              org.apache.avro.Schema.create(org.apache.avro.Schema.Type.INT), Number::intValue,\n              org.apache.avro.Schema.create(org.apache.avro.Schema.Type.LONG), Number::longValue,\n              org.apache.avro.Schema.create(org.apache.avro.Schema.Type.FLOAT), Number::floatValue,\n              org.apache.avro.Schema.create(org.apache.avro.Schema.Type.DOUBLE),\n                  Number::doubleValue);\n\n  /** Convert a value from Beam Row to a vlue used for Avro GenericRecord. */\n  private static @Nullable Object genericFromBeamField(\n      FieldType fieldType, org.apache.avro.Schema avroSchema, @Nullable Object value) {\n    TypeWithNullability typeWithNullability = new TypeWithNullability(avroSchema);\n    if (fieldType.getNullable() != typeWithNullability.nullable) {\n      throw new IllegalArgumentException(\n          \"FieldType \"\n              + fieldType\n              + \" and AVRO schema \"\n              + avroSchema\n              + \" don't have matching nullability\");\n    }\n\n    if (value == null) {\n      return value;\n    }\n\n    if (NUMERIC_CONVERTERS.containsKey(typeWithNullability.type)) {\n      return NUMERIC_CONVERTERS.get(typeWithNullability.type).apply((Number) value);\n    }\n\n    // TODO: should we use Avro Schema as the source-of-truth in general?\n    switch (fieldType.getTypeName()) {\n      case BYTE:","sourceCodeStart":1265,"sourceCodeEnd":1301,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java#L1265-L1301","documentation":"Thrown by AvroUtils.genericFromBeamField when converting a Beam Row field to an Avro GenericRecord value: the Beam FieldType's nullability (fieldType.getNullable()) does not match whether the target Avro schema is nullable (an unwrapped UNION with NULL). The library requires both sides to agree on nullability so the conversion stays type-safe and nulls are handled symmetrically.","triggerScenarios":"Calling toGenericRecord / toAvroType (directly or via AvroUtils.toGenericRecord or schema conversion APIs) with a Beam schema field and an Avro schema whose nullability disagree — e.g. a nullable Beam FieldType mapped to a plain (non-union) Avro schema, or a non-nullable Beam FieldType mapped to an Avro UNION ['null', T].","commonSituations":"Hand-written Avro schemas that forgot (or wrongly added) the ['null', T] union; Avro schemas regenerated after changing a Beam field to/from nullable; registerAvroSchema / Schemadrogel conversions where Beam schema was inferred from a Java class whose field nullability diverged from a stored .avsc file; version upgrades where one side's schema drifted.","solutions":["Align the Avro schema with the Beam FieldType nullability: wrap the Avro type in a union [\"null\", T] if the Beam field is nullable, or remove the union if it is not","Or make the Beam FieldType match the Avro schema: use Schema.FieldType.withNullable(true/false) on the offending field before conversion","Regenerate the Avro schema from the Beam schema (AvroUtils.toAvroSchema) instead of maintaining it by hand, so nullability cannot drift","If the schema came from a stored .avsc file, update it to reflect the current Beam schema"],"exampleFix":"// before: Beam field nullable, Avro schema not\nSchema.FieldType fieldType = Schema.FieldType.STRING.withNullable(true);\norg.apache.avro.Schema avroSchema = org.apache.avro.Schema.create(Type.STRING); // no null branch\n// after\norg.apache.avro.Schema avroSchema = org.apache.avro.Schema.createUnion(\n    org.apache.avro.Schema.create(Type.NULL), org.apache.avro.Schema.create(Type.STRING));","handlingStrategy":"validation","validationCode":"boolean nullabilityMatches(Schema.FieldType beamField, org.apache.avro.Schema avroSchema) {\n  boolean avroNullable = avroSchema.getType() == org.apache.avro.Schema.Type.UNION\n      && avroSchema.getTypes().stream().anyMatch(t -> t.getType() == org.apache.avro.Schema.Type.NULL);\n  return beamField.getNullable() == avroNullable;\n}","typeGuard":"if (!nullabilityMatches(fieldType, avroSchema)) {\n  throw new IllegalStateException(\"Fix schema nullability before conversion\");\n}","tryCatchPattern":"try {\n  GenericRecord record = AvroUtils.toGenericRecord(row, avroSchema);\n} catch (IllegalArgumentException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"don't have matching nullability\")) {\n    avroSchema = AvroUtils.toAvroSchema(beamSchema); // regenerate consistent schema\n  } else { throw e; }\n}","preventionTips":["Generate Avro schemas from Beam schemas (AvroUtils.toAvroSchema) instead of hand-writing them","Run a nullability-check pass comparing Beam FieldType.getNullable() with the Avro union in tests","When changing a field to/from nullable, update both Beam and Avro schemas in the same commit"],"tags":["java","avro","beam-schema","nullability","conversion"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}