{"record":{"id":"059daebc7f9121f8","repo":"apache/beam","slug":"cannot-merge-schemas-with-different-numbers-of-fields","errorCode":null,"errorMessage":"Cannot merge schemas with different numbers of fields. schema1: +schema1+ schema2: +schema2","messagePattern":"Cannot merge schemas with different numbers of fields\\. schema1: \\+schema1\\+ schema2: \\+schema2","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SchemaUtils.java","lineNumber":44,"sourceCode":"import org.apache.beam.sdk.values.Row;\n\n/** A set of utility functions for schemas. */\n@SuppressWarnings({\n  \"nullness\" // TODO(https://github.com/apache/beam/issues/20497)\n})\npublic class SchemaUtils {\n  private static final String INDENT = \"  \";\n\n  /**\n   * Given two schema that have matching types, return a nullable-widened schema.\n   *\n   * <p>The schemas must have matching types, except for field names which can differ. The returned\n   * schema will contain the field names in the first schema. All field types will be nullable if\n   * the corresponding field type is nullable in either of the input schemas.\n   */\n  public static Schema mergeWideningNullable(Schema schema1, Schema schema2) {\n    if (schema1.getFieldCount() != schema2.getFieldCount()) {\n      throw new IllegalArgumentException(\n          \"Cannot merge schemas with different numbers of fields. \"\n              + \"schema1: \"\n              + schema1\n              + \" schema2: \"\n              + schema2);\n    }\n    Schema.Builder builder = Schema.builder();\n    for (int i = 0; i < schema1.getFieldCount(); ++i) {\n      String name = schema1.getField(i).getName();\n      builder.addField(\n          name, widenNullableTypes(schema1.getField(i).getType(), schema2.getField(i).getType()));\n    }\n    return builder.build();\n  }\n\n  static FieldType widenNullableTypes(FieldType fieldType1, FieldType fieldType2) {\n    if (fieldType1.getTypeName() != fieldType2.getTypeName()) {\n      throw new IllegalArgumentException(","sourceCodeStart":26,"sourceCodeEnd":62,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/SchemaUtils.java#L26-L62","documentation":"mergeWideningNullable merges two Schemas field-by-field, producing a schema whose field types are nullable if either input's corresponding field is nullable. It requires both schemas to have exactly the same number of fields; otherwise it throws IllegalArgumentException with both schemas rendered in the message. This is a fail-fast precondition check for schema evolution/merging.","triggerScenarios":"Calling SchemaUtils.mergeWideningNullable(schema1, schema2) where schema1.getFieldCount() != schema2.getFieldCount() — e.g. one side of a merge added or dropped a field (schema evolution, codegen drift between pipeline versions).","commonSituations":"Recomputing an output schema after a Beam schema evolved (new column added upstream); comparing a generated schema (e.g. from Avro/POJO autogen) against a hand-written one; running an old pipeline definition against data whose schema was extended in a newer version.","solutions":["Ensure both schemas have the same field count before merging; add the missing field to the shorter schema.","If schemas legitimately diverged, use Schema.mergeSchemas / Schema.builder() to build a union schema instead of mergeWideningNullable.","Verify both schemas are generated from the same source-of-truth (same class/Avro file/version) and redeploy consistent code.","Catch IllegalArgumentException and log both schemas to identify which field was added/removed."],"exampleFix":"// before\nSchema merged = SchemaUtils.mergeWideningNullable(oldSchema, newSchema); // throws: newSchema has an extra field\n// after\nif (oldSchema.getFieldCount() == newSchema.getFieldCount()) {\n  Schema merged = SchemaUtils.mergeWideningNullable(oldSchema, newSchema);\n} else {\n  Schema merged = SchemaUtils.mergeSchemas(oldSchema, newSchema); // union merge\n}","handlingStrategy":"validation","validationCode":"if (schema1.getFieldCount() != schema2.getFieldCount()) {\n  throw new IllegalStateException(\"mergeWideningNullable precondition failed: \"\n      + schema1.getFieldCount() + \" vs \" + schema2.getFieldCount() + \" fields\");\n}\nSchema merged = SchemaUtils.mergeWideningNullable(schema1, schema2);","typeGuard":"boolean canWidenMerge(Schema s1, Schema s2) {\n  return s1.getFieldCount() == s2.getFieldCount();\n}","tryCatchPattern":"try {\n  Schema merged = SchemaUtils.mergeWideningNullable(schema1, schema2);\n} catch (IllegalArgumentException e) {\n  LOG.error(\"Field count mismatch merging schemas: {}\", e.getMessage());\n  throw new SchemaMergeException(e);\n}","preventionTips":["Generate both schemas from the same source definition and version them together.","Assert field-count equality in a unit test before any schema merge runs.","Prefer SchemaUtils.mergeSchemas when schemas may legitimately diverge."],"tags":["java","beam","schema","illegal-argument"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}