apache/beam · error · IllegalArgumentException

Conflicting field types for field

Error message

Conflicting field types for field '%s': %s vs %s

What it means

UpgradeTableSchema.mergeField merges two BigQuery TableFieldSchema definitions for the same field. If the two definitions declare different field TYPES (e.g. STRING vs INTEGER), the merge is ambiguous, so the library fails fast with this IllegalArgumentException instead of silently picking one. This happens while upgrading/merging table schemas in the BigQuery IO connector.

Solutions

  1. Inspect both schemas and align the field type for the conflicting field (pick the correct BigQuery type in your source schema).
  2. If the type change is intentional, migrate the data first (change the column type in BigQuery) so both schema versions match.
  3. Ensure any automated schema generation (e.g. from Avro/JSON) consistently maps types so the same field never gets different types.

Example fix

// before
TableFieldSchema f1 = new TableFieldSchema().setName("id").setType("STRING");
TableFieldSchema f2 = new TableFieldSchema().setName("id").setType("INTEGER");
mergeField(f1, f2); // throws
// after
TableFieldSchema f2 = new TableFieldSchema().setName("id").setType("STRING");
mergeField(f1, f2); // OK
Defensive patterns

Strategy: validation

Validate before calling

Map<String, String> types1 = byName(fields1), types2 = byName(fields2);
for (String name : Sets.intersection(types1.keySet(), types2.keySet())) {
  if (!types1.get(name).equals(types2.get(name))) {
    throw new IllegalArgumentException("Type conflict on field " + name + ": " + types1.get(name) + " vs " + types2.get(name));
  }
}

Try / catch

try { merged = mergeFields(f1, f2); } catch (IllegalArgumentException e) { log.error("Schema merge failed: {}", e.getMessage()); throw new SchemaMigrationException(e); }

Prevention

When it happens

Trigger: Calling mergeFields on two schema field lists where the same-named field has f1.getType() != f2.getType() (e.g. legacy schema says STRING, upgraded schema says BYTES).

Common situations: Schema evolution mistakes: a column's type was changed between schema versions; hand-written merge/upgrade code supplies mismatched types; auto-generated schemas from different sources disagree on a field type.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ca90bd285d2222a6. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/UpgradeTableSchema.java:206

  private static List<TableFieldSchema> mergeFields(
      List<TableFieldSchema> fields1, List<TableFieldSchema> fields2) {
    // Use LinkedHashMap to preserve the order of fields
    Map<String, TableFieldSchema> mergedFieldsMap = Maps.newLinkedHashMap();

    // Add all fields from schema 1.
    fields1.forEach(f -> mergedFieldsMap.put(f.getName().toLowerCase(), f));

    // Merge or append fields from schema 2
    for (TableFieldSchema f2 : fields2) {
      String lowerName = f2.getName().toLowerCase();
      mergedFieldsMap.compute(lowerName, (k, v) -> v == null ? f2 : mergeField(v, f2));
    }
    return Lists.newArrayList(mergedFieldsMap.values());
  }

  private static TableFieldSchema mergeField(TableFieldSchema f1, TableFieldSchema f2) {
    if (!f1.getType().equals(f2.getType())) {
      throw new IllegalArgumentException(
          String.format(
              "Conflicting field types for field '%s': %s vs %s",
              f1.getName(), f1.getType(), f2.getType()));
    }

    TableFieldSchema.Builder builder = f1.toBuilder().mergeFrom(f2);
    builder.clearFields();

    // Handle mode weakening (NULLABLE > REQUIRED)
    TableFieldSchema.Mode mode1 =
        f1.getMode() == TableFieldSchema.Mode.MODE_UNSPECIFIED
            ? TableFieldSchema.Mode.NULLABLE
            : f1.getMode();
    TableFieldSchema.Mode mode2 =
        f2.getMode() == TableFieldSchema.Mode.MODE_UNSPECIFIED
            ? TableFieldSchema.Mode.NULLABLE
            : f2.getMode();

View on GitHub (pinned to 12126d8942)