What causes "Conflicting field modes for field '%s': %s vs %s" in apache/beam?

Make the field's mode consistent across both schema definitions (choose the correct mode in your source schema).

apache/beam · error · IllegalArgumentException

Conflicting field modes for field

Error message

Conflicting field modes for field '%s': %s vs %s

What it means

mergeField also reconciles the MODE of a field (NULLABLE, REQUIRED, REPEATED). NULLABLE can be merged with anything (the result becomes NULLABLE), and identical modes merge to themselves, but two different non-null modes (e.g. REQUIRED vs REPEATED) are irreconcilable, so the library throws this IllegalArgumentException.

Solutions

Make the field's mode consistent across both schema definitions (choose the correct mode in your source schema).
If one side should tolerate absence, set the mode to NULLABLE in one of the definitions — NULLABLE merges with any mode.
If a REQUIRED field really became REPEATED, migrate the data/model so both schemas describe the same cardinality.

Example fix

// before
TableFieldSchema f1 = new TableFieldSchema().setName("tags").setMode("REQUIRED");
TableFieldSchema f2 = new TableFieldSchema().setName("tags").setMode("REPEATED");
// after
TableFieldSchema f1 = new TableFieldSchema().setName("tags").setMode("REPEATED");
TableFieldSchema f2 = new TableFieldSchema().setName("tags").setMode("REPEATED");

Defensive patterns

Strategy: validation

Validate before calling

if (!Objects.equals(f1.getMode(), f2.getMode())
    && !TableFieldSchema.Mode.NULLABLE.equals(f1.getMode())
    && !TableFieldSchema.Mode.NULLABLE.equals(f2.getMode())) {
  throw new IllegalArgumentException("Mode conflict on field " + f1.getName());
}

Try / catch

try { merged = mergeFields(f1, f2); } catch (IllegalArgumentException e) { log.warn("Mode conflict, treating as NULLABLE: {}", e.getMessage()); return f1.toBuilder().setMode(NULLABLE).build(); }

Prevention

Default new fields to NULLABLE so they merge with any mode.
Never change REQUIRED to REPEATED in place — add a new field instead.
Codify mode rules in schema CI checks.

When it happens

Trigger: Merging two schema versions where the same field has f1.getMode() != f2.getMode() and neither is NULLABLE, e.g. REQUIRED in one and REPEATED in the other.

Common situations: A column changed from REQUIRED to REPEATED (or vice versa) between schema revisions; codegen emitted different mode defaults; manually edited table definitions disagree on mode.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d7d73f2147ba869e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/UpgradeTableSchema.java:234

    TableFieldSchema.Mode mode1 =
        f1.getMode() == TableFieldSchema.Mode.MODE_UNSPECIFIED
            ? TableFieldSchema.Mode.NULLABLE
            : f1.getMode();
    TableFieldSchema.Mode mode2 =
        f2.getMode() == TableFieldSchema.Mode.MODE_UNSPECIFIED
            ? TableFieldSchema.Mode.NULLABLE
            : f2.getMode();

    boolean isNull =
        ((mode1 == TableFieldSchema.Mode.NULLABLE) && (mode2 != TableFieldSchema.Mode.REPEATED))
            || (mode2 == TableFieldSchema.Mode.NULLABLE && mode1 != TableFieldSchema.Mode.REPEATED);
    if (isNull) {
      builder.setMode(TableFieldSchema.Mode.NULLABLE);
    } else if (mode1.equals(mode2)) {
      // Either merging REPEATED with REPEATED or REQUIRED with REQUIRED.
      builder.setMode(mode1);
    } else {
      throw new IllegalArgumentException(
          String.format(
              "Conflicting field modes for field '%s': %s vs %s",
              f1.getName(), f1.getMode(), f2.getMode()));
    }

    // Recursively merge nested fields if the type is STRUCT (Record)
    if (f1.getType() == TableFieldSchema.Type.STRUCT) {
      builder.addAllFields(mergeFields(f1.getFieldsList(), f2.getFieldsList()));
    }

    return builder.build();
  }

  public static boolean isPayloadSchemaOutOfDate(
      StorageApiWritePayload payload,
      ThrowingSupplier<byte[]> schemaHash,
      ThrowingSupplier<Descriptors.Descriptor> schemaDescriptor)
      throws Exception {

View on GitHub (pinned to 12126d8942)