apache/beam · error · IllegalArgumentException

Cast isn't compatible using +validator()+: +reason

Error message

Cast isn't compatible using +validator()+:
	+reason

What it means

Cast.verifyCompatibility validates, before running, that every field of the input schema can be cast to the output schema under the configured validator (upcast/downcast/lossless-unsafe). It accumulates errors as path+message pairs and throws one IllegalArgumentException listing all incompatibilities.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/Cast.java:281

        || type == TypeName.INT32
        || type == TypeName.INT64;
  }

  /** Checks if type is decimal. */
  public static boolean isDecimal(TypeName type) {
    return type == TypeName.FLOAT || type == TypeName.DOUBLE || type == TypeName.DECIMAL;
  }

  public void verifyCompatibility(Schema inputSchema) {
    List<CompatibilityError> errors = validator().apply(inputSchema, outputSchema());

    if (!errors.isEmpty()) {
      String reason =
          errors.stream()
              .map(x -> Joiner.on('.').join(x.path()) + ": " + x.message())
              .collect(Collectors.joining("\n\t"));

      throw new IllegalArgumentException(
          "Cast isn't compatible using " + validator() + ":\n\t" + reason);
    }
  }

  @Override
  public PCollection<Row> expand(PCollection<T> input) {
    Schema inputSchema = input.getSchema();

    verifyCompatibility(inputSchema);

    return input
        .apply(
            ParDo.of(
                new DoFn<T, Row>() {
                  // TODO: This should be the same as resolved so that Beam knows which fields
                  // are being accessed. Currently Beam only supports wildcard descriptors.
                  // Once https://github.com/apache/beam/issues/18903 is fixed, fix this.
                  @FieldAccess("filterFields")

View on GitHub (pinned to 12126d8942)

Solutions

1. Read the reason list in the exception: each line is 'field.path: message' identifying the exact incompatibility.
2. Align the output schema with the input types, or widen input types before the cast.
3. Choose a permissive validator, e.g. Cast.validator(Cast.DefaultValidator.UNSAFE) for lossy casts, or write a custom CastValidator.
4. Use withUnsafe or drop/rename the offending fields before casting.

Example fix

// before (INT64 -> INT32 rejected by lossless validator)
rows.apply(Cast.to(outSchema).validator(Cast.DefaultValidator.LOSSLESS));
// after: allow unsafe/lossy numeric casts
rows.apply(Cast.to(outSchema).validator(Cast.DefaultValidator.UNSAFE));

for (Schema.Field in : inSchema.getFields()) {
  Schema.Field out = outSchema.getField(in.getName());
  if (out != null && !Cast.canCast(in.getType(), out.getType(), validator)) {
    throw new IllegalStateException("incompatible field: " + in.getName());
  }
}

try { rows.apply(Cast.to(outSchema).validator(v)); }
catch (IllegalArgumentException e) {
  if (e.getMessage().startsWith("Cast isn't compatible")) { logInvalidPaths(e.getMessage()); }
  else throw e;
}

When it happens

Trigger: Applying Cast.to(outputSchema) (or castRow with a validator) via PCollection.apply where field types are incompatible, e.g. casting an ARRAY to an INT, or a downcast like INT64->INT32 with Cast.validator(LosslessValidator).

Common situations: Aligning two pipeline stages' schemas after one side changed types; forcing a wide-to-narrow numeric cast that the chosen validator rejects; renaming/reordering fields so required outputs are missing in the input.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Can't cast non-numeric types: +input
• Can't cast numbers to non-numeric type: +output
• input should be array, map, numeric or row
• Document id field '{documentIdField}' must be set on input r
• Collection element type cannot be null.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/6509014719b0bcfa. Report an issue: GitHub.