apache/beam · error · CannotProvideCoderException

is not compatible with Avro

Error message

${typeDescriptor} is not compatible with Avro

What it means

AvroCoder's CoderProvider.coderFor attempts AvroCoder.of(typeDescriptor); if the type cannot be mapped to an Avro schema, Avro's library throws AvroRuntimeException, which is translated to CannotProvideCoderException telling the caller the type is not Avro-compatible. This is the standard Beam coder-inference failure path, not necessarily fatal unless no other coder provider can handle the type.

Solutions

1. Set an explicit coder with pipeline.getCoderRegistry().registerCoderForClass(type, AvroCoder.of(type, schema)) or .setCoder(...) on the PCollection
2. Add Avro reflect annotations (@AvroSchema, @Stringable, @Nullable) so Avro can derive a schema
3. Use a Beam-native coder (SerializableCoder/POJO) or convert the type to an Avro SpecificRecord
4. Inspect the wrapped AvroRuntimeException message to see which field/type failed and fix it

Example fix

// before
p.apply("read", ...).setCoder(??) // inference throws: Foo is not compatible with Avro
// after
output.setCoder(AvroCoder.of(Foo.class, schemaString));

try {
  AvroCoder.of(typeDescriptor);
  return true; // Avro-compatible
} catch (AvroRuntimeException e) {
  return false;
}

boolean isAvroCompatible(TypeDescriptor<?> td) {
  try { AvroCoder.of(td); return true; } catch (AvroRuntimeException e) { return false; }
}

try {
  coder = registry.getCoder(typeDescriptor);
} catch (CannotProvideCoderException e) {
  coder = SerializableCoder.of(typeDescriptor); // fallback for non-Avro types
}

When it happens

Trigger: Beam's default coder registry asks AvroCoder to infer a coder for a type with no Avro-reflectable schema: interfaces, abstract classes, types with unsupported fields, or GenericRecord without schema context passed through TypeDescriptor.

Common situations: PCollections of POJOs with unannotated complex fields (e.g. URI, BigInteger); types missing the org.apache.avro.reflect annotations; anonymous/inner classes; upgrading Beam and a previously reflected type no longer maps.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• AvroCoder for GenericRecord requires a schema
• Cannot provide coder for Create: The elements are not all…
• Cannot provide coder for elements of Create: For their…
• Cannot provide coder for elements of Create: For their…
• Field not nullable

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4c37faf95985753c. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/coders/AvroCoder.java:331

  public static CoderProvider getCoderProvider() {
    return new AvroCoderProvider();
  }

  /**
   * A {@link CoderProvider} that constructs an {@link AvroCoder} for Avro compatible classes.
   *
   * <p>It is unsafe to register this as a {@link CoderProvider} because Avro will reflectively
   * accept dangerous types such as {@link Object}.
   */
  static class AvroCoderProvider extends CoderProvider {
    @Override
    public <T> Coder<T> coderFor(
        TypeDescriptor<T> typeDescriptor, List<? extends Coder<?>> componentCoders)
        throws CannotProvideCoderException {
      try {
        return AvroCoder.of(typeDescriptor);
      } catch (AvroRuntimeException e) {
        throw new CannotProvideCoderException(
            String.format("%s is not compatible with Avro", typeDescriptor), e);
      }
    }
  }

  private final AvroDatumFactory<T> datumFactory;
  private final SerializableSchemaSupplier schemaSupplier;
  private final TypeDescriptor<T> typeDescriptor;

  private final List<String> nonDeterministicReasons;

  // Factories allocated by .get() are thread-safe and immutable.
  private static final EncoderFactory ENCODER_FACTORY = EncoderFactory.get();
  private static final DecoderFactory DECODER_FACTORY = DecoderFactory.get();

  /**
   * A {@link Serializable} object that holds the {@link String} version of a {@link Schema}. This
   * is paired with the {@link SerializableSchemaSupplier} via {@link Serializable}'s usage of the

View on GitHub (pinned to 12126d8942)