apache/beam · error · CannotProvideCoderException

Cannot provide because does not implement the interface

Error message

Cannot provide %s because %s does not implement the interface %s

What it means

WritableCoder.coderFor() refuses to supply a Coder when the requested TypeDescriptor does not implement org.apache.hadoop.io.Writable. The Beam Hadoop IO layer only knows how to serialize Hadoop Writable types, so it throws CannotProvideCoderException to let the pipeline fall back to another CoderProvider.

Solutions

1. Make the record class implement org.apache.hadoop.io.Writable (or WritableComparable) with write()/readFields().
2. Register or specify an explicit Coder for the type instead of relying on the Hadoop coder provider.
3. Wrap the data in an existing Writable type such as Text, IntWritable, or BytesWritable.

Example fix

// before
class MyRecord { String id; }
// after
class MyRecord implements Writable {
  String id;
  public void write(DataOutput out) throws IOException { out.writeUTF(id); }
  public void readFields(DataInput in) throws IOException { id = in.readUTF(); }
}

if (!org.apache.hadoop.io.Writable.class.isAssignableFrom(MyRecord.class)) {
  throw new IllegalArgumentException("MyRecord must implement Writable");
}

static <T> boolean isWritable(Class<T> cls) { return Writable.class.isAssignableFrom(cls); }

try {
  coder = coderProvider.coderFor(typeDesc, comps);
} catch (CannotProvideCoderException e) {
  coder = fallbackCoder; // e.g. SerializableCoder.of(typeDesc)
}

When it happens

Trigger: Using WritableCoder's coderFor() with a TypeDescriptor whose raw type does not implement Writable, e.g. asking for a coder for a plain POJO or java.lang.String through the Hadoop coder provider.

Common situations: Developers building Beam pipelines over HDFS inputs whose record class is a custom POJO that never implemented WritableComparable, or after refactoring a class to drop the Writable interface.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

• cannot encode a null Count-min Sketch
• cannot encode a null Integer
• cannot encode a null String
• cannot encode a null T-Digest sketch
• Cannot encode a null value.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/bc0cebdf153d05bd. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/hadoop-common/src/main/java/org/apache/beam/sdk/io/hadoop/WritableCoder.java:157

  @AutoService(CoderProviderRegistrar.class)
  public static class WritableCoderProviderRegistrar implements CoderProviderRegistrar {

    @Override
    public List<CoderProvider> getCoderProviders() {
      return Collections.singletonList(getCoderProvider());
    }
  }

  /** A {@link CoderProvider} for Hadoop {@link Writable writable types}. */
  private static class WritableCoderProvider extends CoderProvider {
    private static final TypeDescriptor<Writable> WRITABLE_TYPE = new TypeDescriptor<Writable>() {};

    @Override
    public <T> Coder<T> coderFor(
        TypeDescriptor<T> typeDescriptor, List<? extends Coder<?>> componentCoders)
        throws CannotProvideCoderException {
      if (!typeDescriptor.isSubtypeOf(WRITABLE_TYPE)) {
        throw new CannotProvideCoderException(
            String.format(
                "Cannot provide %s because %s does not implement the interface %s",
                WritableCoder.class.getSimpleName(), typeDescriptor, Writable.class.getName()));
      }

      try {
        @SuppressWarnings("unchecked")
        Coder<T> coder = WritableCoder.of((Class) typeDescriptor.getRawType());
        return coder;
      } catch (IllegalArgumentException e) {
        throw new CannotProvideCoderException(e);
      }
    }
  }
}

View on GitHub (pinned to 12126d8942)