apache/beam · error · IllegalStateException

Cannot find coder for

Error message

Cannot find coder for %s  : 

What it means

HadoopFormatIO.getDefaultCoder() throws IllegalStateException("Cannot find coder for %s : ") when the CoderRegistry cannot provide a coder for the InputFormat's key or value type and the type is not a Hadoop Writable. Beam must be able to serialize these types to distribute read results.

Solutions

1. Make the key/value classes implement Writable, so WritableCoder is used automatically.
2. Register a coder for the type in the CoderRegistry before expanding: coderRegistry.registerCoderForClass(MyType.class, new MyTypeCoder()).
3. Set a coder explicitly on the transformed PCollection with setCoder(...) after mapping to a codable type.
4. Translate key/value to standard codable types (String, Long, Avro records) via withKeyTranslation/withValueTranslation.

Example fix

// before
conf.setClass("key.class", MyCustomKey.class, Object.class); // not Writable, no coder
// after
public class MyCustomKey implements Writable { ... }

Class<?> kv = conf.getClass("key.class", null);
if (!Writable.class.isAssignableFrom(kv)) { coderRegistry.registerCoderForClass(kv, myCustomCoder); }

boolean hasCoder(CoderRegistry r, Class<?> c) { try { r.getCoder(c); return true; } catch (CannotProvideCoderException e) { return Writable.class.isAssignableFrom(c); } }

try { pipeline.apply(read); } catch (IllegalStateException e) { if (e.getMessage().startsWith("Cannot find coder for")) { registerCoderFor(conf key/value classes); } throw e; }

When it happens

Trigger: Expanding a HadoopFormatIO.Read where the configured key class or value class is neither registry-codable (e.g. a custom POJO without registered coder) nor implements org.apache.hadoop.io.Writable.

Common situations: Custom InputFormat returning plain Java objects or third-party types; using types like java.net.URI or custom Thrift/Protobuf classes without registering coders; missing avro/writable conversions.

Related errors

• cannot encode a null Count-min Sketch
• cannot encode a null Integer
• cannot encode a null String
• cannot encode a null T-Digest sketch
• cannot encode a null ValueKind

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b2a49631ecdf0c36. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/hadoop-format/src/main/java/org/apache/beam/sdk/io/hadoop/format/HadoopFormatIO.java:606

            String.format(errorMsg, getinputFormatClass().getRawType(), inputType.getRawType()));
      }
    }

    /**
     * Returns the default coder for a given type descriptor. Coder Registry is queried for correct
     * coder, if not found in Coder Registry, then check if the type descriptor provided is of type
     * Writable, then WritableCoder is returned, else exception is thrown "Cannot find coder".
     */
    @SuppressWarnings({"unchecked", "WeakerAccess"})
    public <T> Coder<T> getDefaultCoder(TypeDescriptor<?> typeDesc, CoderRegistry coderRegistry) {
      Class classType = typeDesc.getRawType();
      try {
        return (Coder<T>) coderRegistry.getCoder(typeDesc);
      } catch (CannotProvideCoderException e) {
        if (Writable.class.isAssignableFrom(classType)) {
          return (Coder<T>) WritableCoder.of(classType);
        }
        throw new IllegalStateException(
            String.format("Cannot find coder for %s  : ", typeDesc) + e.getMessage(), e);
      }
    }
  }

  /**
   * Bounded source implementation for {@link HadoopFormatIO}.
   *
   * @param <K> Type of keys to be read.
   * @param <V> Type of values to be read.
   */
  public static class HadoopInputFormatBoundedSource<K, V> extends BoundedSource<KV<K, V>>
      implements Serializable {
    private final SerializableConfiguration conf;
    private final Coder<K> keyCoder;
    private final Coder<V> valueCoder;
    private final @Nullable SimpleFunction<?, K> keyTranslationFunction;
    private final @Nullable SimpleFunction<?, V> valueTranslationFunction;

View on GitHub (pinned to 12126d8942)