apache/beam · error · RuntimeException

Couldn't infer Coder from

Error message

Couldn't infer Coder from 

What it means

KafkaIO.resolveCoder() can only infer a Coder for Deserializer implementations whose decode returns a known boxed type (byte[], Integer, Long, etc.). When the deserializer's return type is unrecognized, it throws a RuntimeException indicating the coder could not be inferred.

Solutions

1. Specify the coder explicitly via withCoder(), keyCoder()/valueCoder() instead of relying on inference.
2. Make the deserializer return one of the supported types (byte[], Integer, Long) or wrap it.
3. Use a known deserializer (ByteArrayDeserializer, Integer/Long deserializers).
4. Improve the code path by adding your type to resolveCoder's inference chain or registering a CoderProvider.

Example fix

// before
KafkaIO.<byte[], String>read().withValueDeserializerAndCoder(MyStringDeserializer.class) // implicit coder inference fails
// after
KafkaIO.<byte[], String>read()
  .withValueDeserializerAndCoder(MyStringDeserializer.class, StringUtf8Coder.of());

Class<?> rt = method.getReturnType();
boolean ok = rt==byte[].class||rt==Integer.class||rt==Long.class;

try { resolveCoder(d); } catch (RuntimeException e) { useExplicitCoder(); }

When it happens

Trigger: Using a custom consumer/producer deserializer class whose return type is not one of the supported types (e.g. String, Double, custom POJO, or an interface) when building via the external config path where the coder must be auto-resolved.

Common situations: Custom deserializers returning String or POJOs; third-party deserializers (e.g. Avro/Json) unsupported by the inference branch; specifying key/value deserializer classes without an explicit key/value coder.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• bad decoding function
• bad encoding function
• cannot encode a null BitSet
• cannot encode a null byte[]
• cannot encode a null Count-min Sketch

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9726030a331877c1. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java:1002

          }
        }
      }

      private static <T> Coder<T> resolveCoder(Class<Deserializer<T>> deserializer) {
        for (Method method : deserializer.getDeclaredMethods()) {
          if (method.getName().equals("deserialize")) {
            Class<?> returnType = method.getReturnType();
            if (returnType.equals(Object.class)) {
              continue;
            }
            if (returnType.equals(byte[].class)) {
              return (Coder<T>) NullableCoder.of(ByteArrayCoder.of());
            } else if (returnType.equals(Integer.class)) {
              return (Coder<T>) NullableCoder.of(VarIntCoder.of());
            } else if (returnType.equals(Long.class)) {
              return (Coder<T>) NullableCoder.of(VarLongCoder.of());
            } else {
              throw new RuntimeException("Couldn't infer Coder from " + deserializer);
            }
          }
        }
        throw new RuntimeException("Couldn't resolve coder for Deserializer: " + deserializer);
      }
    }

    /**
     * Exposes {@link KafkaIO.TypedWithoutMetadata} as an external transform for cross-language
     * usage.
     */
    @AutoService(ExternalTransformRegistrar.class)
    public static class External implements ExternalTransformRegistrar {

      // Using the transform name in the URN so that the corresponding transform can be easily
      // identified.
      public static final String URN_WITH_METADATA =
          "beam:transform:org.apache.beam:kafka_read_with_metadata:v1";

View on GitHub (pinned to 12126d8942)