{"record":{"id":"c4ec3ba174ef592d","repo":"apache/beam","slug":"couldn-t-resolve-coder-for-deserializer","errorCode":null,"errorMessage":"Couldn't resolve coder for Deserializer: ","messagePattern":"Couldn't resolve coder for Deserializer: ","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java","lineNumber":1006,"sourceCode":"      private static <T> Coder<T> resolveCoder(Class<Deserializer<T>> deserializer) {\n        for (Method method : deserializer.getDeclaredMethods()) {\n          if (method.getName().equals(\"deserialize\")) {\n            Class<?> returnType = method.getReturnType();\n            if (returnType.equals(Object.class)) {\n              continue;\n            }\n            if (returnType.equals(byte[].class)) {\n              return (Coder<T>) NullableCoder.of(ByteArrayCoder.of());\n            } else if (returnType.equals(Integer.class)) {\n              return (Coder<T>) NullableCoder.of(VarIntCoder.of());\n            } else if (returnType.equals(Long.class)) {\n              return (Coder<T>) NullableCoder.of(VarLongCoder.of());\n            } else {\n              throw new RuntimeException(\"Couldn't infer Coder from \" + deserializer);\n            }\n          }\n        }\n        throw new RuntimeException(\"Couldn't resolve coder for Deserializer: \" + deserializer);\n      }\n    }\n\n    /**\n     * Exposes {@link KafkaIO.TypedWithoutMetadata} as an external transform for cross-language\n     * usage.\n     */\n    @AutoService(ExternalTransformRegistrar.class)\n    public static class External implements ExternalTransformRegistrar {\n\n      // Using the transform name in the URN so that the corresponding transform can be easily\n      // identified.\n      public static final String URN_WITH_METADATA =\n          \"beam:transform:org.apache.beam:kafka_read_with_metadata:v1\";\n      public static final String URN_WITHOUT_METADATA =\n          \"beam:transform:org.apache.beam:kafka_read_without_metadata:v1\";\n\n      @Override","sourceCodeStart":988,"sourceCodeEnd":1024,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java#L988-L1024","documentation":"KafkaIO's external-transform builder calls resolveCoder to derive a Coder for a given key/value Deserializer class. It only recognizes a small set of well-known deserializers (ByteArray, Long, Integer, String, Avro/Protobuf via SchemaRegistry); for anything else it cannot infer a Coder and throws. This happens when the KafkaIO read is exported to a cross-language pipeline and the configured deserializer is not one of the supported types.","triggerScenarios":"Calling setupExternalBuilder / expanding an external KafkaIO.Read (e.g. via SchemaTransform or Python xlang) with a keyDeserializer or valueDeserializer class not in resolveCoder's known list, or a custom org.apache.kafka.common.serialization.Deserializer implementation.","commonSituations":"Using custom deserializers (e.g. company-specific JSON deserializer), or deserializers like LongSerializer without nullable wrapper in external/cross-language mode; also typos in the deserializer class name.","solutions":["Use a supported deserializer: ByteArrayDeserializer, StringDeserializer, LongSerializer/Integer variants, or KafkaAvroDeserializer/ProtobufConfluentByteUtils-based ones.","Wrap byte-array style deserializers so the inferred coder is NullableCoder(ByteArrayCoder) where required.","If a custom deserializer is required, don't use the external/cross-language path; build the read directly in Java with explicit .withKeyCoder/.withValueCoder.","Verify the deserializer class name string resolves to the intended class (a wrong name yields the generic failure)."],"exampleFix":"// before\n.withValueDeserializerAndCoder(MyCustomDeserializer.class, myCoder) // external path\n// after\n.withValueDeserializer(ByteArrayDeserializer.class) // resolvable by external builder","handlingStrategy":"validation","validationCode":"Set<String> supported = Set.of(\"ByteArrayDeserializer\",\"StringDeserializer\",\"LongDeserializer\",\"IntegerDeserializer\",\"ByteBufferDeserializer\",\"KafkaAvroDeserializer\");\nif (!supported.stream().anyMatch(d -> deserializerCls.getName().contains(d))) {\n  throw new IllegalArgumentException(\"External KafkaIO read cannot infer coder for \" + deserializerCls.getName());\n}","typeGuard":"boolean isExternallyResolvable(Class<?> d) {\n  return Arrays.stream(KafkaIO.Read.Builder.resolveCoder(d).getClass().getDeclaredFields()).count() >= 0\n      && (Coder.class.isAssignableFrom(d.getClass()) || SupportedDeserializers.ALL.contains(d.getName()));\n}","tryCatchPattern":null,"preventionTips":["Stick to ByteArray/String/Long deserializers for cross-language Kafka reads","Test external expansion locally with DirectRunner before production","Convert non-byte payloads downstream, not at the deserializer level"],"tags":["java","kafka","cross-language","coder"],"backgroundTag":"unsupported-config-value","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}