apache/beam · critical · CoderException

`UnknownCoderWrapper` was used to perform an actual encoding

Error message

`UnknownCoderWrapper` was used to perform an actual encoding in the Java SDK. Potentially a `PCollection` that was generated by a cross-language transform, that uses a coder that is not available in the Java SDK, is being consumed by a Java transform. Please make sure that cross-language transforms at the language boundary use Beam portable coders.

What it means

UnknownCoderWrapper is a placeholder coder created during pipeline graph translation when a cross-language (e.g. Python) transform produces a PCollection whose coder the Java SDK cannot resolve. It exists only to keep the graph intact; calling its encode() intentionally throws this CoderException because the Java SDK has no real implementation for that coder.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/UnknownCoderWrapper.java:45

 * Represents a {@code Coder} that is not defined in Java SDK, for example, a coder that is
 * available in an external SDK that cannot be fully interpretted in the Java SDK.
 */
public class UnknownCoderWrapper extends AtomicCoder<Object> {
  private String urn;
  private byte[] payload;

  private UnknownCoderWrapper(String urn, byte[] payload) {
    this.urn = urn;
    this.payload = payload;
  }

  public static UnknownCoderWrapper of(String urn, byte[] payload) {
    return new UnknownCoderWrapper(urn, payload);
  }

  @Override
  public void encode(Object value, OutputStream outStream) throws CoderException, IOException {
    throw new CoderException(
        "`UnknownCoderWrapper` was used to perform an actual encoding in the Java SDK. "
            + "Potentially a `PCollection` that was generated by a cross-language transform, "
            + "that uses a coder that is not available in the Java SDK, is being consumed by a Java"
            + " transform. Please make sure that cross-language transforms at the language "
            + "boundary use Beam portable coders.");
  }

  @Override
  public Object decode(InputStream inStream) throws CoderException, IOException {
    throw new CoderException(
        "`UnknownCoderWrapper` was used to perform an actual decoding in the Java SDK. "
            + "Potentially a Java transform is being followed by a cross-language transform that "
            + "uses a coder that is not available in the Java SDK. Please make sure that Python "
            + "transforms at the multi-language boundary use Beam portable coders.");
  }

  public String getUrn() {
    return urn;

View on GitHub (pinned to 12126d8942)

Solutions

1. Make the cross-language transform's output use a Beam portable coder (a coder with a well-known URN resolvable in both SDKs).
2. If you own the Python transform, wrap custom coders with coders registry entries that map to portable URNs instead of language-specific coders.
3. On the Java side, register/re-map the unknown coder URN to a known Java coder in the coder registry used by expansion.
4. Upgrade both SDKs to a version that supports the coder; verify the expansion service and pipeline environments use compatible Beam versions.

Example fix

// before (Python side): custom non-portable coder on output PCollection
custom_pcoll = beam.Map(my_fn).with_output_types(MyCustomType)
// after: annotate the transform to use a portable coder at the boundary
custom_pcoll = beam.Map(my_fn).with_output_types(MyCustomType)
# and register: MyCustomCoder implements to_type_hint/to_runner_api with a portable URN

// Before consuming a cross-language output in Java, verify the coder is not the unknown wrapper
Coder<?> c = output.getCoder();
if (c instanceof UnknownCoderWrapper) {
  throw new IllegalStateException(
      "Cross-language output uses an unknown coder URN " + ((UnknownCoderWrapper) c).getUrn()
      + "; register a portable Java coder before consuming.");
}

boolean isUnknownCoder(Coder<?> coder) {
  return coder instanceof org.apache.beam.sdk.util.construction.UnknownCoderWrapper;
}

try {
  output.apply("JavaConsumer", javaTransform);
} catch (CoderException e) {
  if (String.valueOf(e.getMessage()).contains("UnknownCoderWrapper")) {
    // re-map the coder URN to a concrete Java coder, then resubmit
  }
  throw e;
}

When it happens

Trigger: A Java transform consumes the output of a cross-language transform whose PCollection uses a non-portable (language-specific) coder, and the runtime actually attempts to serialize an element with UnknownCoderWrapper.encode(Object, OutputStream).

Common situations: See trigger scenarios.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• `UnknownCoderWrapper` was used to perform an actual decoding
• cannot encode a null Integer
• cannot encode a null Long
• cannot encode a null Short
• cannot encode a null BitSet

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/230c631b3f8a3791. Report an issue: GitHub.