apache/beam · critical · CoderException

`UnknownCoderWrapper` was used to perform an actual decoding

Error message

`UnknownCoderWrapper` was used to perform an actual decoding in the Java SDK. Potentially a Java transform is being followed by a cross-language transform that uses a coder that is not available in the Java SDK. Please make sure that Python transforms at the multi-language boundary use Beam portable coders.

What it means

The decode() counterpart of UnknownCoderWrapper: when the Java SDK must deserialize elements from a PCollection produced upstream by a cross-language transform whose coder it cannot resolve, decode() intentionally throws this CoderException. It signals the multi-language boundary is not using Beam portable coders.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/util/construction/UnknownCoderWrapper.java:55

  }

  public static UnknownCoderWrapper of(String urn, byte[] payload) {
    return new UnknownCoderWrapper(urn, payload);
  }

  @Override
  public void encode(Object value, OutputStream outStream) throws CoderException, IOException {
    throw new CoderException(
        "`UnknownCoderWrapper` was used to perform an actual encoding in the Java SDK. "
            + "Potentially a `PCollection` that was generated by a cross-language transform, "
            + "that uses a coder that is not available in the Java SDK, is being consumed by a Java"
            + " transform. Please make sure that cross-language transforms at the language "
            + "boundary use Beam portable coders.");
  }

  @Override
  public Object decode(InputStream inStream) throws CoderException, IOException {
    throw new CoderException(
        "`UnknownCoderWrapper` was used to perform an actual decoding in the Java SDK. "
            + "Potentially a Java transform is being followed by a cross-language transform that "
            + "uses a coder that is not available in the Java SDK. Please make sure that Python "
            + "transforms at the multi-language boundary use Beam portable coders.");
  }

  public String getUrn() {
    return urn;
  }

  public byte[] getPayload() {
    return payload;
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the coder used at the multi-language boundary is a Beam portable coder resolvable in both the Java and Python SDKs.
2. If you own the Python transform, change its input/output coder registration to a portable (URN-based) coder.
3. Register a mapping for the unknown coder URN in the Java coder registry so a concrete Java coder is resolved instead of the wrapper.
4. Align Beam SDK versions across Java, Python, and the expansion service so coder URNs are mutually known.

Example fix

// before: piping a Java PCollection into a Python transform with a custom Python coder
PCollection<MyType> out = pc.apply("PyStep", PyTransform.of(args));
// after: explicitly set a portable coder (e.g. beam row / protobuf-based) on the boundary
PCollection<MyType> out = pc
    .setCoder(MyPortableRowCoder.of())
    .apply("PyStep", PyTransform.of(args));

// Before piping a Java PCollection into a cross-language transform, ensure the coder is portable
Coder<?> c = input.getCoder();
if (c instanceof UnknownCoderWrapper) {
  throw new IllegalStateException(
      "Input to cross-language transform uses an unknown coder; set a portable coder first.");
}

boolean isUnknownCoder(Coder<?> coder) {
  return coder instanceof org.apache.beam.sdk.util.construction.UnknownCoderWrapper;
}

try {
  input.apply("CrossLanguageStep", pyTransform);
} catch (CoderException e) {
  if (String.valueOf(e.getMessage()).contains("UnknownCoderWrapper")) {
    // set an explicit portable coder on the boundary PCollection and resubmit
  }
  throw e;
}

When it happens

Trigger: A Java transform (or the runner materializing data) reads a PCollection whose coder is an UnknownCoderWrapper — i.e., an upstream Java transform is followed by a cross-language transform using a coder not available in the Java SDK — and UnknownCoderWrapper.decode(InputStream) is invoked.

Common situations: Java pipeline step feeding a Python transform whose input coder was overridden with a Python-specific custom coder; expansion service registers coders under URNs the Java SDK doesn't know; Beam version skew between the two SDKs.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• `UnknownCoderWrapper` was used to perform an actual encoding
• Invalid encoded string length: {}
• error when decoding a textual integer
• EOF encountered decoding a ValueKind
• Unknown ValueKind number: {}

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/197b7f9aec07aac5. Report an issue: GitHub.