apache/beam · error · CoderException

Unknown encoding

Error message

Unknown encoding 

What it means

PaneInfoCoder.encode switches over the Encoding chosen for a PaneInfo value; if chooseEncoding ever returns an encoding without a write case, the default branch throws CoderException("Unknown encoding "). This is an internal exhaustiveness guard rather than a user-input error.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/windowing/PaneInfo.java:390

    public void encode(PaneInfo value, final OutputStream outStream)
        throws CoderException, IOException {
      Encoding encoding = chooseEncoding(value);
      byte elementMetadata = value.containsElementMetadata ? ELEMENT_METADATA_MASK : 0x00;
      switch (chooseEncoding(value)) {
        case FIRST:
          outStream.write(value.encodedByte | elementMetadata);
          break;
        case ONE_INDEX:
          outStream.write(value.encodedByte | encoding.tag | elementMetadata);
          VarInt.encode(value.index, outStream);
          break;
        case TWO_INDICES:
          outStream.write(value.encodedByte | encoding.tag | elementMetadata);
          VarInt.encode(value.index, outStream);
          VarInt.encode(value.nonSpeculativeIndex, outStream);
          break;
        default:
          throw new CoderException("Unknown encoding " + encoding);
      }
    }

    @Override
    protected long getEncodedElementByteSize(PaneInfo value) throws Exception {
      Encoding encoding = chooseEncoding(value);
      switch (encoding) {
        case FIRST:
          return 1;
        case ONE_INDEX:
          return 1L + VarInt.getLength(value.index);
        case TWO_INDICES:
          return 1L + VarInt.getLength(value.index) + VarInt.getLength(value.nonSpeculativeIndex);
        default:
          throw new CoderException("Unknown encoding " + encoding);
      }
    }

View on GitHub (pinned to 12126d8942)

Solutions

1. Align the Apache Beam SDK version of user code and the runner/worker harness (e.g. set worker image / --sdkLocation to the same version).
2. Run mvn dependency:tree (or equivalent) to detect and exclude duplicate/mixed beam-sdks-java-core versions on the classpath.
3. Upgrade all components to a release where Encoding and its coder switch are consistent.
4. Catch CoderException and log the encoding value to confirm the version-mismatch hypothesis before redeploying.

Example fix

// before (pom.xml)
<dependency><groupId>org.apache.beam</groupId><artifactId>beam-sdks-java-core</artifactId><version>2.40.0</version></dependency>
// runner worker image: 2.50.0
// after
<dependency><groupId>org.apache.beam</groupId><artifactId>beam-sdks-java-core</artifactId><version>2.50.0</version></dependency>
// runner worker image: 2.50.0 (aligned)

// check for version skew before deploying:
// org.apache.beam.sdk.Pipeline: SDK version == runner harness version
String sdk = Pipeline.class.getPackage().getImplementationVersion();
assert sdk.equals(runnerWorkerVersion) : "Beam version mismatch: " + sdk + " vs " + runnerWorkerVersion;

try {
  coder.encode(value, out, Coder.Context.OUTER);
} catch (CoderException e) {
  LOG.error("PaneInfo encode failed (likely Beam version skew): {}", e.getMessage(), e);
  throw new IllegalStateException("Align SDK and runner Beam versions", e);
}

When it happens

Trigger: A new PaneInfo.Encoding enum constant added in a newer Beam version being encoded by an older switch statement (mixed Beam versions in classpath or runner harness), or programmatic misuse passing a null/unsupported encoding into the coder path.

Common situations: Runner harness and user code on mismatched Beam versions (e.g. Dataflow/Flink worker images older than the SDK); shading/dependency conflicts leaving two Beam versions on the classpath.

Related errors

• cannot encode a null Integer
• cannot encode a null Long
• cannot encode a null Short
• cannot encode a null BitSet
• cannot encode a null byte[]

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/bcf5ba4aabad1a6e. Report an issue: GitHub.