apache/beam · error · RuntimeException

Error deserializing via Coder

Error message

Error deserializing via Coder

What it means

In SerializableViaCoder.get(), the cached encoded byte array is lazily decoded back into the value using the coder. A CoderException during decodeFromByteArray is wrapped in a RuntimeException with this message, typically after the value was re-materialized on a deserialization side (e.g. a worker).

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/testing/SerializableMatchers.java:797

    private Coder<T> coder;

    public SerializableViaCoder(Coder<T> coder, T value) {
      this.coder = coder;
      this.value = value;
      try {
        this.encodedValue = CoderUtils.encodeToByteArray(coder, value);
      } catch (CoderException exc) {
        throw new RuntimeException("Error serializing via Coder", exc);
      }
    }

    @Override
    public T get() {
      if (value == null) {
        try {
          value = CoderUtils.decodeFromByteArray(coder, encodedValue);
        } catch (CoderException exc) {
          throw new RuntimeException("Error deserializing via Coder", exc);
        }
      }
      return value;
    }
  }

  /**
   * Wraps any array with values that can be encoded via a {@link Coder} to make it {@link
   * Serializable}. This is not likely to be a good encoding, so should be used only for tests,
   * where data volume is small and minor costs are not critical.
   */
  private static class SerializableArrayViaCoder<T> implements SerializableSupplier<T[]> {
    /** Cached value that is not serialized. */
    @SuppressFBWarnings("SE_TRANSIENT_FIELD_NOT_RESTORED")
    private transient T @Nullable [] value;

    /** The bytes of {@link #value} when encoded via {@link #coder}. */
    private byte[] encodedValue;

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the coder dependency versions match between the job submission environment and workers.
2. Catch and inspect the CoderException cause to identify the exact decode failure.
3. Verify the encoded bytes are intact — re-encode and compare in a unit test.

Example fix

// before
Coder<T> coder = loadCoderDynamically();
// after
Coder<T> coder = theStableCoderMatchingEncodingVersion; // pin coder version in pom

// round-trip check before relying on the matcher
coder.verifyDeterministic(); CoderUtils.decodeFromByteArray(coder, CoderUtils.encodeToByteArray(coder, sample));

try { T v = viaCoder.get(); } catch (RuntimeException e) { logger.error("decode failed; check coder version skew", e.getCause()); }

When it happens

Trigger: Calling get() on a SerializableViaCoder whose encoded bytes were produced with a different coder version/type than the one deserializing, or whose bytes are corrupt/truncated after Java serialization round-trip.

Common situations: Coder class changed between build and runtime environments (dependency version skew driver vs worker); coders that are not deterministic across versions; classpath differences on cluster workers.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

• cannot encode a null Integer
• cannot encode a null Long
• cannot encode a null Short
• cannot encode a null BitSet
• Expected 0 or 1, got %d

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/94dffae15a5513c7. Report an issue: GitHub.