apache/beam · error · NonDeterministicException

Java Serialization may be non-deterministic.

Error message

Java Serialization may be non-deterministic.

What it means

SerializableCoder.verifyDeterministic() always throws NonDeterministicException: Java serialization output is not guaranteed to be byte-identical for equal objects (field order, writeObject overrides, identity handling), so Beam cannot treat SerializableCoder as deterministic. Operations requiring deterministic keys reject it.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/SerializableCoder.java:214

  @Override
  public T decode(InputStream inStream) throws IOException, CoderException {
    try {
      ObjectInputStream ois = new ObjectInputStream(inStream);
      return type.cast(ois.readObject());
    } catch (ClassNotFoundException e) {
      throw new CoderException("unable to deserialize record", e);
    }
  }

  /**
   * {@inheritDoc}
   *
   * @throws NonDeterministicException always. Java serialization is not deterministic with respect
   *     to {@link Object#equals} for all types.
   */
  @Override
  public void verifyDeterministic() throws NonDeterministicException {
    throw new NonDeterministicException(this, "Java Serialization may be non-deterministic.");
  }

  @Override
  @SuppressWarnings("EqualsGetClass")
  public boolean equals(@Nullable Object other) {
    return !(other == null || getClass() != other.getClass())
        && type == ((SerializableCoder<?>) other).type;
  }

  @Override
  public int hashCode() {
    return type.hashCode();
  }

  @Override
  public TypeDescriptor<T> getEncodedTypeDescriptor() {
    if (typeDescriptor == null) {
      typeDescriptor = TypeDescriptor.of(type);

View on GitHub (pinned to 12126d8942)

Solutions

1. Write a custom deterministic Coder<T> (fixed field order, canonical representation) and use it for keys
2. Use AvroCoder.of(T.class) or schema-based RowCoder, which have deterministic encodings
3. Convert the key type to a primitive/String that Beam can encode deterministically

Example fix

// before
.apply(WithKeys.of(rec -> rec)) // SerializableCoder<MyRec> key -> NonDeterministicException
// after
.apply(WithKeys.of(rec -> rec.getId())) // String key, deterministic coder

try { coder.verifyDeterministic(); } catch (NonDeterministicException e) { /* pick deterministic coder */ }

null

try { coder.verifyDeterministic(); } catch (NonDeterministicException e) { switch to AvroCoder or custom deterministic coder }

When it happens

Trigger: Using SerializableCoder for a key or element in determinism-sensitive operations like GroupByKey, or calling verifyDeterministic() during CoderRegistry validation.

Common situations: Grouping POJOs encoded with SerializableCoder; switching a deterministic custom coder to SerializableCoder and failing pipeline validation.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• NonDeterministicException(target, message, e)
• Floating point encodings are not guaranteed to be determinis
• Ordering of entries in a Map may be non-deterministic.
• Cannot provide SerializableCoder because {} does not impleme
• Ordering of elements in a set may be non-deterministic.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3134d15e2d5b9d17. Report an issue: GitHub.