apache/beam · error · NonDeterministicException

NonDeterministicException(target, message, e)

Error message

NonDeterministicException(target, message, e)

What it means

Coder.verifyDeterministic(target, message, coders) aggregates determinism violations from component coders: when any component coder fails its own verifyDeterministic, it throws NonDeterministicException against the target with the given message. It means a composite coder is non-deterministic because one of its component coders is.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/Coder.java:201

   *   <li>the {@code Coder} always produces a canonical encoding, which is the same for an instance
   *       of an object even if produced on different computers at different times.
   * </ul>
   *
   * @throws Coder.NonDeterministicException if this coder is not deterministic.
   */
  public abstract void verifyDeterministic() throws Coder.NonDeterministicException;

  /**
   * Verifies all of the provided coders are deterministic. If any are not, throws a {@link
   * NonDeterministicException} for the {@code target} {@link Coder}.
   */
  public static void verifyDeterministic(Coder<?> target, String message, Iterable<Coder<?>> coders)
      throws NonDeterministicException {
    for (Coder<?> coder : coders) {
      try {
        coder.verifyDeterministic();
      } catch (NonDeterministicException e) {
        throw new NonDeterministicException(target, message, e);
      }
    }
  }

  public static <T> long getEncodedElementByteSizeUsingCoder(Coder<T> target, T value)
      throws Exception {
    return target.getEncodedElementByteSize(value);
  }

  /**
   * Verifies all of the provided coders are deterministic. If any are not, throws a {@link
   * NonDeterministicException} for the {@code target} {@link Coder}.
   */
  public static void verifyDeterministic(Coder<?> target, String message, Coder<?>... coders)
      throws NonDeterministicException {
    verifyDeterministic(target, message, Arrays.asList(coders));
  }

View on GitHub (pinned to 12126d8942)

Solutions

1. Replace the non-deterministic component with a deterministic encoding, e.g. encode doubles via Double.toString or a scaled long
2. Use a deterministic wrapper coder for the offending type
3. Change the GBK key type to one with a deterministic coder (string, long, byte[] with ByteArrayCoder)
4. Remove the offending field from the key

Example fix

// before
Coder<MyKey> coder = new MyKeyCoder(DoubleCoder.of(), StringUtf8Coder.of());
// after
Coder<MyKey> coder = new MyKeyCoder(
    StringUtf8Coder.of(), // double encoded via Double.toString -> deterministic
    StringUtf8Coder.of());

try { coder.verifyDeterministic(); } catch (Coder.NonDeterministicException e) { /* fix coder before launching pipeline */ }

try { Coder.verifyDeterministic(targetCoder, "key coder must be deterministic", componentCoders); } catch (Coder.NonDeterministicException e) { throw new IllegalArgumentException("Use a deterministic key encoding", e); }

When it happens

Trigger: Calling verifyDeterministic on a composite coder (e.g. a custom coder for a struct/Pair) whose component list includes a known non-deterministic coder such as DoubleCoder, BigDecimalCoder, or a coder for unordered collections.

Common situations: Using a struct with a double/float/BigDecimal field as a GroupByKey key, pipeline-construction key validation rejecting a coder, Beam GBK setup failing on non-deterministic keys.

Related errors

• Floating point encodings are not guaranteed to be determinis
• Ordering of entries in a Map may be non-deterministic.
• Java Serialization may be non-deterministic.
• Ordering of elements in a set may be non-deterministic.
• Ordering of entries in a Map may be non-deterministic.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/bb5e196c1ac28252. Report an issue: GitHub.