apache/beam · error · NonDeterministicException

IterableLikeCoder can not guarantee deterministic ordering.

Error message

IterableLikeCoder can not guarantee deterministic ordering.

What it means

IterableLikeCoder.verifyDeterministic() always throws NonDeterministicException: for a general Iterable the encoded byte order depends on the iterable's iteration order and concrete type, so two equal objects can encode differently. Deterministic coders are required for grouping and keyed operations.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/IterableLikeCoder.java:170

      return decodeToIterable(elements, count, inStream);
    }
  }

  @Override
  public List<? extends Coder<?>> getCoderArguments() {
    return Arrays.asList(elementCoder);
  }

  /**
   * {@inheritDoc}
   *
   * @throws NonDeterministicException always. Encoding is not deterministic for the general {@link
   *     Iterable} case, as it depends upon the type of iterable. This may allow two objects to
   *     compare as equal while the encoding differs.
   */
  @Override
  public void verifyDeterministic() throws NonDeterministicException {
    throw new NonDeterministicException(
        this, "IterableLikeCoder can not guarantee deterministic ordering.");
  }

  /**
   * {@inheritDoc}
   *
   * @return {@code true} if the iterable is of a known class that supports lazy counting of byte
   *     size, since that requires minimal extra computation.
   */
  @Override
  public boolean isRegisterByteSizeObserverCheap(IterableT iterable) {
    return iterable instanceof ElementByteSizeObservableIterable;
  }

  @Override
  public void registerByteSizeObserver(IterableT iterable, ElementByteSizeObserver observer)
      throws Exception {
    if (iterable == null) {

View on GitHub (pinned to 12126d8942)

Solutions

1. Replace the iterable with a deterministically ordered List (sort elements in a DoFn before grouping).
2. Use ListCoder with sorted lists instead of SetCoder when order-independent semantics allow it and determinism is required.
3. Keep deterministic element coders and accept ordering guarantees from the producing transform.

Example fix

// before
PCollection<KV<String, Set<String>>> pc; // GroupByKey throws on SetCoder
// after
PCollection<KV<String, List<String>>> sorted =
    pc.apply(MapElements.via(kv -> KV.of(kv.getKey(), new ArrayList<>(new TreeSet<>(kv.getValue())))));

try { coder.verifyDeterministic(); } catch (Coder.NonDeterministicException e) { /* sort or switch to ListCoder */ }

When it happens

Trigger: Using ListCoder/IterableCoder/SetCoder elements (or nested in a KV coder) in GroupByKey, Combine.perKey, stateful processing, or any explicit verifyDeterministic() call when iteration order is not guaranteed.

Common situations: Grouping values that are Sets or unordered Iterables; pushing unordered collection data through key-based shuffles.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• CustomCoder implementations must override verifyDeterministi
• Floating point encodings are not guaranteed to be determinis
• cannot encode a null {iterableName}
• cannot encode a null Iterable
• NonDeterministicException(target, message, e)

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b3902254b8ac9812. Report an issue: GitHub.