apache/beam · error · NonDeterministicException

Ordering of entries in a Map may be non-deterministic.

Error message

Ordering of entries in a Map may be non-deterministic.

What it means

MapCoder.verifyDeterministic() always throws NonDeterministicException because Java Maps (e.g. HashMap) iterate in arbitrary order, so encoding the same map twice can yield different bytes. Beam requires deterministic encodings in operations that group and re-encode keys (e.g. GroupByKey on coded keys) to guarantee correctness.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/MapCoder.java:141

   * {@inheritDoc}
   *
   * @return a {@link List} containing the key coder at index 0 at the and value coder at index 1.
   */
  @Override
  public List<? extends Coder<?>> getCoderArguments() {
    return Arrays.asList(keyCoder, valueCoder);
  }

  /**
   * {@inheritDoc}
   *
   * @throws NonDeterministicException always. Not all maps have a deterministic encoding. For
   *     example, {@code HashMap} comparison does not depend on element order, so two {@code
   *     HashMap} instances may be equal but produce different encodings.
   */
  @Override
  public void verifyDeterministic() throws NonDeterministicException {
    throw new NonDeterministicException(
        this, "Ordering of entries in a Map may be non-deterministic.");
  }

  @Override
  public boolean consistentWithEquals() {
    return keyCoder.consistentWithEquals() && valueCoder.consistentWithEquals();
  }

  @Override
  public Object structuralValue(Map<K, V> value) {
    if (consistentWithEquals()) {
      return value;
    } else {
      Map<Object, Object> ret = Maps.newHashMapWithExpectedSize(value.size());
      for (Map.Entry<K, V> entry : value.entrySet()) {
        ret.put(
            keyCoder.structuralValue(entry.getKey()), valueCoder.structuralValue(entry.getValue()));
      }

View on GitHub (pinned to 12126d8942)

Solutions

1. Verify a deterministic coder explicitly instead: new MapCoder<>(new OrderedCoder(keyCoder), new OrderedCoder(valueCoder)).verifyDeterministic() on an ordered map type
2. Use a deterministic key type (e.g. a custom POJO with fixed field order, or a String) instead of a Map
3. Use KV<K,V> with atomic key/value coders rather than a Map, ensuring keyCoder/valueCoder are themselves deterministic

Example fix

// before
.apply(GroupByKey.create()) // MapCoder<...> key -> NonDeterministicException
// after
MapCoder<MyKey, MyVal> c = MapCoder.of(OrderedCoder.of(keyCoder), valueCoder);
c.verifyDeterministic(); // only if key/value coders deterministic and map iteration ordered

try { mapCoder.verifyDeterministic(); } catch (NonDeterministicException e) { /* choose ordered/deterministic coder */ }

null

try { coder.verifyDeterministic(); } catch (NonDeterministicException e) { log.warn("using deterministic fallback coder", e); }

When it happens

Trigger: Using a Map (via MapCoder) as a PCollection element or key in an operation requiring determinism, e.g. GroupByKey, Combine, or Coder.verifyDeterministic() checks during pipeline validation.

Common situations: Grouping records keyed by a HashMap; tests calling CoderRegistry verifyDeterministic; pipeline construction fails validation at submit time.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• NonDeterministicException(target, message, e)
• Floating point encodings are not guaranteed to be determinis
• Java Serialization may be non-deterministic.
• Ordering of elements in a set may be non-deterministic.
• Ordering of entries in a Map may be non-deterministic.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1c45adc845befce9. Report an issue: GitHub.