apache/beam · error · NonDeterministicException

CustomCoder implementations must override verifyDeterministi

Error message

CustomCoder implementations must override verifyDeterministic, or they are presumed nondeterministic.

What it means

CustomCoder.verifyDeterministic is intentionally left throwing: any coder that does not override it is presumed nondeterministic, because determinism cannot be verified automatically. Beam throws NonDeterministicException whenever such a coder's verifyDeterministic() is called (e.g. during GBK / group-by-key planning on keyed PCollections).

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/CustomCoder.java:49

  /**
   * {@inheritDoc}.
   *
   * <p>Returns an empty list. A {@link CustomCoder} has no default argument {@link Coder coders}.
   */
  @Override
  public List<? extends Coder<?>> getCoderArguments() {
    return Collections.emptyList();
  }

  /**
   * {@inheritDoc}
   *
   * @throws NonDeterministicException a {@link CustomCoder} is presumed nondeterministic.
   */
  @Override
  public void verifyDeterministic() throws NonDeterministicException {
    throw new NonDeterministicException(
        this,
        "CustomCoder implementations must override verifyDeterministic,"
            + " or they are presumed nondeterministic.");
  }

  // This coder inherits isRegisterByteSizeObserverCheap,
  // getEncodedElementByteSize and registerByteSizeObserver
  // from Coder. Override if we can do better.
}

View on GitHub (pinned to 12126d8942)

Solutions

Override verifyDeterministic() as a no-op (or with component checks) if encoding is truly deterministic.
If genuinely nondeterministic, avoid operations requiring determinism (GBK) or switch to a deterministic key encoding.
For composite coders, delegate to component coders' verifyDeterministic inside your override.

Example fix

// before
class MyCoder extends CustomCoder<MyT> { /* no verifyDeterministic */ }
// after
class MyCoder extends CustomCoder<MyT> {
  @Override
  public void verifyDeterministic() throws NonDeterministicException {} // encoding is stable
}

Defensive patterns

Strategy: try-catch

Validate before calling

try {
  myCoder.verifyDeterministic();
} catch (NonDeterministicException e) {
  // coder needs an override or must not be used as a key coder
}

Try / catch

try {
  keyed.apply(GroupByKey.create());
} catch (NonDeterministicException e) {
  // override verifyDeterministic in your CustomCoder or pick a deterministic key coder
}

Prevention

Always override verifyDeterministic when subclassing CustomCoder.
Call verifyDeterministic() in coder unit tests.
Only use deterministic coders for key types in GBK/Combine operations.

When it happens

Trigger: Subclassing CustomCoder without overriding verifyDeterministic and using the coder where Beam checks determinism — most commonly a keyed PCollection entering GroupByKey, Combine, or Join.

Common situations: Writing a first custom coder and forgetting the override; a coder that is actually deterministic but whose author never declared it; upgrading Beam so a determinism check newly runs on an existing coder.

Understand the failure class

Background: "NotImplementedError: Subclasses should override this method" / "must be implemented" — abstract method errors explained — this error's family across 40 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/eb6eb3334751d4d3. Report an issue: GitHub.