apache/beam · error · NonDeterministicException

Floating point encodings are not guaranteed to be determinis

Error message

Floating point encodings are not guaranteed to be deterministic.

What it means

DoubleCoder.verifyDeterministic always throws NonDeterministicException because IEEE-754 floating-point encoding/behavior is not guaranteed deterministic across platforms (NaN payloads, signed zeros, hardware differences). Beam rejects such coders where determinism is required (e.g. keys for GroupByKey, state, ordering).

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/DoubleCoder.java:70

    try {
      return Double.longBitsToDouble(BitConverters.readBigEndianLong(inStream));
    } catch (EOFException | UTFDataFormatException exn) {
      // These exceptions correspond to decoding problems, so change
      // what kind of exception they're branded as.
      throw new CoderException(exn);
    }
  }

  /**
   * {@inheritDoc}
   *
   * @throws NonDeterministicException always. Floating-point operations are not guaranteed to be
   *     deterministic, even if the storage format might be, so floating point representations are
   *     not recommended for use in operations that require deterministic inputs.
   */
  @Override
  public void verifyDeterministic() throws NonDeterministicException {
    throw new NonDeterministicException(
        this, "Floating point encodings are not guaranteed to be deterministic.");
  }

  /**
   * {@inheritDoc}
   *
   * @return {@code true}. This coder is injective.
   */
  @Override
  public boolean consistentWithEquals() {
    return true;
  }

  /**
   * {@inheritDoc}
   *
   * @return {@code true}. {@link DoubleCoder#getEncodedElementByteSize} returns a constant.
   */

View on GitHub (pinned to 12126d8942)

Solutions

Convert doubles to a deterministic representation before grouping, e.g. quantize to long (Math.round(x * 1e6)) or fixed-point BigDecimal string.
Use a wrapper coder that defines a deterministic encoding and override verifyDeterministic only if you can prove determinism.
Restructure the pipeline so doubles are values, not keys.
If grouping on floats is truly required, normalize values (canonicalize NaN/-0.0) and use a custom coder.

Example fix

// before
KV<Double, String> kv = KV.of(score, id);
... apply(GroupByKey.create()) // NonDeterministicException

// after
KV<Long, String> kv = KV.of(Math.round(score * 1_000_000L), id); // quantized key

Defensive patterns

Strategy: validation

Validate before calling

try {
  Coder<?> keyCoder = ...;
  keyCoder.verifyDeterministic();
} catch (NonDeterministicException e) {
  throw new IllegalStateException("Key coder must be deterministic: " + e.getMessage());
}

Type guard

boolean deterministicKey(Class<?> keyClass) { return !Double.class.equals(keyClass) && !Float.class.equals(keyClass); }

Try / catch

try {
  coder.verifyDeterministic();
} catch (NonDeterministicException e) {
  // switch to quantized long key or deterministic custom coder
}

Prevention

Never use raw double/float as GroupByKey keys; quantize to long or string.
Canonicalize NaN and -0.0 if floats must be keys.
Document determinism constraints in custom coder implementations.

When it happens

Trigger: Using Double (or a type whose resolved coder is DoubleCoder) as a key in GroupByKey/Combine PerKey, as state, or anywhere Beam calls coder.verifyDeterministic().

Common situations: Grouping computed double values; using double timestamps/scores as keys; applying Combine.perKey on KV<Double, V>.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/997e624187531350. Report an issue: GitHub.