apache/beam · error · NonDeterministicException

Floating point encodings are not guaranteed to be determinis

Error message

Floating point encodings are not guaranteed to be deterministic.

What it means

FloatCoder.verifyDeterministic() unconditionally throws NonDeterministicException because floating-point encodings cannot be relied on to be byte-for-byte deterministic (storage format aside, FP semantics like NaN payloads and signed zeros make representations unsuitable for operations needing deterministic inputs). Beam requires deterministic coders for grouping, state, and cross-run keying.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/FloatCoder.java:70

    try {
      return Float.intBitsToFloat(BitConverters.readBigEndianInt(inStream));
    } catch (EOFException | UTFDataFormatException exn) {
      // These exceptions correspond to decoding problems, so change
      // what kind of exception they're branded as.
      throw new CoderException(exn);
    }
  }

  /**
   * {@inheritDoc}
   *
   * @throws NonDeterministicException always. Floating-point operations are not guaranteed to be
   *     deterministic, even if the storage format might be, so floating point representations are
   *     not recommended for use in operations that require deterministic inputs.
   */
  @Override
  public void verifyDeterministic() throws NonDeterministicException {
    throw new NonDeterministicException(
        this, "Floating point encodings are not guaranteed to be deterministic.");
  }

  /**
   * {@inheritDoc}
   *
   * @return {@code true}. This coder is injective.
   */
  @Override
  public boolean consistentWithEquals() {
    return true;
  }

  /**
   * {@inheritDoc}
   *
   * @return {@code true}. {@link FloatCoder#getEncodedElementByteSize} returns a constant.
   */

View on GitHub (pinned to 12126d8942)

Solutions

Use an integral or string representation instead: encode the float as a FixedBytesCoder of its IEEE-754 bits, or key on a normalized string.
Wrap the float in a custom Coder whose verifyDeterministic() succeeds and whose encode() writes a canonical byte representation (e.g. normalize -0.0 and NaN first).
Restructure the pipeline so float values are not used as grouping keys (key on an id, value on the float).

Example fix

// before
pipeline.apply(GroupByKey.create()) // PCollection<KV<Float, ...>> -> throws
// after
PCollection<KV<String, ...>> grouped =
    pc.apply(MapElements.into(kvOf(strings(), floats())).via(f -> KV.of(String.valueOf(f), f)))
      .apply(GroupByKey.create());

Defensive patterns

Strategy: fallback

Validate before calling

try { FloatCoder.of().verifyDeterministic(); } catch (Coder.NonDeterministicException e) { /* switch to deterministic coder */ }

Prevention

Never use raw Float/Double coders for grouping keys or stateful operations.
Prefer keying on ids/strings and carrying floats as values.
Run coder.verifyDeterministic() in pipeline unit tests before shuffle stages.

When it happens

Trigger: Using FloatCoder (or DoubleCoder) as the element coder for a PCollection/key coder in any context that calls verifyDeterministic, e.g. GroupByKey, CoGroupByKey, Combine perKey, stateful DoFns, or explicitly calling coder.verifyDeterministic().

Common situations: Grouping a PCollection of floats/doubles keyed or valued with FloatCoder; building a KV coder whose key coder is FloatCoder; using FloatCoder in a SetCoder/MapCoder requiring deterministic element coders.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8b4372fe9213fa8f. Report an issue: GitHub.