apache/beam · warning · NonDeterministicException

TableRows are not deterministic.

Error message

TableRows are not deterministic.

What it means

TableRowInfoCoder.verifyDeterministic always throws NonDeterministicException because TableRow JSON encodings are not byte-stable (field ordering / arbitrary object values). Beam requires deterministic coders for keyed state, grouping, and exact-dedup semantics, so this coder cannot be used where determinism is required.

Solutions

1. Do not use TableRowInfoCoder (or TableRow as a key) where determinism is required; key on a stable primitive like the table spec or unique id string instead.
2. Override the coder check with coder.verifyDeterministic() bypass only if you accept non-deterministic grouping semantics (risky, last resort).
3. Restructure the pipeline so TableRows are values, not keys.

Example fix

// before
KV<TableRow, Row> keyed = KV.of(tableRow, row); // nondeterministic key coder
// after
KV<String, Row> keyed = KV.of(tableRow.get("id").toString(), row);

// Decide at graph construction time
try {
  coder.verifyDeterministic();
} catch (NonDeterministicException e) {
  LOG.warn("Coder non-deterministic, use stable key: {}", e.getMessage());
}

When it happens

Trigger: Using TableRowInfoCoder in an operation requiring deterministic encoding: GroupByKey on coders marked deterministic, stateful DoFns, or calls to verifyDeterministic() during pipeline construction/validation.

Common situations: Pipelines that group or deduplicate by a TableRow-containing key in BigQuery IO paths; users attempting to reuse the coder as a KV key coder.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• TableCell can hold arbitrary instances, which may be…
• cannot encode a null value
• cannot encode a null value
• BigQuery data contained value
• cannot encode a null Count-min Sketch

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/1a57224d92c3ba9e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowInfoCoder.java:69

      throw new CoderException("cannot encode a null value");
    }
    elementCoder.encode(value.tableRow, outStream);
    idCoder.encode(value.uniqueId, outStream, context);
  }

  @Override
  public TableRowInfo<ElementT> decode(InputStream inStream) throws IOException {
    return decode(inStream, Context.NESTED);
  }

  @Override
  public TableRowInfo<ElementT> decode(InputStream inStream, Context context) throws IOException {
    return new TableRowInfo<>(elementCoder.decode(inStream), idCoder.decode(inStream, context));
  }

  @Override
  public void verifyDeterministic() throws NonDeterministicException {
    throw new NonDeterministicException(this, "TableRows are not deterministic.");
  }

  StringUtf8Coder idCoder = StringUtf8Coder.of();
}

View on GitHub (pinned to 12126d8942)