apache/beam · warning · NonDeterministicException

TableCell can hold arbitrary instances, which may be…

Error message

TableCell can hold arbitrary instances, which may be non-deterministic.

What it means

TableRowJsonCoder.verifyDeterministic always throws NonDeterministicException: a TableRow can hold arbitrary Object instances (nested cells, Jackson JsonNode values) whose encodings are not guaranteed byte-identical across encodes of equal rows. Beam therefore forbids its use wherever deterministic encoding (grouping keys, state) is required.

Solutions

1. Use a deterministic key type (String, Long, custom Avro/protobuf coder) instead of TableRow as a grouping key.
2. Extract a deterministic subset of fields into a custom POJO with a deterministic coder.
3. Avoid relying on byte-level equality of TableRow encodings; compare via explicit fields.

Example fix

// before
.apply(GroupByKey.<TableRow, Iterable<Row>>create());
// after
.apply(MapElements.via(...)) // project to String key first
.apply(GroupByKey.<String, Iterable<Row>>create());

try {
  TableRowJsonCoder.of().verifyDeterministic();
} catch (NonDeterministicException e) {
  // use a deterministic custom coder / String key instead
}

When it happens

Trigger: Registering TableRowJsonCoder as a key coder for GroupByKey/Combine/stateful transforms, or any pipeline-validation call to verifyDeterministic() on this coder.

Common situations: Users grouping by TableRow keys in BigQuery-related pipelines; streaming pipelines requiring deterministic serialization for dedup or Exactly-Once semantics.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• TableRows are not deterministic.
• cannot encode a null value
• cannot encode a null value
• BigQuery data contained value
• cannot encode a null Count-min Sketch

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/90ce8629a0f7793d. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowJsonCoder.java:99

            .registerModule(new JavaTimeModule())
            .registerModule(new JodaModule())
            .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
            .disable(SerializationFeature.FAIL_ON_EMPTY_BEANS);
    INSTANCE = new TableRowJsonCoder();
    TYPE_DESCRIPTOR = new TypeDescriptor<TableRow>() {};
  }

  private TableRowJsonCoder() {}

  /**
   * {@inheritDoc}
   *
   * @throws NonDeterministicException always. A {@link TableRow} can hold arbitrary {@link Object}
   *     instances, which makes the encoding non-deterministic.
   */
  @Override
  public void verifyDeterministic() throws NonDeterministicException {
    throw new NonDeterministicException(
        this, "TableCell can hold arbitrary instances, which may be non-deterministic.");
  }

  @Override
  public TypeDescriptor<TableRow> getEncodedTypeDescriptor() {
    return TYPE_DESCRIPTOR;
  }
}

View on GitHub (pinned to 12126d8942)