apache/beam · error · EncodingException

EncodingException

Error message

EncodingException

What it means

SizeEstimator.sizeOf measures an element's encoded size via CodedSizeObserver; any exception raised during coder.registerByteSizeObserver or observer.advance() is rethrown as an org.apache.beam.sdk.coders.CoderException (EncodingException in this context). It means the element cannot be sized/encoded by its declared Coder — usually the element does not match the coder, or encoding fails mid-way.

Solutions

1. Check the wrapped cause 'e' — it names the coder and element that failed; align the element type with the coder.
2. Ensure the registered Coder (e.g. for ChangeStreamRecord/PartitionMetadata) matches the actual runtime element, including generics.
3. Fix the custom coder's encode/registerByteSizeObserver implementation if it throws on valid input (e.g. null handling).
4. Verify all fields of the element are encodable (transient/non-serializable fields removed or made encodable).

Example fix

// before
Coder<ChangeStreamRecord> coder = new MyCoder(); // handles only DataChangeRecord
// after
Coder<ChangeStreamRecord> coder = ChangeStreamRecordsCoder.of(); // covers all record subtypes

try { long size = estimator.sizeOf(element); } catch (CoderException e) {
  log.error("Cannot encode element of type {}", element.getClass(), e);
}

When it happens

Trigger: Estimating the size of a change stream record/partition whose runtime class does not match the registered Coder; a custom coder throwing while encoding (non-serializable field, null nested value); observer.advance() failing on malformed data.

Common situations: Pipelines where the element type changed but the coder wasn't updated; custom elements containing non-encodable fields used with Spanner change stream size estimation; serialization bugs in user-defined types flowing through the estimator.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• cannot encode a null Count-min Sketch
• cannot encode a null Integer
• cannot encode a null String
• cannot encode a null T-Digest sketch
• cannot encode a null ValueKind

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ab9bfcb36e0f8f1e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/estimator/SizeEstimator.java:65

  public SizeEstimator(Coder<T> coder) {
    this.coder = coder;
    this.observer = new SizeEstimatorObserver();
  }

  /**
   * Estimates the size in bytes of the given element with the configured {@link Coder} .
   *
   * @param element the element instance to be estimated
   * @return the estimated size in bytes of the given element
   */
  public long sizeOf(T element) {
    try {
      coder.registerByteSizeObserver(element, observer);
      observer.advance();

      return observer.observedBytes;
    } catch (Exception e) {
      throw new EncodingException(e);
    }
  }
}

View on GitHub (pinned to 12126d8942)