apache/beam · error · CoderException

Could not write value. Error

Error message

Could not write value. Error: %s

What it means

ThriftCoder.encode serializes a TBase value to the output stream via a Thrift protocol. If value.write(protocol) throws any exception other than TTransportException, it is wrapped in a CoderException with this message. It typically means the Thrift object itself failed to serialize (bad field state, union with unset field, etc.), not a transport problem.

Solutions

1. Inspect the wrapped exception message (the cause text after 'Error:') to identify the failing field or protocol issue
2. Ensure the thrift class on the classpath was generated from the same .thrift IDL used to produce the data
3. Validate the TBase value (required fields set, unions have exactly one field) before encoding
4. If it is a transport-level problem, it would be reported as 'Could not transport value' instead — check the underlying OutputStream for closure/corruption

Example fix

// before
Coder<ThriftRecord> coder = ThriftCoder.of();
out.write(coder.encode(record));
// after
if (record == null || !record.isSetRequiredField()) {
  throw new IllegalArgumentException("ThriftRecord missing required field before encode");
}
out.write(coder.encode(record));

if (value == null || !value.isSetRequiredField()) { throw new IllegalArgumentException("value not ready for thrift encoding"); }

try { bytes = coder.encode(record); } catch (CoderException e) { LOG.error("thrift encode failed: {}", e.getMessage(), e); throw new UntranslatableException("encode", e); }

When it happens

Trigger: Calling encode on a ThriftCoder with a TBase whose write(TProtocol) throws, e.g. TProtocolException from deeply nested or oversized data, TException subclasses thrown by custom thrift structs, or null/unset required struct fields.

Common situations: Pipelines serializing Thrift records to Kafka/BigQuery; a thrift class compiled against a mismatched .thrift IDL version fails to write; union fields left unset; stream closed mid-write surfacing as a generic exception.

Understand the failure class

Background: "JSON serialization failed", "not JSON serializable", "Failed to serialize": why JSON marshaling errors happen and how to fix them — this error's family across 46 libraries.

Related errors

• Could not read value. Error
• Could not transport value. Error
• Azure credential provider type
• Cannot decode object from input stream.
• cannot encode a null BitSet

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ab7bdd93e9952a22. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/thrift/src/main/java/org/apache/beam/sdk/io/thrift/ThriftCoder.java:81

  /**
   * Encodes the given value of type {@code T} onto the given output stream using provided {@link
   * ThriftCoder#protocolFactory}.
   *
   * @param value {@link org.apache.thrift.TBase} to encode.
   * @param outStream stream to output encoded value to.
   * @throws IOException if writing to the {@code OutputStream} fails for some reason
   */
  @Override
  public void encode(T value, OutputStream outStream) throws CoderException, IOException {
    try {
      TProtocol protocol = protocolFactory.getProtocol(new TIOStreamTransport(outStream));
      TBase<?, ?> tBase = (TBase<?, ?>) value;
      tBase.write(protocol);
    } catch (TTransportException tte) {
      throw new CoderException("Could not transport value. Error: " + tte.getMessage());
    } catch (Exception te) {
      throw new CoderException("Could not write value. Error: " + te.getMessage());
    }
  }

  /**
   * Decodes a value of type {@code T} from the given input stream using provided {@link
   * ThriftCoder#protocolFactory}. Returns the decoded value.
   *
   * @param inStream stream of input values to be decoded
   * @throws IOException if reading from the {@code InputStream} fails for some reason
   * @throws CoderException if the value could not be decoded for some reason
   * @return {@link TBase} decoded object
   */
  @Override
  public T decode(InputStream inStream) throws CoderException, IOException {
    try {
      TProtocol protocol = protocolFactory.getProtocol(new TIOStreamTransport(inStream));
      TBase<?, ?> value = (TBase<?, ?>) type.getDeclaredConstructor().newInstance();
      value.read(protocol);

View on GitHub (pinned to 12126d8942)