apache/beam · error · org.apache.beam.sdk.coders.CoderException

cannot encode a null

Error message

cannot encode a null ${messageClassName}

What it means

ProtoCoder does not accept null values during encoding; a null proto message cannot be serialized to the output stream. This CoderException signals a contract violation by the pipeline element that tried to encode null.

Solutions

1. Filter nulls before the coder is used: pcollection.apply(Filter.by(m -> m != null)).
2. Replace nulls with a default instance (e.g. Message.getDefaultInstance()).
3. Fix the upstream transform to never emit null.
4. Use a nullable representation (Optional/PCollectionList) instead of encoding nulls.

Example fix

// before
return maybeMessage; // may be null
// after
return maybeMessage != null ? maybeMessage : MyProto.getDefaultInstance();

if (msg == null) msg = MyProto.getDefaultInstance(); // before handing to coder

try { coder.encode(msg, out, Context.OUTER); } catch (CoderException e) { log.error("null or malformed message", e); }

When it happens

Trigger: A PCollection containing null elements is encoded with ProtoCoder, e.g. when a DoFn/ParDo emits null or a downstream sink encodes null messages, and encode(T, OutputStream, Context) is invoked with value == null.

Common situations: Map/filter chains producing null outputs; aggregations over missing keys yielding null; accidental null returns in user code feeding a coded sink or shuffle.

Related errors

• Aliased enumerations not currently supported.
• Any not yet supported
• Cannot create creator for
• cannot encode a null Integer
• cannot encode a null String

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/aa7203da0d2b096d. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/protobuf/src/main/java/org/apache/beam/sdk/extensions/protobuf/ProtoCoder.java:188

  /**
   * See {@link #withExtensionsFrom(Iterable)}.
   *
   * <p>Does not modify this object.
   */
  public ProtoCoder<T> withExtensionsFrom(Class<?>... moreExtensionHosts) {
    return withExtensionsFrom(Arrays.asList(moreExtensionHosts));
  }

  @Override
  public void encode(T value, OutputStream outStream) throws IOException {
    encode(value, outStream, Context.NESTED);
  }

  @Override
  public void encode(T value, OutputStream outStream, Context context) throws IOException {
    if (value == null) {
      throw new CoderException("cannot encode a null " + protoMessageClass.getSimpleName());
    }
    if (context.isWholeStream) {
      value.writeTo(outStream);
    } else {
      value.writeDelimitedTo(outStream);
    }
  }

  @Override
  public T decode(InputStream inStream) throws IOException {
    return decode(inStream, Context.NESTED);
  }

  @Override
  public T decode(InputStream inStream, Context context) throws IOException {
    if (context.isWholeStream) {
      return getParser().parseFrom(inStream, getExtensionRegistry());
    } else {

View on GitHub (pinned to 12126d8942)