apache/beam · error · CoderException

cannot encode a null Double

Error message

cannot encode a null Double

What it means

DoubleCoder.encode rejects null values by throwing CoderException, because a primitive double encoding has no null representation in Beam's fixed 8-byte format. Beam coders generally require nullable values to be wrapped in a NullableCoder.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/DoubleCoder.java:45

/** A {@link DoubleCoder} encodes {@link Double} values in 8 bytes using Java serialization. */
public class DoubleCoder extends AtomicCoder<Double> {

  public static DoubleCoder of() {
    return INSTANCE;
  }

  /////////////////////////////////////////////////////////////////////////////

  private static final DoubleCoder INSTANCE = new DoubleCoder();
  private static final TypeDescriptor<Double> TYPE_DESCRIPTOR = new TypeDescriptor<Double>() {};

  private DoubleCoder() {}

  @Override
  public void encode(Double value, OutputStream outStream) throws IOException, CoderException {
    if (value == null) {
      throw new CoderException("cannot encode a null Double");
    }
    new DataOutputStream(outStream).writeDouble(value);
  }

  @Override
  public Double decode(InputStream inStream) throws IOException, CoderException {
    try {
      return Double.longBitsToDouble(BitConverters.readBigEndianLong(inStream));
    } catch (EOFException | UTFDataFormatException exn) {
      // These exceptions correspond to decoding problems, so change
      // what kind of exception they're branded as.
      throw new CoderException(exn);
    }
  }

  /**
   * {@inheritDoc}
   *

View on GitHub (pinned to 12126d8942)

Solutions

1. Replace nulls before encoding: use a DoFn/Map to substitute a default (e.g. 0.0 or Double.NaN) or filter them out.
2. Wrap the coder with NullableCoder.of(DoubleCoder.of()) so nulls are encodable.
3. Change the element type to a nullable-aware representation (e.g. Optional<Double> with an appropriate coder).
4. Trace the source of the null and fix the producing transform to never emit null.

Example fix

// before
.apply(MapElements.into(TypeDescriptors.doubles()).via(x -> x.length() > 0 ? parse(x) : null))

// after
.apply(Filter.by(Objects::nonNull))
.apply(MapElements.into(TypeDescriptors.doubles()).via(x -> parse(x)))

if (value == null) { value = 0.0; /* or filter */ }

boolean isEncodable(Double d) { return d != null; }

try {
  pipeline.run().waitUntilFinish();
} catch (CoderException | RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("null Double")) {
    // add NullableCoder.of(DoubleCoder.of()) or null-filtering and rerun
  }
  throw e;
}

When it happens

Trigger: Encoding a PCollection<Double> or KV containing a null Double through DoubleCoder, e.g. when a map/DoFn emits null and the coder resolved for the collection is DoubleCoder.

Common situations: DoFns computing statistics that emit null for empty input; parsing rows where a field is missing; passing java.util.Optional misuse with null values.

Related errors

• cannot encode a null Byte
• cannot estimate size for unsupported null value
• cannot encode a null ReadableDuration
• cannot encode a null Float
• cannot encode a null Map

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/77fa2d5a76503e80. Report an issue: GitHub.