apache/beam · error · CoderException

cannot encode a null byte[]

Error message

cannot encode a null byte[]

What it means

ByteArrayCoder.encode received a null byte[] and threw CoderException: this coder writes a length-prefixed byte sequence with no null sentinel, so null cannot be encoded. Wrap with NullableCoder.of(ByteArrayCoder.of()) if the pipeline genuinely produces null byte arrays.

Solutions

1. Filter null byte arrays with Filter.by(Objects::nonNull).
2. Use NullableCoder.of(ByteArrayCoder.of()) via setCoder() on the PCollection.
3. Substitute an empty array (new byte[0]) for null where acceptable.
4. Null-check before calling encode() in custom serialization code.

Example fix

// before
out.output(payload == null ? null : payload.toByteArray());
// after
out.output(payload == null ? new byte[0] : payload.toByteArray());

if (value == null) { throw new IllegalArgumentException("byte[] element is null; filter before encoding"); }

boolean isEncodable(byte[] v) { return v != null; }

try {
  coder.encode(value, out, Context.OUTER);
} catch (CoderException e) {
  LOG.error("Null byte[] element", e);
  throw e;
}

When it happens

Trigger: Encoding a null byte[] directly, or a PCollection<byte[]> containing null elements reaching serialization.

Common situations: Null byte arrays from I/O helpers (file/read returning null on failure), protobuf toByteArray() on a null message, manual coder use in tests or custom sinks.

Related errors

• cannot encode a null BitSet
• cannot encode a null Integer
• cannot encode a null Long
• cannot encode a null Short
• cannot encode a null Count-min Sketch

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4a8f23ca5ff946f6. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/ByteArrayCoder.java:63

  }

  /////////////////////////////////////////////////////////////////////////////

  private static final ByteArrayCoder INSTANCE = new ByteArrayCoder();
  private static final TypeDescriptor<byte[]> TYPE_DESCRIPTOR = new TypeDescriptor<byte[]>() {};

  private ByteArrayCoder() {}

  @Override
  public void encode(byte[] value, OutputStream outStream) throws IOException, CoderException {
    encode(value, outStream, Context.NESTED);
  }

  @Override
  public void encode(byte[] value, OutputStream outStream, Context context)
      throws IOException, CoderException {
    if (value == null) {
      throw new CoderException("cannot encode a null byte[]");
    }
    if (!context.isWholeStream) {
      VarInt.encode(value.length, outStream);
      outStream.write(value);
    } else {
      outStream.write(value);
    }
  }

  /**
   * Encodes the provided {@code value} with the identical encoding to {@link #encode}, but with
   * optimizations that take ownership of the value.
   *
   * <p>Once passed to this method, {@code value} should never be observed or mutated again.
   */
  public void encodeAndOwn(byte[] value, OutputStream outStream, Context context)
      throws IOException, CoderException {
    if (!context.isWholeStream) {

View on GitHub (pinned to 12126d8942)