apache/beam · error · CoderException

cannot encode a null byte[]

Error message

cannot encode a null byte[]

What it means

ByteArrayCoder.encode received a null byte[] and threw CoderException: this coder writes a length-prefixed byte sequence with no null sentinel, so null cannot be encoded. Wrap with NullableCoder.of(ByteArrayCoder.of()) if the pipeline genuinely produces null byte arrays.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/ByteArrayCoder.java:63

  }

  /////////////////////////////////////////////////////////////////////////////

  private static final ByteArrayCoder INSTANCE = new ByteArrayCoder();
  private static final TypeDescriptor<byte[]> TYPE_DESCRIPTOR = new TypeDescriptor<byte[]>() {};

  private ByteArrayCoder() {}

  @Override
  public void encode(byte[] value, OutputStream outStream) throws IOException, CoderException {
    encode(value, outStream, Context.NESTED);
  }

  @Override
  public void encode(byte[] value, OutputStream outStream, Context context)
      throws IOException, CoderException {
    if (value == null) {
      throw new CoderException("cannot encode a null byte[]");
    }
    if (!context.isWholeStream) {
      VarInt.encode(value.length, outStream);
      outStream.write(value);
    } else {
      outStream.write(value);
    }
  }

  /**
   * Encodes the provided {@code value} with the identical encoding to {@link #encode}, but with
   * optimizations that take ownership of the value.
   *
   * <p>Once passed to this method, {@code value} should never be observed or mutated again.
   */
  public void encodeAndOwn(byte[] value, OutputStream outStream, Context context)
      throws IOException, CoderException {
    if (!context.isWholeStream) {

View on GitHub (pinned to 12126d8942)

Solutions

1. Filter null byte arrays with Filter.by(Objects::nonNull).
2. Use NullableCoder.of(ByteArrayCoder.of()) via setCoder() on the PCollection.
3. Substitute an empty array (new byte[0]) for null where acceptable.
4. Null-check before calling encode() in custom serialization code.

Example fix

// before
out.output(payload == null ? null : payload.toByteArray());
// after
out.output(payload == null ? new byte[0] : payload.toByteArray());

if (value == null) { throw new IllegalArgumentException("byte[] element is null; filter before encoding"); }

boolean isEncodable(byte[] v) { return v != null; }

try {
  coder.encode(value, out, Context.OUTER);
} catch (CoderException e) {
  LOG.error("Null byte[] element", e);
  throw e;
}

When it happens

Trigger: Encoding a null byte[] directly, or a PCollection<byte[]> containing null elements reaching serialization.

Common situations: Null byte arrays from I/O helpers (file/read returning null on failure), protobuf toByteArray() on a null message, manual coder use in tests or custom sinks.

Related errors

• cannot encode a null Integer
• cannot encode a null Long
• cannot encode a null Short
• cannot encode a null BitSet
• cannot encode a null value

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4a8f23ca5ff946f6. Report an issue: GitHub.