apache/beam · error · CoderException

cannot encode a null Short

Error message

cannot encode a null Short

What it means

BigEndianShortCoder.encode refuses a null Short: AtomicCoder implementations encode a fixed 2-byte big-endian representation and have no null marker, so a null value passed to encode cannot be serialized. The null Short argument is the input at fault.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/BigEndianShortCoder.java:44

/** A {@link BigEndianShortCoder} encodes {@link Short Shorts} in 2 bytes, big-endian. */
public class BigEndianShortCoder extends AtomicCoder<Short> {

  public static BigEndianShortCoder of() {
    return INSTANCE;
  }

  /////////////////////////////////////////////////////////////////////////////

  private static final BigEndianShortCoder INSTANCE = new BigEndianShortCoder();
  private static final TypeDescriptor<Short> TYPE_DESCRIPTOR = new TypeDescriptor<Short>() {};

  private BigEndianShortCoder() {}

  @Override
  public void encode(Short value, OutputStream outStream) throws IOException {
    if (value == null) {
      throw new CoderException("cannot encode a null Short");
    }
    BitConverters.writeBigEndianShort(value, outStream);
  }

  @Override
  public Short decode(InputStream inStream) throws IOException, CoderException {
    try {
      return BitConverters.readBigEndianShort(inStream);
    } catch (EOFException | UTFDataFormatException exn) {
      // These exceptions correspond to decoding problems, so change
      // what kind of exception they're branded as.
      throw new CoderException(exn);
    }
  }

  @Override
  public void verifyDeterministic() {}

View on GitHub (pinned to 12126d8942)

Solutions

1. Filter nulls with Filter.by(Objects::nonNull) before the stage.
2. Use NullableCoder.of(BigEndianShortCoder.of()) via setCoder().
3. Coerce nulls to a default short value upstream.
4. Add an explicit null check in custom code before calling encode().

Example fix

// before
out.output(raw == null ? null : Short.parseShort(raw));
// after
Short s = raw == null ? null : Short.parseShort(raw);
if (s != null) { out.output(s); }

if (value == null) { throw new IllegalArgumentException("Short element is null; filter before encoding"); }

boolean isEncodable(Short v) { return v != null; }

try {
  coder.encode(value, out);
} catch (CoderException e) {
  LOG.error("Null Short element", e);
  throw e;
}

When it happens

Trigger: Calling BigEndianShortCoder.of().encode(null, out) directly, or a PCollection<Short> containing null elements reaching an encode step.

Common situations: Null Shorts from boxed conversions (e.g. Number.shortValue() on null), sparse data mapped to Short, hand-rolled DoFns emitting null on parse failure.

Related errors

• cannot encode a null Integer
• cannot encode a null Long
• cannot encode a null BitSet
• cannot encode a null byte[]
• cannot encode a null value

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/840595d86d0c4d20. Report an issue: GitHub.