apache/beam · error · CoderException

cannot encode a null Integer

Error message

cannot encode a null Integer

What it means

BigEndianIntegerCoder.encode() requires a non-null Integer; Apache Beam coders do not encode null values by default, so encoding null throws CoderException with this message. Nulls must be handled by wrapping the coder (NullableCoder/lengthPrefix) or filtered before encoding.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/BigEndianIntegerCoder.java:44

/** A {@link BigEndianIntegerCoder} encodes {@link Integer Integers} in 4 bytes, big-endian. */
public class BigEndianIntegerCoder extends AtomicCoder<Integer> {

  public static BigEndianIntegerCoder of() {
    return INSTANCE;
  }

  /////////////////////////////////////////////////////////////////////////////

  private static final BigEndianIntegerCoder INSTANCE = new BigEndianIntegerCoder();
  private static final TypeDescriptor<Integer> TYPE_DESCRIPTOR = new TypeDescriptor<Integer>() {};

  private BigEndianIntegerCoder() {}

  @Override
  public void encode(Integer value, OutputStream outStream) throws IOException {
    if (value == null) {
      throw new CoderException("cannot encode a null Integer");
    }
    BitConverters.writeBigEndianInt(value, outStream);
  }

  @Override
  public Integer decode(InputStream inStream) throws IOException, CoderException {
    try {
      return BitConverters.readBigEndianInt(inStream);
    } catch (EOFException | UTFDataFormatException exn) {
      // These exceptions correspond to decoding problems, so change
      // what kind of exception they're branded as.
      throw new CoderException(exn);
    }
  }

  @Override
  public void verifyDeterministic() {}

View on GitHub (pinned to 12126d8942)

Solutions

1. Filter nulls before the pipeline stage: p.apply(Filter.by(Objects::nonNull)).
2. Replace null with a sentinel/default value appropriate to your domain.
3. Wrap the element type: PCollection<Integer> with coder NullableCoder.of(BigEndianIntegerCoder.of()) via setCoder().
4. If calling encode() manually, assert value != null first and throw a domain-specific error.

Example fix

// before
PCollection<Integer> nums = p.apply("Parse", ParDo.of(new ParseIntFn()));
// after (nulls filtered)
PCollection<Integer> nums = p.apply("Parse", ParDo.of(new ParseIntFn()))
    .apply(Filter.by(Objects::nonNull));

if (value == null) { throw new IllegalArgumentException("Integer element is null; filter before encoding"); }

boolean isEncodable(Integer v) { return v != null; }

try {
  coder.encode(value, out);
} catch (CoderException e) {
  LOG.error("Null Integer in pipeline element", e);
  throw e;
}

When it happens

Trigger: Passing a null Integer to encode() directly, or a pipeline stage (PCollection<Integer>) whose coder is BigEndianIntegerCoder while the collection can contain null elements.

Common situations: Emitting null from a DoFn/map into a PCollection<Integer> without converting to NullableCoder; deserialized data with missing values; Java auto-boxing of an uninitialized int wrapper.

Related errors

• cannot encode a null Long
• cannot encode a null Short
• cannot encode a null BitSet
• cannot encode a null byte[]
• cannot encode a null value

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/60881642439aa6f1. Report an issue: GitHub.