apache/beam · error · CoderException

cannot encode a null Integer

Error message

cannot encode a null Integer

What it means

VarIntCoder encodes Java Integers as variable-length ints in Beam pipelines. It explicitly rejects null values because the varint wire format has no representation for null; nulls must be handled by wrapping coders (e.g. NullableCoder) before encoding. This fail-fast check prevents corrupted or ambiguous encoded bytes.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/VarIntCoder.java:49

 * are known to often be large or negative.
 */
public class VarIntCoder extends AtomicCoder<Integer> {

  public static VarIntCoder of() {
    return INSTANCE;
  }

  /////////////////////////////////////////////////////////////////////////////

  private static final VarIntCoder INSTANCE = new VarIntCoder();
  private static final TypeDescriptor<Integer> TYPE_DESCRIPTOR = new TypeDescriptor<Integer>() {};

  private VarIntCoder() {}

  @Override
  public void encode(Integer value, OutputStream outStream) throws IOException, CoderException {
    if (value == null) {
      throw new CoderException("cannot encode a null Integer");
    }
    VarInt.encode(value, outStream);
  }

  @Override
  public Integer decode(InputStream inStream) throws IOException, CoderException {
    try {
      return VarInt.decodeInt(inStream);
    } catch (EOFException | UTFDataFormatException exn) {
      // These exceptions correspond to decoding problems, so change
      // what kind of exception they're branded as.
      throw new CoderException(exn);
    }
  }

  @Override
  public void verifyDeterministic() {}

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the Integer value is non-null before encoding (substitute a default such as 0 or skip the element).
2. Wrap the coder with NullableCoder.of(VarIntCoder.of()) so nulls are supported.
3. Filter out or guard null elements upstream in the pipeline before the coder serializes them.

Example fix

// before
coder.encode(value, outStream); // NPE-ish CoderException when value == null
// after
if (value != null) {
  coder.encode(value, outStream);
} else {
  Coder<Integer> nullable = NullableCoder.of(VarIntCoder.of());
  nullable.encode(value, outStream);
}

if (value == null) { value = 0; /* or filter/skip */ }
coder.encode(value, outStream);

boolean isEncodable(Integer v) { return v != null; }

try { coder.encode(v, out); } catch (CoderException e) { log.error("null Integer at encode", e); }

When it happens

Trigger: Calling VarIntCoder.of().encode(null, outputStream) directly, or a pipeline element whose Integer value is null reaches this coder during encoding (e.g. via a MapElements output or PCollection of Integer containing null without a nullable wrapper).

Common situations: Pipeline data contains null Integer values (missing fields in parsed records, absent metrics) flowing through a non-nullable coder; DoFn emits null outputs; user code serializes nulls with raw coders in tests.

Related errors

• cannot encode a null Long
• cannot encode a null String
• cannot encode a null Integer
• cannot encode a null ValueKind
• Could not decode the default value with the provided coder %

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/dc0e43070ceeae2f. Report an issue: GitHub.