apache/beam · error · CoderException

CoderException

Error message

CoderException

What it means

VarLongCoder.decode reads a varint-encoded long and rebrands EOFException and UTFDataFormatException — the signals of truncated or corrupt input — as CoderException. This means the byte stream ended mid-value or contained bytes that do not form a valid varint long.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/coders/VarLongCoder.java:62

  private VarLongCoder() {}

  @Override
  public void encode(Long value, OutputStream outStream) throws IOException, CoderException {
    if (value == null) {
      throw new CoderException("cannot encode a null Long");
    }
    VarInt.encode(value, outStream);
  }

  @Override
  public Long decode(InputStream inStream) throws IOException, CoderException {
    try {
      return VarInt.decodeLong(inStream);
    } catch (EOFException | UTFDataFormatException exn) {
      // These exceptions correspond to decoding problems, so change
      // what kind of exception they're branded as.
      throw new CoderException(exn);
    }
  }

  @Override
  public List<? extends Coder<?>> getCoderArguments() {
    return Collections.emptyList();
  }

  @Override
  public void verifyDeterministic() {}

  /**
   * {@inheritDoc}
   *
   * @return {@code true}. {@link VarLongCoder} is injective.
   */
  @Override
  public boolean consistentWithEquals() {

View on GitHub (pinned to 12126d8942)

Solutions

1. Verify the byte stream is complete and correctly framed; check that all written bytes are flushed before decoding.
2. Ensure the same coder (and version) used for encoding is used for decoding.
3. Wrap decode() in try/catch for CoderException and log/handle corrupt data instead of crashing the pipeline.

Example fix

// before
Long v = VarLongCoder.of().decode(inStream); // throws on truncated input
// after
try {
  Long v = VarLongCoder.of().decode(inStream);
} catch (CoderException e) {
  // handle truncated/corrupt element: log offset, skip, or fail with context
  throw new IOException("Corrupt encoded Long at stream position " + pos, e);
}

if (inStream.available() < 1) { throw new EOFException("empty buffer before VarLong decode"); }

try { v = VarLongCoder.of().decode(in); } catch (CoderException e) { /* truncated/corrupt: log, resync stream, or fail bundle */ }

When it happens

Trigger: Decoding from a truncated buffer/stream (fewer bytes than the varint needs), decoding a stream written with an incompatible coder/format, or reading past the end of a message.

Common situations: Network/gRPC framing bugs that split or truncate messages, mixing coder versions between writer and reader, offset bugs when decoding elements out of a shared byte buffer.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• source decode failed
• stream value decode failed
• retrieving time data: %v
• invalid length " + length
• EOF encountered decoding 1 byte from input stream

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4a2deeb6db48a4e1. Report an issue: GitHub.