apache/beam · error · RuntimeError

VarLong too long.

Error message

VarLong too long.

What it means

Beam's OutputStream.read_var_int64 decodes a LEB128-style varint from the byte stream. If the decoded value would need more bits than a 64-bit integer (shift exceeding 64 bits, or the last allowed byte carrying bits above the sign bit), the coder raises RuntimeError('VarLong too long.') to prevent silent overflow/corruption.

Solutions

1. Verify the stream/writer Beam SDK version matches the reader (coder compatibility)
2. Check for stream misalignment: ensure prior reads (lengths, headers) consumed the correct byte counts
3. Validate the integrity of the source data (re-run/rewrite the affected shard)
4. Reproduce with a hexdump of the offending bytes and confirm the varint is <= 10 bytes with valid termination

Example fix

null

try:
    value = stream.read_var_int64()
except RuntimeError as e:
    if 'VarLong too long' in str(e):
        mark_source_corrupt(); resync_or_rewrite_stream()
    else:
        raise

When it happens

Trigger: Reading a byte stream whose varint encoding exceeds 64 bits — corrupted data, reading from the wrong offset (misaligned stream), or a producer encoding values with a different/incompatible varint scheme.

Common situations: Beam pipeline reading shards written by an incompatible writer version; byte-offset corruption after a failed write; custom coders misaligning the stream so a length prefix is decoded as a varint.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• VarInt not terminated.
• Bad coder for input of
• Bad coder for output of
• 'Cannot provide coder for
• Coder registration requires a coder class object. Received…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d39cec5b3cfa15b7. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/coders/stream.pyx:229

  cpdef bytes read_all(self, bint nested=False):
    return self.read(<ssize_t>self.read_var_int64() if nested else self.size())

  cpdef libc.stdint.int64_t read_var_int64(self) except? -1:
    """Decode a variable-length encoded long from a stream."""
    # Inline common case.
    cdef long byte = <unsigned char> self.allc[self.pos]
    self.pos += 1
    if byte <= 0x7F:
      return byte

    cdef libc.stdint.int64_t bits
    cdef long shift = 0
    cdef libc.stdint.int64_t result = 0
    while True:
      bits = byte & 0x7F
      if (shift >= sizeof(libc.stdint.int64_t) * 8 or
          (shift >= (sizeof(libc.stdint.int64_t) * 8 - 1) and bits > 1)):
        raise RuntimeError('VarLong too long.')
      result |= bits << shift
      shift += 7
      if not (byte & 0x80):
        break
      byte = self.read_byte()
      if byte < 0:
        raise RuntimeError('VarInt not terminated.')

    return result

  cpdef libc.stdint.int32_t read_var_int32(self) except? -1:
    """Decode a variable-length encoded int32 from a stream."""
    cdef libc.stdint.int64_t v = self.read_var_int64()
    return <libc.stdint.int32_t>(v);

  cpdef libc.stdint.int64_t read_bigendian_int64(self) except? -1:
    return self.read_bigendian_uint64()

View on GitHub (pinned to 12126d8942)