apache/beam · error · ValueError

Value too large (negative).

Error message

Value too large (negative).

What it means

OutputStream.write_var_int64 encodes a 64-bit unsigned varint; negative Python ints are first offset by 2**64. If adding 2**64 still yields a non-positive value (i.e. v <= -(2**64)), the value cannot fit in 64 bits and a ValueError 'Value too large (negative).' is raised.

Solutions

1. Validate the value is within int64 range (-(2**63) <= v <= 2**63-1) before calling write_var_int64
2. If the value should be signed, encode a zig-zag or explicit sign marker rather than raw negatives
3. Fix upstream computation producing values beyond 64-bit range

Example fix

// before
stream.write_var_int64(delta)  # delta may be < -(2**63)
// after
assert -(1 << 63) <= delta < (1 << 63), 'value out of int64 range'
stream.write_var_int64(delta)

if not (-(1 << 64) < v <= (1 << 64) - 1):
    raise ValueError(f'value {v} cannot be written as a 64-bit varint')

def in_uint64_varint_range(v: int) -> bool:
    return -(1 << 64) < v <= (1 << 64) - 1

try:
    stream.write_var_int64(v)
except ValueError as e:
    log.error('varint out of range: %s', v)
    raise

When it happens

Trigger: Writing a varint value <= -(2**64) (out of the int64/uint64 varint range) to a slow_stream OutputStream via write_var_int64 (also reached through write_var_int32/write/ByteCountingOutputStream).

Common situations: Corrupted or unvalidated numeric values flowing into custom coders/streams; arithmetic bugs producing extreme negative numbers before serialization; manually implementing a coder that writes raw negative values.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• Attempted to encode null for non-nullable field
• Cannot estimate size for integer value
• Cannot safely index records from
• Decode not implemented
• Encode not implemented

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a5781bb1786a212d. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/coders/slow_stream.py:50

    self.data: list[bytes] = []
    self.byte_count = 0

  def write(self, b: bytes, nested: bool = False) -> None:
    assert isinstance(b, bytes)
    if nested:
      self.write_var_int64(len(b))
    self.data.append(b)
    self.byte_count += len(b)

  def write_byte(self, val):
    self.data.append(chr(val).encode('latin-1'))
    self.byte_count += 1

  def write_var_int64(self, v: int) -> None:
    if v < 0:
      v += 1 << 64
      if v <= 0:
        raise ValueError('Value too large (negative).')
    while True:
      bits = v & 0x7F
      v >>= 7
      if v:
        bits |= 0x80
      self.write_byte(bits)
      if not v:
        break

  def write_var_int32(self, v: int) -> None:
    self.write_var_int64(int(v) & 0xFFFFFFFF)

  def write_bigendian_int64(self, v):
    self.write(struct.pack('>q', v))

  def write_bigendian_uint64(self, v):
    self.write(struct.pack('>Q', v))

View on GitHub (pinned to 12126d8942)