apache/beam · error · OverflowError

Integer value '{value}' is out of the encodable range for Va

Error message

Integer value '{value}' is out of the encodable range for VarIntCoder. This coder is limited to values that fit within a 64-bit signed integer (-(2**63) to 2**63 - 1). Original error: {e}

What it means

VarIntCoderImpl.encode_to_stream calls out.write_var_int64, which only accepts 64-bit signed integers. If the value overflows that range, OverflowError is re-raised with a message naming the offending value and the allowed range -(2**63) to 2**63-1. Beam raises this so callers know the value simply cannot be represented by VarIntCoder.

Source

Thrown at sdks/python/apache_beam/coders/coder_impl.py:1049

            in_stream, True),
        hold_timestamp=self._timestamp_coder_impl.decode_from_stream(
            in_stream, True),
        paneinfo=self._pane_info_coder_impl.decode_from_stream(in_stream, True))


small_ints = [chr(_).encode('latin-1') for _ in range(128)]


class VarIntCoderImpl(StreamCoderImpl):
  """For internal use only; no backwards-compatibility guarantees.

  A coder for int objects."""
  def encode_to_stream(self, value, out, nested):
    # type: (int, create_OutputStream, bool) -> None
    try:
      out.write_var_int64(value)
    except OverflowError as e:
      raise OverflowError(
          f"Integer value '{value}' is out of the encodable range for "
          f"VarIntCoder. This coder is limited to values that fit "
          f"within a 64-bit signed integer (-(2**63) to 2**63 - 1). "
          f"Original error: {e}") from e

  def decode_from_stream(self, in_stream, nested):
    # type: (create_InputStream, bool) -> int
    return in_stream.read_var_int64()

  def encode(self, value):
    ivalue = value  # type cast
    if 0 <= ivalue < len(small_ints):
      return small_ints[ivalue]
    return StreamCoderImpl.encode(self, value)

  def decode(self, encoded):
    if len(encoded) == 1:
      i = ord(encoded)

View on GitHub (pinned to 12126d8942)

Solutions

1. Clamp or validate values to the 64-bit signed range before coding
2. Use a different representation for huge numbers (e.g., encode as a string or bytes via a custom/Map coder)
3. Scale units down (e.g., nanoseconds to milliseconds) so values fit in int64
4. Fix upstream arithmetic (use modular arithmetic or floats) so intermediate results stay in range

Example fix

// before
coder = VarIntCoder()
coder.encode(huge_ns_timestamp)  # OverflowError
// after
INT64_MIN, INT64_MAX = -(2**63), 2**63 - 1
if not (INT64_MIN <= value <= INT64_MAX):
    value = value // 1000  # rescale ns -> us
coder.encode(value)

INT64_MIN, INT64_MAX = -(2**63), 2**63 - 1
assert INT64_MIN <= value <= INT64_MAX, f'value {value} not encodable by VarIntCoder'

def fits_int64(v: int) -> bool:
    return -(2**63) <= v <= 2**63 - 1

try:
    coder.encode(value)
except OverflowError as e:
    log.warning('VarInt overflow: %s', e)
    value = rescale_or_clamp(value)
    coder.encode(value)

When it happens

Trigger: Calling VarIntCoder().encode(value) (or a pipeline coding a PCollection of ints) where value >= 2**63 or value < -(2**63), e.g. huge counters, timestamps in nanoseconds since epoch far in the future, or unbounded arithmetic results.

Common situations: Computing timestamps/durations in nanoseconds with int64 overflow; aggregating counts that exceed 2**63; using Python's arbitrary-precision ints from user data without range validation before coding.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• Cannot estimate size for integer value '{value}'. Value is o
• Encountered unexpected value for null indicator: '%s'
• Number of components does not match number of coders.
• Cannot read state-written iterable without state reader.
• Invalid PaneInfoEncoding: %s

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3f366f7ac25c3786. Report an issue: GitHub.