{"record":{"id":"a119190161f633a6","repo":"apache/beam","slug":"cannot-estimate-size-for-integer-value-value-value-is-out-of","errorCode":null,"errorMessage":"Cannot estimate size for integer value '{value}'. Value is out of the range for VarIntCoder (64-bit signed integer). Original error: {e}","messagePattern":"Cannot estimate size for integer value '(.+?)'\\. Value is out of the range for VarIntCoder \\(64-bit signed integer\\)\\. Original error: (.+?)","errorType":"exception","errorClass":"OverflowError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/coders/coder_impl.py","lineNumber":1078,"sourceCode":"    ivalue = value  # type cast\n    if 0 <= ivalue < len(small_ints):\n      return small_ints[ivalue]\n    return StreamCoderImpl.encode(self, value)\n\n  def decode(self, encoded):\n    if len(encoded) == 1:\n      i = ord(encoded)\n      if 0 <= i < 128:\n        return i\n    return StreamCoderImpl.decode(self, encoded)\n\n  def estimate_size(self, value, nested=False):\n    # type: (Any, bool) -> int\n    # Note that VarInts are encoded the same way regardless of nesting.\n    try:\n      return get_varint_size(value)\n    except OverflowError as e:\n      raise OverflowError(\n          f\"Cannot estimate size for integer value '{value}'. \"\n          f\"Value is out of the range for VarIntCoder (64-bit signed integer). \"\n          f\"Original error: {e}\") from e\n\n\nclass VarInt32CoderImpl(StreamCoderImpl):\n  \"\"\"For internal use only; no backwards-compatibility guarantees.\n\n  A coder for int32 objects.\"\"\"\n  def encode_to_stream(self, value, out, nested):\n    # type: (int, create_OutputStream, bool) -> None\n    out.write_var_int32(value)\n\n  def decode_from_stream(self, in_stream, nested):\n    # type: (create_InputStream, bool) -> int\n    return in_stream.read_var_int32()\n\n  def encode(self, value):","sourceCodeStart":1060,"sourceCodeEnd":1096,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/coders/coder_impl.py#L1060-L1096","documentation":"VarIntCoderImpl.estimate_size computes the encoded size via get_varint_size, which requires the value to fit a 64-bit signed integer. On overflow it re-raises OverflowError explaining that the size cannot be estimated. This mirrors the encode-side limit: a value outside int64 cannot be VarInt-encoded at all.","triggerScenarios":"Calling VarIntCoder().estimate_size(value) (directly or via pipeline size estimation of an int PCollection) with value outside -(2**63) to 2**63-1.","commonSituations":"Same root cause as encode overflow: nanosecond timestamps, giant counters, or arbitrary-precision Python ints reaching size estimation before encode fails; often seen in runner-side size estimation of elements.","solutions":["Validate/clamp the int into the 64-bit signed range before it reaches the coder","Rescale units (ns to ms/us) so the value fits int64","Switch huge numbers to a string/bytes-based coder where size estimation is well-defined","Fix the upstream computation that produces out-of-range ints"],"exampleFix":"// before\nsize = VarIntCoder().estimate_size(bignum)  # OverflowError\n// after\nINT64_MAX = 2**63 - 1\nif bignum > INT64_MAX or bignum < -(2**63):\n    bignum = max(-(2**63), min(INT64_MAX, bignum))  # clamp\nsize = VarIntCoder().estimate_size(bignum)","handlingStrategy":"validation","validationCode":"INT64_MIN, INT64_MAX = -(2**63), 2**63 - 1\nassert INT64_MIN <= value <= INT64_MAX, f'cannot estimate VarInt size for {value}'","typeGuard":"def fits_int64(v: int) -> bool:\n    return -(2**63) <= v <= 2**63 - 1","tryCatchPattern":"try:\n    size = coder.estimate_size(value)\nexcept OverflowError as e:\n    log.warning('Size estimation failed: %s', e)\n    size = coder.estimate_size(clamp_to_int64(value))","preventionTips":["Apply the same int64 range validation used before encode, since estimate_size shares the limit","Clamp/rescale values before they reach any coder call path, including runner size estimation","Keep intermediate arithmetic within int64 or switch representation early"],"tags":["python","apache-beam","coding","integer-overflow"],"backgroundTag":"value-out-of-range","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}