{"record":{"id":"d39cec5b3cfa15b7","repo":"apache/beam","slug":"varlong-too-long-stream","errorCode":null,"errorMessage":"VarLong too long.","messagePattern":"VarLong too long\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/coders/stream.pyx","lineNumber":229,"sourceCode":"  cpdef bytes read_all(self, bint nested=False):\n    return self.read(<ssize_t>self.read_var_int64() if nested else self.size())\n\n  cpdef libc.stdint.int64_t read_var_int64(self) except? -1:\n    \"\"\"Decode a variable-length encoded long from a stream.\"\"\"\n    # Inline common case.\n    cdef long byte = <unsigned char> self.allc[self.pos]\n    self.pos += 1\n    if byte <= 0x7F:\n      return byte\n\n    cdef libc.stdint.int64_t bits\n    cdef long shift = 0\n    cdef libc.stdint.int64_t result = 0\n    while True:\n      bits = byte & 0x7F\n      if (shift >= sizeof(libc.stdint.int64_t) * 8 or\n          (shift >= (sizeof(libc.stdint.int64_t) * 8 - 1) and bits > 1)):\n        raise RuntimeError('VarLong too long.')\n      result |= bits << shift\n      shift += 7\n      if not (byte & 0x80):\n        break\n      byte = self.read_byte()\n      if byte < 0:\n        raise RuntimeError('VarInt not terminated.')\n\n    return result\n\n  cpdef libc.stdint.int32_t read_var_int32(self) except? -1:\n    \"\"\"Decode a variable-length encoded int32 from a stream.\"\"\"\n    cdef libc.stdint.int64_t v = self.read_var_int64()\n    return <libc.stdint.int32_t>(v);\n\n  cpdef libc.stdint.int64_t read_bigendian_int64(self) except? -1:\n    return self.read_bigendian_uint64()\n","sourceCodeStart":211,"sourceCodeEnd":247,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/coders/stream.pyx#L211-L247","documentation":"Beam's OutputStream.read_var_int64 decodes a LEB128-style varint from the byte stream. If the decoded value would need more bits than a 64-bit integer (shift exceeding 64 bits, or the last allowed byte carrying bits above the sign bit), the coder raises RuntimeError('VarLong too long.') to prevent silent overflow/corruption.","triggerScenarios":"Reading a byte stream whose varint encoding exceeds 64 bits — corrupted data, reading from the wrong offset (misaligned stream), or a producer encoding values with a different/incompatible varint scheme.","commonSituations":"Beam pipeline reading shards written by an incompatible writer version; byte-offset corruption after a failed write; custom coders misaligning the stream so a length prefix is decoded as a varint.","solutions":["Verify the stream/writer Beam SDK version matches the reader (coder compatibility)","Check for stream misalignment: ensure prior reads (lengths, headers) consumed the correct byte counts","Validate the integrity of the source data (re-run/rewrite the affected shard)","Reproduce with a hexdump of the offending bytes and confirm the varint is <= 10 bytes with valid termination"],"exampleFix":"null","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    value = stream.read_var_int64()\nexcept RuntimeError as e:\n    if 'VarLong too long' in str(e):\n        mark_source_corrupt(); resync_or_rewrite_stream()\n    else:\n        raise","preventionTips":["Match reader/writer Beam SDK versions for coder compatibility","Validate data integrity (checksums) for stored shards","Never hand-decode/re-encode varints with a different scheme"],"tags":["coders","varint","corruption","python"],"backgroundTag":"value-out-of-range","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}