{"record":{"id":"26a18488ccb79bb6","repo":"apache/beam","slug":"varint-not-terminated-stream","errorCode":null,"errorMessage":"VarInt not terminated.","messagePattern":"VarInt not terminated\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/coders/stream.pyx","lineNumber":236,"sourceCode":"    self.pos += 1\n    if byte <= 0x7F:\n      return byte\n\n    cdef libc.stdint.int64_t bits\n    cdef long shift = 0\n    cdef libc.stdint.int64_t result = 0\n    while True:\n      bits = byte & 0x7F\n      if (shift >= sizeof(libc.stdint.int64_t) * 8 or\n          (shift >= (sizeof(libc.stdint.int64_t) * 8 - 1) and bits > 1)):\n        raise RuntimeError('VarLong too long.')\n      result |= bits << shift\n      shift += 7\n      if not (byte & 0x80):\n        break\n      byte = self.read_byte()\n      if byte < 0:\n        raise RuntimeError('VarInt not terminated.')\n\n    return result\n\n  cpdef libc.stdint.int32_t read_var_int32(self) except? -1:\n    \"\"\"Decode a variable-length encoded int32 from a stream.\"\"\"\n    cdef libc.stdint.int64_t v = self.read_var_int64()\n    return <libc.stdint.int32_t>(v);\n\n  cpdef libc.stdint.int64_t read_bigendian_int64(self) except? -1:\n    return self.read_bigendian_uint64()\n\n  cpdef libc.stdint.uint64_t read_bigendian_uint64(self) except? -1:\n    self.pos += 8\n    return (<unsigned char>self.allc[self.pos - 1]\n      | <libc.stdint.uint64_t><unsigned char>self.allc[self.pos - 2] <<  8\n      | <libc.stdint.uint64_t><unsigned char>self.allc[self.pos - 3] << 16\n      | <libc.stdint.uint64_t><unsigned char>self.allc[self.pos - 4] << 24\n      | <libc.stdint.uint64_t><unsigned char>self.allc[self.pos - 5] << 32","sourceCodeStart":218,"sourceCodeEnd":254,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/coders/stream.pyx#L218-L254","documentation":"read_var_int64 keeps consuming bytes while the continuation bit (0x80) is set; read_byte() returning a negative value signals end-of-stream. If EOF is hit before a terminating byte (0x00–0x7F) is seen, the coder raises RuntimeError('VarInt not terminated.') because the encoded integer is truncated.","triggerScenarios":"Reading a truncated stream — a varint's bytes are cut off by EOF (truncated file/shard, closed gRPC stream, short read) while the last byte still has the continuation bit set.","commonSituations":"Partially written output files read before flush/close; network streams cut mid-record; wrong buffer sizing causing reads past the record boundary.","solutions":["Check the source for truncation (file size, shard completeness) and re-read/rewrite the data","Ensure the writer flushed/closed the stream before the reader consumed it","Verify buffer framing: don't call read_var_int64 past the end of a length-delimited record","Catch the RuntimeError and treat the stream as corrupt/ended rather than retrying the read"],"exampleFix":"# before\nvalue = stream.read_var_int64()  # raises on truncated tail\n# after\ntry:\n    value = stream.read_var_int64()\nexcept RuntimeError as e:\n    if 'VarInt not terminated' in str(e):\n        handle_truncated_stream()  # stop reading, mark source incomplete\n    else:\n        raise","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    value = stream.read_var_int64()\nexcept RuntimeError as e:\n    if 'VarInt not terminated' in str(e):\n        handle_truncated_input()  # stop, mark incomplete\n    else:\n        raise","preventionTips":["Ensure writers flush/close before readers consume the stream","Check file/shard completeness before reading","Respect length-delimited record boundaries in buffer reads"],"tags":["coders","varint","truncated-input","python"],"backgroundTag":"unexpected-response-shape","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}