{"record":{"id":"b6672c2fd0b1dcdc","repo":"apache/beam","slug":"varint-not-terminated","errorCode":null,"errorMessage":"varint not terminated","messagePattern":"varint not terminated","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/util/VarInt.java","lineNumber":124,"sourceCode":"    if (r < 0 || r >= 1L << 32) {\n      throw new IOException(\"varint overflow \" + r);\n    }\n    return (int) r;\n  }\n\n  /** Decodes a long value from the given stream. */\n  public static long decodeLong(InputStream stream) throws IOException {\n    long result = 0;\n    int shift = 0;\n    int b;\n    do {\n      // Get 7 bits from next byte\n      b = stream.read();\n      if (b < 0) {\n        if (shift == 0) {\n          throw new EOFException();\n        } else {\n          throw new IOException(\"varint not terminated\");\n        }\n      }\n      long bits = b & 0x7F;\n      if (shift >= 64 || (shift == 63 && bits > 1)) {\n        // Out of range\n        throw new IOException(\"varint too long\");\n      }\n      result |= bits << shift;\n      shift += 7;\n    } while ((b & 0x80) != 0);\n    return result;\n  }\n\n  /** Returns the length of the encoding of the given value (in bytes). */\n  public static int getLength(int v) {\n    return CodedOutputStream.computeUInt32SizeNoTag(v);\n  }\n","sourceCodeStart":106,"sourceCodeEnd":142,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/util/VarInt.java#L106-L142","documentation":"decodeLong reads a varint until it encounters a byte with the high bit clear. If the stream ends (read() returns -1) while mid-varint (some bytes already consumed, shift != 0), the encoding is truncated and an IOException 'varint not terminated' is thrown. If no bytes were read at all, an EOFException is thrown instead.","triggerScenarios":"Truncated input stream: the encoded data ends in the middle of a multi-byte varint; reading past the end of a record because length prefixes were misparsed; reading from a socket/file closed early.","commonSituations":"Corrupt or partially written files; network streams cut off mid-message; decoder/writer mismatch where a reader expects a varint at a position where the writer wrote none (schema drift); reading with the wrong offset after a failed earlier decode.","solutions":["Check that the source data is complete and uncorrupted; re-transfer or re-generate the input","Fix framing logic so decodeLong is only called when a full varint is available (e.g. read into a byte[] and bounds-check first)","Catch EOFException vs IOException to distinguish clean end-of-stream from truncation","Ensure writer flushes/closes completely before the reader consumes"],"exampleFix":"// before\nlong v = VarInt.decodeLong(stream); // may throw on truncation\n// after\nbyte[] buf = readFully(stream, maxVarintBytes); // returns null on clean EOF\nlong v = (buf == null) ? -1 : VarInt.decodeLong(new ByteArrayInputStream(buf));","handlingStrategy":"try-catch","validationCode":"// check remaining data before decoding\nif (available(stream) <= 0) return endOfStream; // custom bounds check on buffered input","typeGuard":null,"tryCatchPattern":"try { long v = VarInt.decodeLong(stream); } catch (EOFException e) { /* clean end of stream */ } catch (IOException e) { /* truncated varint: input corrupt or framing bug */ }","preventionTips":["Buffer records fully before decoding varints","Distinguish EOFException (clean EOF) from 'varint not terminated' (truncation)","Ensure writers flush and close before readers consume"],"tags":["java","io","varint","deserialization","truncated-data"],"backgroundTag":"unexpected-response-shape","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}