{"record":{"id":"606688732f72b2b8","repo":"nathanmarz/storm","slug":"not-able-to-skip-len-bytes-possibly-due-to-end-of-input","errorCode":null,"errorMessage":"Not able to skip ${len} bytes, possibly due to end of input.","messagePattern":"Not able to skip (.+?) bytes, possibly due to end of input\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"storm-core/src/jvm/backtype/storm/utils/WritableUtils.java","lineNumber":371,"sourceCode":"    return (dataBits + 7) / 8 + 1;\n  }\n\n  /**\n   * Skip <i>len</i> number of bytes in input stream<i>in</i>\n   * @param in input stream\n   * @param len number of bytes to skip\n   * @throws IOException when skipped less number of bytes\n   */\n  public static void skipFully(DataInput in, int len) throws IOException {\n    int total = 0;\n    int cur = 0;\n\n    while ((total<len) && ((cur = in.skipBytes(len-total)) > 0)) {\n        total += cur;\n    }\n\n    if (total<len) {\n      throw new IOException(\"Not able to skip \" + len + \" bytes, possibly \" +\n                            \"due to end of input.\");\n    }\n  }\n}\n","sourceCodeStart":353,"sourceCodeEnd":376,"githubUrl":"https://github.com/nathanmarz/storm/blob/cdb116e942666973bc4eaa0df098d5bab82739e7/storm-core/src/jvm/backtype/storm/utils/WritableUtils.java#L353-L376","documentation":"WritableUtils.skipFully repeatedly calls DataInputStream.skipBytes until it has skipped the requested number of bytes. If the stream ends before that many bytes were skipped, it throws IOException, indicating truncated or corrupted input.","triggerScenarios":"Calling skipCompressedByteArray/skipFully with a length larger than the remaining bytes in the stream — reading past the end of a truncated file, a socket stream closed early, or a byte array written with a corrupted/overstated length prefix.","commonSituations":"Partially written or truncated serialization files; network interruption mid-record; mismatched reader/writer versions where a length header claims more data than was written; reading a compressed byte array whose underlying data is shorter than advertised.","solutions":["Verify the source data is complete and untruncated (file size, transfer integrity); re-transfer or regenerate the input.","Validate the length prefix before skipping: if len is implausibly large vs. known record size, the stream is corrupt — abort and resync/rebuild.","Wrap the read in try-catch (IOException) and treat it as end-of-stream/corruption: stop consuming and fail the record gracefully.","Check reader/writer version compatibility so length headers are interpreted correctly."],"exampleFix":"// before\nWritableUtils.skipCompressedByteArray(in); // throws at EOF on truncated input\n// after\ntry {\n    WritableUtils.skipCompressedByteArray(in);\n} catch (IOException e) {\n    LOG.error(\"Truncated record, aborting read: \" + e.getMessage());\n    throw new CorruptedRecordException(e);\n}","handlingStrategy":"try-catch","validationCode":"if (len < 0) throw new IllegalArgumentException(\"Negative skip length: \" + len);\n// optionally: if (len > in.available() && in instanceof DataInputStream && knownBounds) warn truncated;","typeGuard":null,"tryCatchPattern":"try {\n    WritableUtils.skipCompressedByteArray(in);\n} catch (IOException e) {\n    // stream ended early: treat as truncated/corrupt input\n    throw new EOFException(\"Record truncated: \" + e.getMessage());\n}","preventionTips":["Verify input integrity (checksums, complete file transfer) before deserializing","Check reader/writer version compatibility for record length headers","Catch IOException around stream reads and fail the single record instead of the whole job"],"tags":["java","io","stream","deserialization"],"backgroundTag":"file-read-failed","analyzedSha":"cdb116e942666973bc4eaa0df098d5bab82739e7","analyzedAt":"2026-09-12T14:30:00.714Z","contentChangedAt":"2026-09-12T14:30:00.714Z","schemaVersion":2},"datasetVersion":"2026-09-16T09:17:16.951Z"}