nathanmarz/storm · error · IOException

Not able to skip bytes, possibly due to end of input.

Error message

Not able to skip ${len} bytes, possibly due to end of input.

What it means

WritableUtils.skipFully repeatedly calls DataInputStream.skipBytes until it has skipped the requested number of bytes. If the stream ends before that many bytes were skipped, it throws IOException, indicating truncated or corrupted input.

Solutions

1. Verify the source data is complete and untruncated (file size, transfer integrity); re-transfer or regenerate the input.
2. Validate the length prefix before skipping: if len is implausibly large vs. known record size, the stream is corrupt — abort and resync/rebuild.
3. Wrap the read in try-catch (IOException) and treat it as end-of-stream/corruption: stop consuming and fail the record gracefully.
4. Check reader/writer version compatibility so length headers are interpreted correctly.

Example fix

// before
WritableUtils.skipCompressedByteArray(in); // throws at EOF on truncated input
// after
try {
    WritableUtils.skipCompressedByteArray(in);
} catch (IOException e) {
    LOG.error("Truncated record, aborting read: " + e.getMessage());
    throw new CorruptedRecordException(e);
}

if (len < 0) throw new IllegalArgumentException("Negative skip length: " + len);
// optionally: if (len > in.available() && in instanceof DataInputStream && knownBounds) warn truncated;

try {
    WritableUtils.skipCompressedByteArray(in);
} catch (IOException e) {
    // stream ended early: treat as truncated/corrupt input
    throw new EOFException("Record truncated: " + e.getMessage());
}

When it happens

Trigger: Calling skipCompressedByteArray/skipFully with a length larger than the remaining bytes in the stream — reading past the end of a truncated file, a socket stream closed early, or a byte array written with a corrupted/overstated length prefix.

Common situations: Partially written or truncated serialization files; network interruption mid-record; mismatched reader/writer versions where a length header claims more data than was written; reading a compressed byte array whose underlying data is shorter than advertised.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Error when launching multilang subprocess
• Could not get canonical path for
• Could not find component with id
• Don't know how to convert
• Version already exists or data already exists

AI-assisted analysis of nathanmarz/storm@cdb116e942 (2026-09-12). Data as JSON: /api/errors/606688732f72b2b8. Report an issue: GitHub.

Appendix: source

Thrown at storm-core/src/jvm/backtype/storm/utils/WritableUtils.java:371

    return (dataBits + 7) / 8 + 1;
  }

  /**
   * Skip <i>len</i> number of bytes in input stream<i>in</i>
   * @param in input stream
   * @param len number of bytes to skip
   * @throws IOException when skipped less number of bytes
   */
  public static void skipFully(DataInput in, int len) throws IOException {
    int total = 0;
    int cur = 0;

    while ((total<len) && ((cur = in.skipBytes(len-total)) > 0)) {
        total += cur;
    }

    if (total<len) {
      throw new IOException("Not able to skip " + len + " bytes, possibly " +
                            "due to end of input.");
    }
  }
}

View on GitHub (pinned to cdb116e942)