{"record":{"id":"93e4d4748ec40e0d","repo":"apache/hadoop","slug":"premature-eof-from-inputstream-after-skipping-len","errorCode":null,"errorMessage":"Premature EOF from inputStream after skipping {len-amt} byte(s).","messagePattern":"Premature EOF from inputStream after skipping (.+?) byte\\(s\\)\\.","errorType":"exception","errorClass":"EOFException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/IOUtils.java","lineNumber":239,"sourceCode":"  }\n  \n  /**\n   * Similar to readFully(). Skips bytes in a loop.\n   * @param in The InputStream to skip bytes from\n   * @param len number of bytes to skip.\n   * @throws IOException if it could not skip requested number of bytes \n   * for any reason (including EOF)\n   */\n  public static void skipFully(InputStream in, long len) throws IOException {\n    long amt = len;\n    while (amt > 0) {\n      long ret = in.skip(amt);\n      if (ret == 0) {\n        // skip may return 0 even if we're not at EOF.  Luckily, we can \n        // use the read() method to figure out if we're at the end.\n        int b = in.read();\n        if (b == -1) {\n          throw new EOFException( \"Premature EOF from inputStream after \" +\n              \"skipping \" + (len - amt) + \" byte(s).\");\n        }\n        ret = 1;\n      }\n      amt -= ret;\n    }\n  }\n  \n  /**\n   * Close the Closeable objects and <b>ignore</b> any {@link Throwable} or\n   * null pointers. Must only be used for cleanup in exception handlers.\n   *\n   * @param logger the log to record problems to at debug level. Can be null.\n   * @param closeables the objects to close\n   */\n  public static void cleanupWithLogger(Logger logger,\n      java.io.Closeable... closeables) {\n    for (java.io.Closeable c : closeables) {","sourceCodeStart":221,"sourceCodeEnd":257,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/IOUtils.java#L221-L257","documentation":"Thrown by IOUtils.skipFully(InputStream, long) when skipping hits end of stream. If in.skip() repeatedly returns 0 the method probes with a single read(); a -1 return proves EOF, and it throws EOFException stating exactly how many of the requested bytes were skipped before the stream ran out. The skipped-count in the message localizes where the stream ended.","triggerScenarios":"Calling skipFully() to position past a region that extends beyond the stream's end: seeking to a record offset from an index (e.g. MapFile index positions), skipping block prefixes of a truncated file, or passing a 'len' larger than the file. Also triggered when skip() returns 0 on a stream that cannot skip (some wrapper streams) and the stream is actually at EOF.","commonSituations":"A MapFile/SequenceFile index referencing offsets beyond a truncated data file (interrupted write, missing HDFS block); reading a file segment [start, start+len) where the split math overshoots the file; input streams wrapped in buffering layers that return 0 from skip near the buffer end combined with real EOF.","solutions":["Check the requested offset+length against the actual file size before skipping (getFileStatus().getLen()).","If the underlying file is truncated, restore or re-write it — the index/split pointing past EOF is a symptom, not the cause.","If the stream class returns 0 from skip() when data remains (custom/wrapper streams), read into a scratch buffer instead of relying on skip().","Catch EOFException specifically — it carries the skipped count, which tells you the exact offset where data ended."],"exampleFix":"// before: blindly trusts an index offset\nIOUtils.skipFully(dataIn, position);\n\n// after: bound-check the offset against the real file length first\nlong fileLen = fs.getFileStatus(dataFile).getLen();\nif (position > fileLen) {\n  throw new EOFException(\"Index offset \" + position + \" beyond EOF \" + fileLen\n      + \" of \" + dataFile + \" — file is truncated or index is stale\");\n}\nIOUtils.skipFully(dataIn, position);","handlingStrategy":"validation","validationCode":"long remaining = fileLen - ((FSDataInputStream) in).getPos();\nif (remaining < bytesToSkip) {\n  throw new EOFException(path + \" ends before skip target\");\n}\nIOUtils.skipFully(in, bytesToSkip);","typeGuard":null,"tryCatchPattern":"try {\n  IOUtils.skipFully(in, len);\n} catch (EOFException e) {\n  // message states how many bytes were skipped — treat as end of usable data\n  LOG.warn(\"stream ended after skip: {}\", e.getMessage());\n  return Record.EOF;\n}","preventionTips":["Bound-check offset+len against the real file length before seeking/skipping.","Regenerate index files (MapFile.fix) after truncation instead of trusting stale offsets.","For custom InputStream wrappers, verify skip() is correct or fall back to read-into-scratch-buffer.","Catch EOFException specifically — unlike IOException it cannot hide unrelated read failures."],"tags":["io","eof","seek","truncated-file","hadoop-common"],"backgroundTag":"truncated-input-stream","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}