{"record":{"id":"da15c8eed7686b73","repo":"apache/hadoop","slug":"attempted-to-seek-or-read-past-the-end-of-the-file-da15c8","errorCode":null,"errorMessage":"Attempted to seek or read past the end of the file \" + pos","messagePattern":"Attempted to seek or read past the end of the file \" \\+ pos","errorType":"exception","errorClass":"EOFException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/prefetch/S3ARemoteInputStream.java","lineNumber":465,"sourceCode":"\n    StringBuilder sb = new StringBuilder();\n    sb.append(String.format(\"nextReadPos = (%d)%n\", nextReadPos));\n    sb.append(String.format(\"fpos = (%s)\", fpos));\n    return sb.toString();\n  }\n\n  protected void throwIfClosed() throws IOException {\n    if (closed) {\n      throw new IOException(\n          name + \": \" + FSExceptionMessages.STREAM_IS_CLOSED);\n    }\n  }\n\n  protected void throwIfInvalidSeek(long pos) throws EOFException {\n    if (pos < 0) {\n      throw new EOFException(FSExceptionMessages.NEGATIVE_SEEK + \" \" + pos);\n    } else if (pos > this.getBlockData().getFileSize()) {\n      throw new EOFException(FSExceptionMessages.CANNOT_SEEK_PAST_EOF + \" \" + pos);\n    }\n  }\n\n  // Unsupported functions.\n\n  @Override\n  public void mark(int readlimit) {\n    throw new UnsupportedOperationException(\"mark not supported\");\n  }\n\n  @Override\n  public void reset() {\n    throw new UnsupportedOperationException(\"reset not supported\");\n  }\n\n  @Override\n  public long skip(long n) {\n    throw new UnsupportedOperationException(\"skip not supported\");","sourceCodeStart":447,"sourceCodeEnd":483,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/prefetch/S3ARemoteInputStream.java#L447-L483","documentation":"S3ARemoteInputStream.throwIfInvalidSeek() throws EOFException(FSExceptionMessages.CANNOT_SEEK_PAST_EOF + \" \" + pos) when the seek target exceeds the size of the block data it is serving (pos > getBlockData().getFileSize()). Because this stream reads prefetcher block/cache data, the mismatch means either a genuine seek past the object end or a shrunken/corrupt local block file.","triggerScenarios":"seek(pos) where pos >= object length (e.g. seeking to fileLen instead of fileLen-1); prefetch block data file truncated on disk (cache dir out of space, cleanup deleted it); object replaced by a shorter one mid-read and cached metadata is stale.","commonSituations":"Apps computing end-of-file positions with <= vs < errors; disk-full nodes truncating files under fs.s3a.prefetch.buffer.dir / block cache; long-lived readers on objects that were overwritten concurrently.","solutions":["Validate the target against the authoritative length: if (pos >= fs.getFileStatus(path).getLen()) stop or error before seeking.","If you suspect cache corruption, clear the prefetch/block cache directory and reopen the stream.","Ensure the cache directory has disk space and is not aggressively cleaned by external tools.","For objects that change underneath, reopen the stream (fresher metadata) rather than seeking far ahead on an old handle."],"exampleFix":"// before\nlong len = status.getLen();\nin.seek(len); // one past the last byte -> EOFException\n\n// after\nif (pos < fs.getFileStatus(path).getLen()) {\n  in.seek(pos);\n} else {\n  // end of file reached\n}","handlingStrategy":"validation","validationCode":"long len = fs.getFileStatus(path).getLen();\nif (pos >= len) {\n  // genuine end of file: do not seek\n  return -1;\n}\nin.seek(pos);","typeGuard":null,"tryCatchPattern":"try {\n  in.seek(pos);\n} catch (EOFException e) {\n  if (pos >= fs.getFileStatus(path).getLen()) {\n    // true EOF: handle as end-of-input\n  } else {\n    // block cache shrank/corrupt: evict cache dir and reopen\n  }\n}","preventionTips":["Always compare seek targets against a fresh getFileStatus().getLen().","Use pos < len (strictly less) for the last valid byte.","Monitor disk space on the prefetch cache directory."],"tags":["hadoop-aws","prefetch","seek","eof","cache-corruption"],"backgroundTag":"seek-past-eof","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}