What causes "Attempted to seek or read past the end of the file" in apache/hadoop?

Clamp the seek target to the stream's own length: n = Math.min(n, in.length())

apache/hadoop · error · EOFException

Attempted to seek or read past the end of the file

Error message

Attempted to seek or read past the end of the file

What it means

AbfsInputStream.seek(long n) throws EOFException("Attempted to seek or read past the end of the file") when n > contentLength. contentLength is the length captured when the stream was opened (appends after open are not reflected), so the check uses the stream's snapshot, not a live stat. Seeking exactly to contentLength is legal (EOF position).

Source

Thrown at hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsInputStream.java:779

  }

  /**
   * Seek to given position in stream.
   * @param n position to seek to
   * @throws IOException if there is an error
   * @throws EOFException if attempting to seek past end of file
   */
  @Override
  public synchronized void seek(long n) throws IOException {
    LOG.debug("requested seek to position {}", n);
    if (closed) {
      throw new IOException(FSExceptionMessages.STREAM_IS_CLOSED);
    }
    if (n < 0) {
      throw new EOFException(FSExceptionMessages.NEGATIVE_SEEK);
    }
    if (n > contentLength) {
      throw new EOFException(FSExceptionMessages.CANNOT_SEEK_PAST_EOF);
    }

    if (streamStatistics != null) {
      streamStatistics.seek(n, fCursor);
    }

    // next read will read from here
    nextReadPos = n;
    LOG.debug("set nextReadPos to {}", nextReadPos);
  }

  @Override
  public synchronized long skip(long n) throws IOException {
    if (closed) {
      throw new IOException(FSExceptionMessages.STREAM_IS_CLOSED);
    }
    long currentPos = getPos();
    if (currentPos == contentLength) {

View on GitHub (pinned to 2add963021)

Solutions

Clamp the seek target to the stream's own length: n = Math.min(n, in.length())
Stat fresh before seeking when files can change: fs.getFileStatus(path).getLen()
If the file legitimately shrank, reopen the stream so contentLength is refreshed
Handle the end case explicitly: if target >= length, treat as EOF instead of seeking

Example fix

// before
in.seek(offsetFromIndexFile);  // offset > current file length -> EOFException

// after
long target = Math.min(offsetFromIndexFile, in.length());
if (target >= in.length()) {
  return -1; // or handle EOF explicitly
}
in.seek(target);

Defensive patterns

Strategy: validation

Validate before calling

// Bound every seek by the stream's own length snapshot
long len = in.length();
if (target > len) target = len;          // clamp to EOF
if (target >= len && needData) {
  // nothing left to read — treat as EOF, don't seek
}
in.seek(target);

Try / catch

try {
  in.seek(target);
} catch (EOFException e) {
  long len = fs.getFileStatus(path).getLen();   // file may have shrunk
  if (target > len) { /* stale index/length — rebuild it */ }
  else throw e;
}

Prevention

Re-stat files that can be rewritten concurrently before trusting cached lengths
Validate index offsets against fs.getFileStatus(path).getLen() before use
Treat seek-to-EOF as a normal end condition, not an error path

When it happens

Trigger: Calling seek(n) with n greater than the file's length: hardcoded expected sizes, indexes built against a longer version of the file, stale FileStatus/FileLength from an earlier stat, or a file truncated/overwritten between stat and seek; off-by-one bugs seeking to len+1.

Common situations: File replaced concurrently by a writer producing a shorter file; directory metadata caches handing out old lengths; merge/compaction jobs that shrink files while readers hold open streams; input-split computation done from outdated FileStatus.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/76978868849e1788. Report an issue: GitHub.