apache/hadoop · error · EOFException

Attempted to seek or read past the end of the file

Error message

Attempted to seek or read past the end of the file

What it means

reopen(pos) raises EOFException('Attempted to seek or read past the end of the file') when pos exceeds fileSize — the stream knows the object length (store.getFileLength) and refuses positions beyond it. The boundary matters: pos == fileSize is allowed (positioned at EOF), only pos > fileSize throws. Reads at EOF then return -1 rather than throwing.

Source

Thrown at hadoop-cloud-storage-project/hadoop-cos/src/main/java/org/apache/hadoop/fs/cosn/CosNInputStream.java:150

    this.preReadPartSize = conf.getLong(
        CosNConfigKeys.READ_AHEAD_BLOCK_SIZE_KEY,
        CosNConfigKeys.DEFAULT_READ_AHEAD_BLOCK_SIZE);
    this.maxReadPartNumber = conf.getInt(
        CosNConfigKeys.READ_AHEAD_QUEUE_SIZE,
        CosNConfigKeys.DEFAULT_READ_AHEAD_QUEUE_SIZE);

    this.readAheadExecutorService = readAheadExecutorService;
    this.readBufferQueue = new ArrayDeque<>(this.maxReadPartNumber);
    this.closed = false;
  }

  private synchronized void reopen(long pos) throws IOException {
    long partSize;

    if (pos < 0) {
      throw new EOFException(FSExceptionMessages.NEGATIVE_SEEK);
    } else if (pos > this.fileSize) {
      throw new EOFException(FSExceptionMessages.CANNOT_SEEK_PAST_EOF);
    } else {
      if (pos + this.preReadPartSize > this.fileSize) {
        partSize = this.fileSize - pos;
      } else {
        partSize = this.preReadPartSize;
      }
    }

    this.buffer = null;

    boolean isRandomIO = true;
    if (pos == this.nextPos) {
      isRandomIO = false;
    } else {
      while (this.readBufferQueue.size() != 0) {
        if (this.readBufferQueue.element().getStart() != pos) {
          this.readBufferQueue.poll();
        } else {

View on GitHub (pinned to 2add963021)

Solutions

1. Fix exclusive bounds: seeking to len is legal, beyond it is not — change <= to <.
2. Re-stat the file (getFileStatus().getLen()) if its length may have changed since open.
3. Clamp: seek(Math.min(pos, fs.getFileStatus(path).getLen())).

Example fix

// before
long len = fs.getFileStatus(p).getLen();
for (long pos = 0; pos <= len; pos += chunk) { in.seek(pos); } // throws at pos == len + ... 

// after
for (long pos = 0; pos < len; pos += chunk) { in.seek(pos); }

long len = in.getPos() >= 0 ? fs.getFileStatus(p).getLen() : 0;
if (pos > len) {
  pos = len; // position at EOF; subsequent read() returns -1
}
in.seek(pos);

static boolean isSeekPastEof(Throwable t) {
  return t instanceof EOFException && t.getMessage() != null
      && t.getMessage().contains('past the end');
}

try {
  in.seek(pos);
} catch (EOFException e) {
  if (e.getMessage() != null && e.getMessage().contains('past the end')) {
    in.seek(fs.getFileStatus(p).getLen()); // clamp to EOF
  } else { throw e; }
}

When it happens

Trigger: seek(len + 1) after an off-by-one (seek(len) is legal); readers computing end offsets with '<=' loops; seeking to a stale larger length after the object was replaced by a shorter one; tail-like tools polling a growing file before the new length is visible.

Common situations: Split end computed as start + splitSize without clamping to file length; file shrank between status call and read; loop conditions using pos <= len instead of pos < len.

Related errors

• Attempted to seek or read past the end of the file " + targe
• Cannot seek to a negative offset
• Attempted to seek or read past the end of the file
• Cannot seek to a negative offset " + targetPos
• Null IO stream

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/493f8b433062b342. Report an issue: GitHub.