What causes "Attempted to read past end of file" in apache/hadoop?

Treat this as EOF: stop reading and re-stat the file before continuing

apache/hadoop · error · IOException

Attempted to read past end of file

Error message

Attempted to read past end of file

What it means

blockSeekTo() refuses to open a DataNode reader when the target byte position is at or past the stream's current file length - sequential reads must never reach past EOF. Encountered when the reader's position exceeds the real data extent, most often because the cached length is stale after the file was truncated or the last block finalized shorter than expected.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java:612

        LocatedBlock blk = fetchBlockAt(curOff, remaining, true);
        assert curOff >= blk.getStartOffset() : "Block not found";
        blockRange.add(blk);
        long bytesRead = blk.getStartOffset() + blk.getBlockSize() - curOff;
        remaining -= bytesRead;
        curOff += bytesRead;
      }
      return blockRange;
    }
  }

  /**
   * Open a DataInputStream to a DataNode so that it can be read from.
   * We get block ID and the IDs of the destinations at startup, from the namenode.
   */
  private synchronized DatanodeInfo blockSeekTo(long target)
      throws IOException {
    if (target >= getFileLength()) {
      throw new IOException("Attempted to read past end of file");
    }

    maybeRegisterBlockRefresh();

    // Will be getting a new BlockReader.
    closeCurrentBlockReaders();

    //
    // Connect to best DataNode for desired Block, with potential offset
    //
    DatanodeInfo chosenNode;
    int refetchToken = 1; // only need to get a new access token once
    int refetchEncryptionKey = 1; // only need to get a new encryption key once

    boolean connectFailedOnce = false;

    while (true) {
      //

View on GitHub (pinned to 2add963021)

Solutions

Treat this as EOF: stop reading and re-stat the file before continuing
Re-open the stream after the file shrinks to refresh block locations and length
Prevent concurrent truncation of files that have active readers

Defensive patterns

Strategy: validation

Validate before calling

if (in.getPos() >= fs.getFileStatus(path).getLen()) {
  return -1; // at EOF; do not trigger blockSeekTo
}
int n = in.read(buf);

Try / catch

try {
  n = in.read(buf);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().equals("Attempted to read past end of file")) {
    n = -1; // file shrank underneath us: treat as EOF
    reopenStream();
  } else throw e;
}

Prevention

In tail loops, stop when pos reaches the current length and re-stat before continuing
Never truncate files that have active sequential readers
Refresh or re-open streams whenever the file may have shrunk

When it happens

Trigger: Sequential read() advancing pos to a value >= getFileLength() while locatedBlocks still describe a longer file; read retries after a concurrent truncate.

Common situations: Tail readers on files being truncated/rotated; long-lived streams whose file shrank underneath them; mixed read+truncate workloads.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/306408c0e956bfe8. Report an issue: GitHub.