apache/hadoop · error · IOException

offset < 0 || offset >= getFileLength(), offset={}, locatedB

Error message

offset < 0 || offset >= getFileLength(), offset={}, locatedBlocks={}

What it means

getBlockAt() hard-validates that the requested byte offset lies in [0, getFileLength()) before locating the block. An offset outside that range throws immediately. Since the check uses the client's current view of the length, it fires both on genuine caller bugs (negative or past-EOF offsets) and on stale-length races where the file was truncated after the stream cached its length.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java:479

  }

  /**
   * Get block at the specified position.
   * Fetch it from the namenode if not cached.
   *
   * @param offset block corresponding to this offset in file is returned
   * @return located block
   * @throws IOException
   */
  protected LocatedBlock getBlockAt(long offset) throws IOException {
    synchronized(infoLock) {
      assert (locatedBlocks != null) : "locatedBlocks is null";

      final LocatedBlock blk;

      //check offset
      if (offset < 0 || offset >= getFileLength()) {
        throw new IOException("offset < 0 || offset >= getFileLength(), offset="
            + offset
            + ", locatedBlocks=" + locatedBlocks);
      }
      else if (offset >= locatedBlocks.getFileLength()) {
        // offset to the portion of the last block,
        // which is not known to the name-node yet;
        // getting the last block
        blk = locatedBlocks.getLastLocatedBlock();
      }
      else {
        // search cached blocks first
        blk = fetchBlockAt(offset, 0, true);
      }
      return blk;
    }
  }

  /** Fetch a block from namenode and cache it */

View on GitHub (pinned to 2add963021)

Solutions

1. Clamp offsets into [0, length-1] using a freshly fetched file length before seeking/reading
2. Re-fetch getFileStatus().getLen() after any event that can shrink the file (truncate by another writer)
3. Fix caller-side math that produces negative offsets (usually underflow after subtraction)

Example fix

// before
long offset = end - remaining; // can go negative or past EOF
in.seek(offset);

// after
long len = fs.getFileStatus(path).getLen();
long offset = Math.max(0, Math.min(end - remaining, len - 1));
in.seek(offset);

long len = fs.getFileStatus(path).getLen();
if (offset < 0 || offset >= len) {
  throw new IllegalArgumentException("offset " + offset + " outside [0," + len + ")");
}

try {
  in.seek(offset);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("offset < 0")) {
    long len = fs.getFileStatus(path).getLen();
    in.seek(Math.max(0, Math.min(offset, len - 1)));
  } else throw e;
}

When it happens

Trigger: seek()/read paths computing an offset >= current file length (or < 0); file truncated by another client between the client's last length refresh and this call.

Common situations: Reader arithmetic that assumes a longer file (stale FileStatus); concurrent truncate racing a tailer; off-by-one loop bounds in custom readers.

Related errors

• Offset: {} exceeds file length: {}
• Offset {startOffset} and length {length} don't match block
• Could not find target position {}
• Path part {s} from URI {p} is not a valid filename.
• {key} = {v} <= 0

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8a1b54df14c9a798. Report an issue: GitHub.