apache/hadoop · error · EOFException

Range extends beyond the file length ({l}): {last}

Error message

Range extends beyond the file length ({l}): {last}

What it means

Companion check in VectoredReadUtils.validateAndSortRanges: the last range (after sorting) must end at or before the file length. A range that starts inside the file but whose offset+length exceeds the length is rejected with EOFException — vectored reads never silently truncate a range to EOF, unlike plain positioned reads.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/VectoredReadUtils.java:385

        if (prev != null) {
          checkArgument(current.getOffset() >= prev.getOffset() + prev.getLength(),
              "Overlapping ranges %s and %s", prev, current);
        }
        prev = current;
      }
    }
    // at this point the final element in the list is the last range
    // so make sure it is not beyond the end of the file, if passed in.
    // where invalid is: starts at or after the end of the file
    if (fileLength.isPresent()) {
      final FileRange last = sortedRanges.get(sortedRanges.size() - 1);
      final Long l = fileLength.get();
      // this check is superfluous, but it allows for different exception message.
      if (last.getOffset() >= l) {
        throw new EOFException("Range starts beyond the file length (" + l + "): " + last);
      }
      if (last.getOffset() + last.getLength() > l) {
        throw new EOFException("Range extends beyond the file length (" + l + "): " + last);
      }
    }
    return sortedRanges;
  }

  /**
   * Sort the input ranges by offset; no validation is done.
   * @param input input ranges.
   * @return a new list of the ranges, sorted by offset.
   */
  public static List<? extends FileRange> sortRangeList(List<? extends FileRange> input) {
    final List<? extends FileRange> l = new ArrayList<>(input);
    l.sort(Comparator.comparingLong(FileRange::getOffset));
    return l;
  }

  /**
   * Sort the input ranges by offset; no validation is done.

View on GitHub (pinned to 2add963021)

Solutions

1. Clamp each range to EOF: int n = (int) Math.min(range.getLength(), len - range.getOffset()); submit new FileRange(range.getOffset(), n, range.getBackend())
2. Stat the file immediately before the read and build ranges from the fresh length
3. If partial trailing data is acceptable, split the range at EOF and read only the in-bounds portion
4. Catch EOFException, re-stat, clamp, and retry once for races against file truncation

Example fix

// before
ranges.add(new FileRange(offset, FIXED_CHUNK));
in.readVectored(ranges); // fails when offset + FIXED_CHUNK > len

// after
long len = fs.getFileStatus(path).getLen();
for (long offset : offsets) {
  int n = (int) Math.min(FIXED_CHUNK, len - offset);
  if (n > 0) {
    ranges.add(new FileRange(offset, n));
  }
}
in.readVectored(ranges);

long len = fs.getFileStatus(path).getLen();
List<FileRange> clamped = new ArrayList<>();
for (FileRange r : ranges) {
  long end = Math.min(r.getOffset() + r.getLength(), len);
  if (r.getOffset() < end) {
    clamped.add(new FileRange(r.getOffset(), (int) (end - r.getOffset())));
  }
}

static boolean isRangeWithinFile(FileRange r, long fileLen) {
  return r.getOffset() >= 0 && r.getOffset() + r.getLength() <= fileLen;
}

try {
  in.readVectored(ranges);
} catch (EOFException e) {
  long fresh = fs.getFileStatus(path).getLen();
  in.readVectored(clampRanges(ranges, fresh)); // retry once with clamped ranges
}

When it happens

Trigger: readVectored with a range such as new FileRange(len - 10, 100) on a file of length len: it starts before EOF but ends 90 bytes past the end. Also ranges sized from a file length fetched before the file shrank (truncation/rewrite).

Common situations: Fixed-size chunk readers that do not clamp the final chunk (fixed-length records, checksum/parity blocks); cached file sizes; files rewritten smaller between status check and read.

Related errors

• Range starts beyond the file length ({l}): {last}
• Attempted to seek or read past the end of the file " + targe
• Attempted to seek or read past the end of the file
• Invalid seek offset: position value (%d) must be between 0 a
• Attempted to seek or read past the end of the file

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/58d42035fe5ab87c. Report an issue: GitHub.