apache/hadoop · error · EOFException

Range starts beyond the file length ({l}): {last}

Error message

Range starts beyond the file length ({l}): {last}

What it means

Thrown by VectoredReadUtils.validateAndSortRanges, the validation pass behind vectored read APIs (FSDataInputStream.readVectored / readFullyVectored). After ranges are sorted by offset, the last range is checked against the known file length; if its offset is at or beyond the length, not a single byte of it can be satisfied and an EOFException is raised. The check only fires when the filesystem supplies the file length (the Optional is present).

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/VectoredReadUtils.java:382

      FileRange prev = null;
      for (final FileRange current : sortedRanges) {
        validateRangeRequest(current);
        if (prev != null) {
          checkArgument(current.getOffset() >= prev.getOffset() + prev.getLength(),
              "Overlapping ranges %s and %s", prev, current);
        }
        prev = current;
      }
    }
    // at this point the final element in the list is the last range
    // so make sure it is not beyond the end of the file, if passed in.
    // where invalid is: starts at or after the end of the file
    if (fileLength.isPresent()) {
      final FileRange last = sortedRanges.get(sortedRanges.size() - 1);
      final Long l = fileLength.get();
      // this check is superfluous, but it allows for different exception message.
      if (last.getOffset() >= l) {
        throw new EOFException("Range starts beyond the file length (" + l + "): " + last);
      }
      if (last.getOffset() + last.getLength() > l) {
        throw new EOFException("Range extends beyond the file length (" + l + "): " + last);
      }
    }
    return sortedRanges;
  }

  /**
   * Sort the input ranges by offset; no validation is done.
   * @param input input ranges.
   * @return a new list of the ranges, sorted by offset.
   */
  public static List<? extends FileRange> sortRangeList(List<? extends FileRange> input) {
    final List<? extends FileRange> l = new ArrayList<>(input);
    l.sort(Comparator.comparingLong(FileRange::getOffset));
    return l;
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Stat the file (fs.getFileStatus(path).getLen()) and require every range offset to be strictly less than the length before issuing the vectored read
2. Filter ranges client-side: drop any range with offset >= len instead of submitting it
3. If the length may be stale, catch EOFException, re-stat the file, rebuild ranges, and retry once
4. For optional trailing data (e.g. footers), treat offset >= length as 'no data present' and skip the range rather than failing the read

Example fix

// before
List<FileRange> ranges = List.of(new FileRange(cachedLen, 128));
in.readVectored(ranges); // EOFException when cachedLen >= real length

// after
long len = fs.getFileStatus(path).getLen();
List<FileRange> valid = ranges.stream()
    .filter(r -> r.getOffset() < len)
    .collect(Collectors.toList());
if (!valid.isEmpty()) {
  in.readVectored(valid);
}

long len = fs.getFileStatus(path).getLen();
List<FileRange> safe = ranges.stream()
    .filter(r -> r.getOffset() >= 0 && r.getOffset() < len)
    .collect(Collectors.toList());
// submit only safe ranges to readVectored

static boolean isRangeStartable(FileRange r, long fileLen) {
  return r.getOffset() >= 0 && r.getOffset() < fileLen;
}

try {
  in.readVectored(ranges);
} catch (EOFException e) {
  // restat and rebuild: the length used to build ranges is stale
  long fresh = fs.getFileStatus(path).getLen();
  ranges = clampRanges(ranges, fresh);
}

When it happens

Trigger: Calling readVectored or readFullyVectored with a FileRange whose offset >= file length, e.g. new FileRange(fileLen, 100) or offset exactly equal to fileLen expecting an empty result. Higher-level readers (Parquet/Spark column readers) hit it when they compute column-chunk ranges from a stale file length.

Common situations: File was truncated or rewritten between getFileStatus and the vectored read; a cached/stale length used to build ranges; passing offset == length expecting a zero-byte read; concurrent writers replacing the file mid-job.

Related errors

• Range extends beyond the file length ({l}): {last}
• Premature EOF from inputStream
• Premature EOF from inputStream after skipping {len-amt} byte
• Attempted to seek or read past the end of the file " + targe
• Attempted to seek or read past the end of the file

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/5be5b949c84b0487. Report an issue: GitHub.