{"record":{"id":"58d42035fe5ab87c","repo":"apache/hadoop","slug":"range-extends-beyond-the-file-length-l-last","errorCode":null,"errorMessage":"Range extends beyond the file length ({l}): {last}","messagePattern":"Range extends beyond the file length \\((.+?)\\): (.+?)","errorType":"validation","errorClass":"EOFException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/VectoredReadUtils.java","lineNumber":385,"sourceCode":"        if (prev != null) {\n          checkArgument(current.getOffset() >= prev.getOffset() + prev.getLength(),\n              \"Overlapping ranges %s and %s\", prev, current);\n        }\n        prev = current;\n      }\n    }\n    // at this point the final element in the list is the last range\n    // so make sure it is not beyond the end of the file, if passed in.\n    // where invalid is: starts at or after the end of the file\n    if (fileLength.isPresent()) {\n      final FileRange last = sortedRanges.get(sortedRanges.size() - 1);\n      final Long l = fileLength.get();\n      // this check is superfluous, but it allows for different exception message.\n      if (last.getOffset() >= l) {\n        throw new EOFException(\"Range starts beyond the file length (\" + l + \"): \" + last);\n      }\n      if (last.getOffset() + last.getLength() > l) {\n        throw new EOFException(\"Range extends beyond the file length (\" + l + \"): \" + last);\n      }\n    }\n    return sortedRanges;\n  }\n\n  /**\n   * Sort the input ranges by offset; no validation is done.\n   * @param input input ranges.\n   * @return a new list of the ranges, sorted by offset.\n   */\n  public static List<? extends FileRange> sortRangeList(List<? extends FileRange> input) {\n    final List<? extends FileRange> l = new ArrayList<>(input);\n    l.sort(Comparator.comparingLong(FileRange::getOffset));\n    return l;\n  }\n\n  /**\n   * Sort the input ranges by offset; no validation is done.","sourceCodeStart":367,"sourceCodeEnd":403,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/VectoredReadUtils.java#L367-L403","documentation":"Companion check in VectoredReadUtils.validateAndSortRanges: the last range (after sorting) must end at or before the file length. A range that starts inside the file but whose offset+length exceeds the length is rejected with EOFException — vectored reads never silently truncate a range to EOF, unlike plain positioned reads.","triggerScenarios":"readVectored with a range such as new FileRange(len - 10, 100) on a file of length len: it starts before EOF but ends 90 bytes past the end. Also ranges sized from a file length fetched before the file shrank (truncation/rewrite).","commonSituations":"Fixed-size chunk readers that do not clamp the final chunk (fixed-length records, checksum/parity blocks); cached file sizes; files rewritten smaller between status check and read.","solutions":["Clamp each range to EOF: int n = (int) Math.min(range.getLength(), len - range.getOffset()); submit new FileRange(range.getOffset(), n, range.getBackend())","Stat the file immediately before the read and build ranges from the fresh length","If partial trailing data is acceptable, split the range at EOF and read only the in-bounds portion","Catch EOFException, re-stat, clamp, and retry once for races against file truncation"],"exampleFix":"// before\nranges.add(new FileRange(offset, FIXED_CHUNK));\nin.readVectored(ranges); // fails when offset + FIXED_CHUNK > len\n\n// after\nlong len = fs.getFileStatus(path).getLen();\nfor (long offset : offsets) {\n  int n = (int) Math.min(FIXED_CHUNK, len - offset);\n  if (n > 0) {\n    ranges.add(new FileRange(offset, n));\n  }\n}\nin.readVectored(ranges);","handlingStrategy":"validation","validationCode":"long len = fs.getFileStatus(path).getLen();\nList<FileRange> clamped = new ArrayList<>();\nfor (FileRange r : ranges) {\n  long end = Math.min(r.getOffset() + r.getLength(), len);\n  if (r.getOffset() < end) {\n    clamped.add(new FileRange(r.getOffset(), (int) (end - r.getOffset())));\n  }\n}","typeGuard":"static boolean isRangeWithinFile(FileRange r, long fileLen) {\n  return r.getOffset() >= 0 && r.getOffset() + r.getLength() <= fileLen;\n}","tryCatchPattern":"try {\n  in.readVectored(ranges);\n} catch (EOFException e) {\n  long fresh = fs.getFileStatus(path).getLen();\n  in.readVectored(clampRanges(ranges, fresh)); // retry once with clamped ranges\n}","preventionTips":["Clamp the last chunk of any fixed-size chunking scheme to the file length","Do not expect vectored reads to truncate ranges at EOF like positioned reads do","Treat 'file shrank between stat and read' as a retryable race, not a bug"],"tags":["vectored-io","file-range","eof","truncation"],"backgroundTag":"read-past-end-of-file","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}