{"record":{"id":"5be5b949c84b0487","repo":"apache/hadoop","slug":"range-starts-beyond-the-file-length-l-last","errorCode":null,"errorMessage":"Range starts beyond the file length ({l}): {last}","messagePattern":"Range starts beyond the file length \\((.+?)\\): (.+?)","errorType":"validation","errorClass":"EOFException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/VectoredReadUtils.java","lineNumber":382,"sourceCode":"      FileRange prev = null;\n      for (final FileRange current : sortedRanges) {\n        validateRangeRequest(current);\n        if (prev != null) {\n          checkArgument(current.getOffset() >= prev.getOffset() + prev.getLength(),\n              \"Overlapping ranges %s and %s\", prev, current);\n        }\n        prev = current;\n      }\n    }\n    // at this point the final element in the list is the last range\n    // so make sure it is not beyond the end of the file, if passed in.\n    // where invalid is: starts at or after the end of the file\n    if (fileLength.isPresent()) {\n      final FileRange last = sortedRanges.get(sortedRanges.size() - 1);\n      final Long l = fileLength.get();\n      // this check is superfluous, but it allows for different exception message.\n      if (last.getOffset() >= l) {\n        throw new EOFException(\"Range starts beyond the file length (\" + l + \"): \" + last);\n      }\n      if (last.getOffset() + last.getLength() > l) {\n        throw new EOFException(\"Range extends beyond the file length (\" + l + \"): \" + last);\n      }\n    }\n    return sortedRanges;\n  }\n\n  /**\n   * Sort the input ranges by offset; no validation is done.\n   * @param input input ranges.\n   * @return a new list of the ranges, sorted by offset.\n   */\n  public static List<? extends FileRange> sortRangeList(List<? extends FileRange> input) {\n    final List<? extends FileRange> l = new ArrayList<>(input);\n    l.sort(Comparator.comparingLong(FileRange::getOffset));\n    return l;\n  }","sourceCodeStart":364,"sourceCodeEnd":400,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/VectoredReadUtils.java#L364-L400","documentation":"Thrown by VectoredReadUtils.validateAndSortRanges, the validation pass behind vectored read APIs (FSDataInputStream.readVectored / readFullyVectored). After ranges are sorted by offset, the last range is checked against the known file length; if its offset is at or beyond the length, not a single byte of it can be satisfied and an EOFException is raised. The check only fires when the filesystem supplies the file length (the Optional is present).","triggerScenarios":"Calling readVectored or readFullyVectored with a FileRange whose offset >= file length, e.g. new FileRange(fileLen, 100) or offset exactly equal to fileLen expecting an empty result. Higher-level readers (Parquet/Spark column readers) hit it when they compute column-chunk ranges from a stale file length.","commonSituations":"File was truncated or rewritten between getFileStatus and the vectored read; a cached/stale length used to build ranges; passing offset == length expecting a zero-byte read; concurrent writers replacing the file mid-job.","solutions":["Stat the file (fs.getFileStatus(path).getLen()) and require every range offset to be strictly less than the length before issuing the vectored read","Filter ranges client-side: drop any range with offset >= len instead of submitting it","If the length may be stale, catch EOFException, re-stat the file, rebuild ranges, and retry once","For optional trailing data (e.g. footers), treat offset >= length as 'no data present' and skip the range rather than failing the read"],"exampleFix":"// before\nList<FileRange> ranges = List.of(new FileRange(cachedLen, 128));\nin.readVectored(ranges); // EOFException when cachedLen >= real length\n\n// after\nlong len = fs.getFileStatus(path).getLen();\nList<FileRange> valid = ranges.stream()\n    .filter(r -> r.getOffset() < len)\n    .collect(Collectors.toList());\nif (!valid.isEmpty()) {\n  in.readVectored(valid);\n}","handlingStrategy":"validation","validationCode":"long len = fs.getFileStatus(path).getLen();\nList<FileRange> safe = ranges.stream()\n    .filter(r -> r.getOffset() >= 0 && r.getOffset() < len)\n    .collect(Collectors.toList());\n// submit only safe ranges to readVectored","typeGuard":"static boolean isRangeStartable(FileRange r, long fileLen) {\n  return r.getOffset() >= 0 && r.getOffset() < fileLen;\n}","tryCatchPattern":"try {\n  in.readVectored(ranges);\n} catch (EOFException e) {\n  // restat and rebuild: the length used to build ranges is stale\n  long fresh = fs.getFileStatus(path).getLen();\n  ranges = clampRanges(ranges, fresh);\n}","preventionTips":["Always stat the file in the same critical section that builds the ranges","Never assume a zero-byte result for offset == length — the API treats it as invalid","In long pipelines, re-stat before each vectored read batch if files can be rewritten concurrently"],"tags":["vectored-io","file-range","eof","hadoop-common"],"backgroundTag":"read-past-end-of-file","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}