{"record":{"id":"acdb802687fc8494","repo":"apache/hadoop","slug":"the-length-to-read-length-exceeds-the-file-leng","errorCode":null,"errorMessage":"The length to read ${length} exceeds the file length ${fin.length}","messagePattern":"The length to read (.+?) exceeds the file length (.+?)","errorType":"exception","errorClass":"EOFException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java","lineNumber":250,"sourceCode":"    if (length == 0) {\n      return 0;\n    }\n    try (InputStream in = openInputStream(position).in) {\n      return in.read(buffer, offset, length);\n    }\n  }\n\n  @Override\n  public void readFully(long position, byte[] buffer, int offset, int length)\n      throws IOException {\n    validatePositionedReadArgs(position, buffer, offset, length);\n    if (length == 0) {\n      return;\n    }\n    final InputStreamAndFileLength fin = openInputStream(position);\n    try {\n      if (fin.length != null && length + position > fin.length) {\n        throw new EOFException(\"The length to read \" + length\n            + \" exceeds the file length \" + fin.length);\n      }\n      int nread = 0;\n      while (nread < length) {\n        int nbytes = fin.in.read(buffer, offset + nread, length - nread);\n        if (nbytes < 0) {\n          throw new EOFException(FSExceptionMessages.EOF_IN_READ_FULLY);\n        }\n        nread += nbytes;\n      }\n    } finally {\n      fin.in.close();\n    }\n  }\n\n  /**\n   * Return the current offset from the start of the file\n   */","sourceCodeStart":232,"sourceCodeEnd":268,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java#L232-L268","documentation":"WebHDFS ByteRangeInputStream.readFully(position, buffer, offset, length) first opens an HTTP range stream and obtains the server-reported file length. Before reading, it rejects a request whose position + length exceeds that length, because the full requested byte range cannot exist. This preserves the FSDataInputStream.readFully contract that either all requested bytes are read or an EOFException is thrown.","triggerScenarios":"Calling readFully(pos, buf, off, len) on a stream opened from a webhdfs:// or swebhdfs:// path where pos + len is greater than the file length reported by the current open response. Typical causes are an off-by-one position, a fixed-record read that starts near EOF, or a length obtained from an earlier getFileStatus after the file was truncated or not appended as expected.","commonSituations":"Reading fixed-size records or footer structures from small files; using a stale FileStatus length in a concurrent pipeline; a file truncated by another job between listing and reading; tests that assume a file is longer than the bytes actually uploaded.","solutions":["Call getFileStatus(path) immediately before the positioned read and clamp length to Math.max(0, fileLen - position).","Correct the position arithmetic, especially the final partial record, offset, and boundary conditions at pos == fileLen.","Refresh the file length and retry once when the file may have changed between the status call and the read.","If the file is expected to be appended concurrently, wait until the writer closes it or use a committed/sentinel file before reading the final record."],"exampleFix":"// before\nlong len = oldStatus.getLen();\nin.readFully(pos, buffer, 0, recordSize); // pos + recordSize > len\n\n// after\nlong len = fs.getFileStatus(path).getLen();\nlong available = len - pos;\nif (available <= 0) {\n  throw new EOFException(\"No bytes at position \" + pos + \" of \" + path);\n}\nint toRead = (int) Math.min(recordSize, available);\nin.readFully(pos, buffer, 0, toRead);","handlingStrategy":"validation","validationCode":"FileStatus status = fs.getFileStatus(path);\nlong available = status.getLen() - position;\nif (available <= 0) {\n  throw new EOFException(\"No bytes to read at position \" + position + \" of \" + path);\n}\nint safeLength = (int) Math.min(length, available);\nin.readFully(position, buffer, offset, safeLength);","typeGuard":null,"tryCatchPattern":"try {\n  in.readFully(position, buffer, offset, length);\n} catch (EOFException e) {\n  // Refresh the length and distinguish a normal EOF from an unexpected transport failure.\n  throw new EOFException(\"Requested \" + (position + length) + \" bytes; current file length is \"\n      + fs.getFileStatus(path).getLen(), e);\n}","preventionTips":["Always calculate bytes remaining from a fresh FileStatus before a positioned full read.","Treat the final partial record explicitly instead of assuming fileLen is a multiple of the record size.","Use committed files when another process may append or truncate during a read."],"tags":["java","hadoop","webhdfs","eof","positioned-read"],"backgroundTag":"read-past-end-of-file","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}