{"record":{"id":"817fc6a2838ef595","repo":"apache/hadoop","slug":"end-of-file-reached-before-reading-fully-817fc6","errorCode":null,"errorMessage":"End of file reached before reading fully.","messagePattern":"End of file reached before reading fully\\.","errorType":"exception","errorClass":"EOFException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java","lineNumber":257,"sourceCode":"\n  @Override\n  public void readFully(long position, byte[] buffer, int offset, int length)\n      throws IOException {\n    validatePositionedReadArgs(position, buffer, offset, length);\n    if (length == 0) {\n      return;\n    }\n    final InputStreamAndFileLength fin = openInputStream(position);\n    try {\n      if (fin.length != null && length + position > fin.length) {\n        throw new EOFException(\"The length to read \" + length\n            + \" exceeds the file length \" + fin.length);\n      }\n      int nread = 0;\n      while (nread < length) {\n        int nbytes = fin.in.read(buffer, offset + nread, length - nread);\n        if (nbytes < 0) {\n          throw new EOFException(FSExceptionMessages.EOF_IN_READ_FULLY);\n        }\n        nread += nbytes;\n      }\n    } finally {\n      fin.in.close();\n    }\n  }\n\n  /**\n   * Return the current offset from the start of the file\n   */\n  @Override\n  public long getPos() throws IOException {\n    return currentPos;\n  }\n\n  /**\n   * Seeks a different copy of the data.  Returns true if","sourceCodeStart":239,"sourceCodeEnd":275,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java#L239-L275","documentation":"After the initial length check, ByteRangeInputStream.readFully reads in a loop until all requested bytes are transferred. If the underlying HTTP stream returns -1 before nread reaches length, it throws the shared FSExceptionMessages.EOF_IN_READ_FULLY EOFException. This means the server ended the response body earlier than the number of bytes the caller requested.","triggerScenarios":"readFully(position, buffer, offset, length) passes the initial range check, but the response body returns EOF after fewer than length bytes. Common concrete causes are concurrent truncation of the file, a stale server-reported length, an intermediate proxy truncating the range response, or a DataNode stream that fails during transfer.","commonSituations":"A file is deleted or truncated while a reader is active; a writer exposes an incomplete file before it is committed; an HTTP proxy or load balancer imposes a response-size limit; a DataNode becomes unhealthy during a large positioned read.","solutions":["Re-run getFileStatus(path), compare the new length with position + length, and either clamp the read or fail with a clear application error.","Retry the read once on a fresh connection or another NameNode/DataNode if the length still covers the range and the failure looks transient.","Inspect proxy or DataNode logs for truncated responses or reset connections when the reported file length is stable.","Read only committed files, or coordinate truncation/append with readers, when another process may modify the file concurrently."],"exampleFix":"// before\nin.readFully(pos, buffer, 0, length);\n\n// after\ntry {\n  in.readFully(pos, buffer, 0, length);\n} catch (EOFException e) {\n  long currentLen = fs.getFileStatus(path).getLen();\n  if (pos + length > currentLen) {\n    throw new EOFException(\"File changed: need \" + (pos + length)\n        + \" bytes, current length is \" + currentLen, e);\n  }\n  throw e;\n}","handlingStrategy":"try-catch","validationCode":"FileStatus status = fs.getFileStatus(path);\nif (position + length > status.getLen()) {\n  length = (int) Math.max(0, status.getLen() - position);\n}\nif (length > 0) {\n  in.readFully(position, buffer, offset, length);\n}","typeGuard":null,"tryCatchPattern":"try {\n  in.readFully(position, buffer, offset, length);\n} catch (EOFException e) {\n  FileStatus latest = fs.getFileStatus(path);\n  if (position + length <= latest.getLen()) {\n    // Length still covers the range: likely a truncated HTTP response or transient DN failure.\n    LOG.warn(\"Premature EOF for {}; retrying\", path, e);\n    return readWithRetry(fs, path, position, buffer, offset, length);\n  }\n  throw e;\n}","preventionTips":["Do not modify files while readers hold positioned ranges unless readers revalidate length after EOF.","Monitor DataNode and proxy connection-reset metrics when these errors recur on stable files.","Keep a single fresh FileStatus close in time to the read and never cache it across long jobs with mutable files."],"tags":["java","hadoop","webhdfs","eof","read-fully"],"backgroundTag":"premature-eof","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}