{"record":{"id":"b8b656efa3bcde89","repo":"apache/hadoop","slug":"premature-eof-pos-pos-filelength-filelength","errorCode":null,"errorMessage":"Premature EOF: pos={pos} < filelength={fileLength}","messagePattern":"Premature EOF: pos=(.+?) < filelength=(.+?)","errorType":"exception","errorClass":"EOFException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java","lineNumber":2567,"sourceCode":"          final URL rurl = new URL(resolvedUrl + \"&\" + new OffsetParam(pos));\n          cachedConnection = new URLRunner(GetOpParam.Op.OPEN, rurl, true,\n              false).run();\n        } catch (IOException ioe) {\n          closeInputStream(RunnerState.DISCONNECTED);\n        }\n      }\n\n      readBuffer = b;\n      readOffset = off;\n      readLength = len;\n\n      int count = -1;\n      count = this.run();\n      if (count >= 0) {\n        statistics.incrementBytesRead(count);\n        pos += count;\n      } else if (pos < fileLength) {\n        throw new EOFException(\n                  \"Premature EOF: pos=\" + pos + \" < filelength=\" + fileLength);\n      }\n      return count;\n    }\n\n    void seek(long newPos) throws IOException {\n      if (pos != newPos) {\n        pos = newPos;\n        closeInputStream(RunnerState.SEEK);\n      }\n    }\n\n    public void close() throws IOException {\n      closeInputStream(RunnerState.CLOSED);\n    }\n\n    /* The following methods are overriding AbstractRunner methods,\n     * to be called within the retry policy context by runWithRetry.","sourceCodeStart":2549,"sourceCodeEnd":2585,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java#L2549-L2585","documentation":"WebHdfsInputStream.read throws EOFException('Premature EOF') when a read over HTTP returns end-of-stream (count < 0) while the tracked position is still below the known file length — the connection delivered fewer bytes than the file (or Content-Length) promised. Causes are a truncated/corrupt replica on the datanode, a datanode or proxy dropping the connection mid-stream, or concurrent truncation of the file. The runner already retries internally, excluding the failing datanode, so this surfaces when the short read persists across attempts.","triggerScenarios":"Reading from an FSDataInputStream opened via WebHdfsFileSystem (webhdfs:// URL) when the datanode HTTP connection ends before fileLength bytes are consumed — e.g. datanode restart/kill mid-read, replica truncated by recovery, or a load balancer idle/short timeout cutting the stream.","commonSituations":"Long reads through HttpFS/S3-gateway-style proxies or LBs with aggressive timeouts; reading a file while an append/crash-recovery truncates it; flaky datanodes or disk errors; paths where the built-in exclude-datanode retry exhausts all replicas.","solutions":["Retry the whole operation: reopen the file, seek to the last good offset, and continue — the stream tracks pos so you can resume from a checkpoint","If it reproduces at the same offset, run 'hdfs fsck /path -files -blocks -locations' to check for corrupt/under-replicated blocks","Check datanode/proxy health and idle timeouts (httpfs, LB, firewall) between client and DN","Avoid mutating (truncating/overwriting) files while readers are active; write to a temp path and rename instead","For integrity-critical large reads prefer the native hdfs:// client with checksums over webhdfs://"],"exampleFix":"// before\ntry (FSDataInputStream in = fs.open(p)) {\n  IOUtils.copyBytes(in, out, 4096, false); // dies mid-stream on EOFException\n}\n// after: resume from a checkpoint on premature EOF\nlong off = 0;\nwhile (true) {\n  try (FSDataInputStream in = fs.open(p)) {\n    in.seek(off);\n    off += IOUtils.copyLargeWithCount(in, out); // updates off as bytes flow\n    break;\n  } catch (EOFException e) {\n    if (!retryable(e)) throw e; // give up after N attempts\n  }\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"long offset = checkpointOffset;\nwhile (true) {\n  try (FSDataInputStream in = fs.open(p)) {\n    in.seek(offset);\n    offset += drainFrom(in, out); // returns bytes consumed\n    break;\n  } catch (EOFException e) {\n    if (++attempts >= MAX_ATTEMPTS) throw e;\n    // built-in datanode exclusion already ran; retry fresh open+seek\n  }\n}","preventionTips":["Checkpoint read offsets in long jobs so premature EOF is resumable, not fatal","Keep files immutable while readers are active (write-temp + atomic rename)","Prefer native hdfs:// with checksums for integrity-critical reads over webhdfs://","Monitor datanode/gateway health and idle-timeout settings on long transfers"],"tags":["webhdfs","io","premature-eof","datanode","retry"],"backgroundTag":"premature-eof","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}