{"record":{"id":"ea850a33ee130f63","repo":"apache/hadoop","slug":"got-eof-but-currentpos-currentpos-filelengt","errorCode":null,"errorMessage":"Got EOF but currentPos = ${currentPos} < filelength = ${fileLength}","messagePattern":"Got EOF but currentPos = (.+?) < filelength = (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java","lineNumber":194,"sourceCode":"    final List<String> values = headers.get(key);\n    if (values != null) {\n      for(String v : values) {\n        for(final StringTokenizer t = new StringTokenizer(v, \",\");\n            t.hasMoreTokens(); ) {\n          if (value.equalsIgnoreCase(t.nextToken())) {\n            return true;\n          }\n        }\n      }\n    }\n    return false;\n  }\n\n  private int update(final int n) throws IOException {\n    if (n != -1) {\n      currentPos += n;\n    } else if (fileLength != null && currentPos < fileLength) {\n      throw new IOException(\"Got EOF but currentPos = \" + currentPos\n          + \" < filelength = \" + fileLength);\n    }\n    return n;\n  }\n\n  @Override\n  public int read() throws IOException {\n    final int b = getInputStream().read();\n    update((b == -1) ? -1 : 1);\n    return b;\n  }\n\n  @Override\n  public int read(@Nonnull byte b[], int off, int len) throws IOException {\n    return update(getInputStream().read(b, off, len));\n  }\n\n  /**","sourceCodeStart":176,"sourceCodeEnd":212,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/ByteRangeInputStream.java#L176-L212","documentation":"ByteRangeInputStream derives fileLength from the HTTP response (Content-Length = startOffset + stream length) and tracks currentPos in update(). When a read returns -1 (EOF) while currentPos is still below fileLength, the server delivered fewer bytes than it promised and this IOException is thrown. It indicates a truncated HTTP body — the connection ended mid-stream — rather than a clean end of file.","triggerScenarios":"Reading a WebHDFS file when the HTTP connection is cut before all promised bytes arrive: network drop between client and DataNode/NameNode, proxy or firewall idle/read timeout killing the connection mid-transfer, or a DataNode dying mid-read. seek() then read() makes it visible immediately on the new ranged request.","commonSituations":"Large file reads over WebHDFS through firewalls/LBs with aggressive timeouts; flaky networks to remote clusters; DataNodes restarting or being decommissioned during the read.","solutions":["Retry by reopening the stream and resuming from getPos() — ranged HTTP makes resumption cheap","Raise idle/read timeouts on proxies and load balancers in the path (they cut long transfers first)","If reproducible at the same offset on the same file, run `hdfs fsck <path>` to check for a corrupted/under-replicated block on the serving DataNode","Prefer native hdfs:// client over webhdfs:// for large reads when network reliability is poor"],"exampleFix":"// resume pattern after premature EOF\nlong pos = in.getPos();\ntry {\n  while (in.read(buf, 0, buf.length) != -1) { /* consume */ }\n} catch (IOException e) { // 'Got EOF but currentPos < filelength'\n  in.close();\n  in = fs.open(path);\n  in.seek(pos);\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"long pos = in.getPos();\ntry {\n  IOUtils.copyFully(in, out); // conceptual full read\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"Got EOF but currentPos\")) {\n    in.close();\n    try (FSDataInputStream resume = fs.open(path)) {\n      resume.seek(pos);\n      IOUtils.copyFully(resume, out); // continue from last known position\n    }\n  } else { throw e; }\n}","preventionTips":["For large WebHDFS reads, checkpoint progress (getPos) so truncated transfers can resume instead of restarting","Raise proxy/LB read timeouts to exceed large transfer durations","Check DN health and `hdfs fsck` when truncation repeats at the same offset"],"tags":["hdfs","webhdfs","http","truncated-response","network"],"backgroundTag":"premature-eof","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}