apache/hadoop · error · EOFException

Premature EOF: pos={pos} < filelength={fileLength}

Error message

Premature EOF: pos={pos} < filelength={fileLength}

What it means

WebHdfsInputStream.read throws EOFException('Premature EOF') when a read over HTTP returns end-of-stream (count < 0) while the tracked position is still below the known file length — the connection delivered fewer bytes than the file (or Content-Length) promised. Causes are a truncated/corrupt replica on the datanode, a datanode or proxy dropping the connection mid-stream, or concurrent truncation of the file. The runner already retries internally, excluding the failing datanode, so this surfaces when the short read persists across attempts.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java:2567

          final URL rurl = new URL(resolvedUrl + "&" + new OffsetParam(pos));
          cachedConnection = new URLRunner(GetOpParam.Op.OPEN, rurl, true,
              false).run();
        } catch (IOException ioe) {
          closeInputStream(RunnerState.DISCONNECTED);
        }
      }

      readBuffer = b;
      readOffset = off;
      readLength = len;

      int count = -1;
      count = this.run();
      if (count >= 0) {
        statistics.incrementBytesRead(count);
        pos += count;
      } else if (pos < fileLength) {
        throw new EOFException(
                  "Premature EOF: pos=" + pos + " < filelength=" + fileLength);
      }
      return count;
    }

    void seek(long newPos) throws IOException {
      if (pos != newPos) {
        pos = newPos;
        closeInputStream(RunnerState.SEEK);
      }
    }

    public void close() throws IOException {
      closeInputStream(RunnerState.CLOSED);
    }

    /* The following methods are overriding AbstractRunner methods,
     * to be called within the retry policy context by runWithRetry.

View on GitHub (pinned to 2add963021)

Solutions

1. Retry the whole operation: reopen the file, seek to the last good offset, and continue — the stream tracks pos so you can resume from a checkpoint
2. If it reproduces at the same offset, run 'hdfs fsck /path -files -blocks -locations' to check for corrupt/under-replicated blocks
3. Check datanode/proxy health and idle timeouts (httpfs, LB, firewall) between client and DN
4. Avoid mutating (truncating/overwriting) files while readers are active; write to a temp path and rename instead
5. For integrity-critical large reads prefer the native hdfs:// client with checksums over webhdfs://

Example fix

// before
try (FSDataInputStream in = fs.open(p)) {
  IOUtils.copyBytes(in, out, 4096, false); // dies mid-stream on EOFException
}
// after: resume from a checkpoint on premature EOF
long off = 0;
while (true) {
  try (FSDataInputStream in = fs.open(p)) {
    in.seek(off);
    off += IOUtils.copyLargeWithCount(in, out); // updates off as bytes flow
    break;
  } catch (EOFException e) {
    if (!retryable(e)) throw e; // give up after N attempts
  }
}

long offset = checkpointOffset;
while (true) {
  try (FSDataInputStream in = fs.open(p)) {
    in.seek(offset);
    offset += drainFrom(in, out); // returns bytes consumed
    break;
  } catch (EOFException e) {
    if (++attempts >= MAX_ATTEMPTS) throw e;
    // built-in datanode exclusion already ran; retry fresh open+seek
  }
}

When it happens

Trigger: Reading from an FSDataInputStream opened via WebHdfsFileSystem (webhdfs:// URL) when the datanode HTTP connection ends before fileLength bytes are consumed — e.g. datanode restart/kill mid-read, replica truncated by recovery, or a load balancer idle/short timeout cutting the stream.

Common situations: Long reads through HttpFS/S3-gateway-style proxies or LBs with aggressive timeouts; reading a file while an append/crash-recovery truncates it; flaky datanodes or disk errors; paths where the built-in exclude-datanode retry exhausts all replicas.

Related errors

• Usernames not matched: expecting null but name={name}
• Usernames not matched: name={shortName} != expected={expecte
• Copy of file ${file} size ${file.length()} into file ${tmpFi
• Failed to move meta file for {b} from {metadataURI} to {dstm
• Failed to move block file for {b} from {blockURI} to {absolu

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b8b656efa3bcde89. Report an issue: GitHub.