apache/hadoop · error · IOException

Unexpected EOS from the reader

Error message

Unexpected EOS from the reader

What it means

In StripeReader.readToBuffer, data is read from a BlockReader until strategy's target length is satisfied; a negative return from readFromBlock means end-of-stream before targetLength bytes arrived, which is unexpected because the reader was positioned and sized for exactly this chunk. It signals the DataNode closed the connection early — node crash/restart mid-read, truncated or corrupted chunk, or reader/protobuf state corruption — rather than a clean short read.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/StripeReader.java:243

    for (int i = 0; i < strategies.length; i++) {
      ByteBuffer buffer = chunk.getChunkBuffer().getSlice(i);
      strategies[i] = new ByteBufferStrategy(buffer,
              dfsStripedInputStream.getReadStatistics(),
              dfsStripedInputStream.getDFSClient());
    }
    return strategies;
  }

  private int readToBuffer(BlockReader blockReader,
      DatanodeInfo currentNode, ByteBufferStrategy strategy,
      ExtendedBlock currentBlock) throws IOException {
    final int targetLength = strategy.getTargetLength();
    int length = 0;
    try {
      while (length < targetLength) {
        int ret = strategy.readFromBlock(blockReader);
        if (ret < 0) {
          throw new IOException("Unexpected EOS from the reader");
        }
        length += ret;
      }
      return length;
    } catch (ChecksumException ce) {
      DFSClient.LOG.warn("Found Checksum error for "
          + currentBlock + " from " + currentNode
          + " at " + ce.getPos());
      //Clear buffer to make next decode success
      strategy.getReadBuffer().clear();
      // we want to remember which block replicas we have tried
      corruptedBlocks.addCorruptedBlock(currentBlock, currentNode);
      if (blockReader != null) {
        blockReader.close();
      }
      throw ce;
    } catch (IOException e) {
      DFSClient.LOG.warn("Exception while reading from "

View on GitHub (pinned to 2add963021)

Solutions

1. Retry the read/job — DFSClient striped reads attempt other chunks/replicas, and transient node failures typically clear
2. Check DataNode logs around the failure time for OOM, disk errors, or restarts on the hosts named in preceding warnings
3. Run hdfs fsck on the file to rule out truncated/corrupted chunks if the error persists on the same offset
4. Keep client and DataNode Hadoop versions current — several premature-EOS bugs in striped reads were fixed in later 3.x patch releases

try {
  readFully(in, buf, off, len);
} catch (IOException e) {
  if (e.getMessage().equals("Unexpected EOS from the reader")) {
    // DataNode dropped mid-chunk; reopen, re-locate, and retry the range
    reopenAt(offset); // DFSClient retries other replicas on next read
  } else throw e;
}

When it happens

Trigger: Erasure-coded read where a DataNode serving one chunk dies or resets the connection mid-transfer; block/chunk length on the DataNode shorter than the LocatedStripedBlock metadata claims; transient network device drops during large striped reads.

Common situations: DataNode OOM/restart under load while an EC job reads; network appliances idle-resetting long-lived DataNode connections; rare corruption cases where on-disk chunk size disagrees with block metadata; seen more during heavy parallel scan jobs over EC data.

Related errors

• Unknown BlockChecksumType: " + groupChecksumType
• {} has no enough internal blocks(current: {}), unable to sta
• Unresolved host: {}
• Fail to get block checksum for {}
• {} missing blocks, the stripe is: {}; locatedBlocks is: {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a034c5b481db2d5f. Report an issue: GitHub.