{"record":{"id":"88fa8d643714c0d5","repo":"apache/hadoop","slug":"premature-eof-from-inputstream-88fa8d","errorCode":null,"errorMessage":"Premature EOF from inputStream","messagePattern":"Premature EOF from inputStream","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/BlockReaderUtil.java","lineNumber":52,"sourceCode":"    int n = 0;\n    for (;;) {\n      int nread = reader.read(buf, offset + n, len - n);\n      if (nread <= 0)\n        return (n == 0) ? nread : n;\n      n += nread;\n      if (n >= len)\n        return n;\n    }\n  }\n\n  /* See {@link BlockReader#readFully(byte[], int, int)} */\n  public static void readFully(BlockReader reader,\n      byte[] buf, int off, int len) throws IOException {\n    int toRead = len;\n    while (toRead > 0) {\n      int ret = reader.read(buf, off, toRead);\n      if (ret < 0) {\n        throw new IOException(\"Premature EOF from inputStream\");\n      }\n      toRead -= ret;\n      off += ret;\n    }\n  }\n}\n","sourceCodeStart":34,"sourceCodeEnd":59,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/impl/BlockReaderUtil.java#L34-L59","documentation":"BlockReaderUtil.readFully loops until the requested byte count is filled; a read() returning -1 before then means the block stream ended early — the block has fewer readable bytes than the NameNode's metadata promises. It is the classic truncated-block signature.","triggerScenarios":"Reading a block whose replica on the serving datanode is shorter than the recorded block length: data lost after datanode crashes, under-written or corrupted replicas, or reading files whose writer failed mid-commit.","commonSituations":"Files left inconsistent after writer crashes; failing disks truncating block files; consumers reading files a producer is still writing (length advanced but block not finalized); metadata/block mismatch after a NameNode restore.","solutions":["Run 'hdfs fsck /file -files -blocks -locations' to identify missing or truncated blocks.","Retry after re-replication heals the block (or trigger healing by restarting the affected datanode / using fsck -delete).","For concurrently-written files, read only up to the last finalized length or use a commit marker.","If no replica has the bytes, restore the file from source or snapshot — the data does not exist in HDFS."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"long pos = 0;\nfor (int attempt = 1; attempt <= 3; attempt++) {\n  try (FSDataInputStream in = dfs.open(path)) {\n    in.seek(pos);\n    return readRest(in);\n  } catch (IOException e) {\n    if (attempt == 3 || !isPrematureEof(e)) throw e;\n    // give NN time to re-replicate, then resume from last good position\n    sleepBackoff(attempt);\n    pos = lastGoodPosition();\n  }\n}","preventionTips":["Wrap HDFS reads in position-tracking retry loops so truncated replicas are re-read after re-replication.","For files being written concurrently, bound reads to the last finalized length or a commit marker.","React to premature EOFs with 'hdfs fsck' — they usually mean a replica is physically truncated."],"tags":["hdfs","block-read","truncated-block","eof","replication"],"backgroundTag":"premature-end-of-stream","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}