{"record":{"id":"e129510b1cb82191","repo":"apache/hadoop","slug":"error-skipping-over-blocks","errorCode":null,"errorMessage":"Error skipping over blocks","messagePattern":"Error skipping over blocks","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/ImageLoaderCurrent.java","lineNumber":384,"sourceCode":"   * @param in Datastream to process\n   * @param v Visitor to walk over inodes\n   * @param skipBlocks Walk over each block?\n   */\n  private void processBlocks(DataInputStream in, ImageVisitor v,\n      int numBlocks, boolean skipBlocks) throws IOException {\n    v.visitEnclosingElement(ImageElement.BLOCKS,\n                            ImageElement.NUM_BLOCKS, numBlocks);\n    \n    // directory or symlink or reference node, no blocks to process    \n    if(numBlocks < 0) { \n      v.leaveEnclosingElement(); // Blocks\n      return;\n    }\n    \n    if(skipBlocks) {\n      int bytesToSkip = ((Long.SIZE * 3 /* fields */) / 8 /*bits*/) * numBlocks;\n      if(in.skipBytes(bytesToSkip) != bytesToSkip)\n        throw new IOException(\"Error skipping over blocks\");\n      \n    } else {\n      for(int j = 0; j < numBlocks; j++) {\n        v.visitEnclosingElement(ImageElement.BLOCK);\n        v.visit(ImageElement.BLOCK_ID, in.readLong());\n        v.visit(ImageElement.NUM_BYTES, in.readLong());\n        v.visit(ImageElement.GENERATION_STAMP, in.readLong());\n        v.leaveEnclosingElement(); // Block\n      }\n    }\n    v.leaveEnclosingElement(); // Blocks\n  }\n\n  /**\n   * Extract the INode permissions stored in the fsimage file.\n   *\n   * @param in Datastream to process\n   * @param v Visitor to walk over inodes","sourceCodeStart":366,"sourceCodeEnd":402,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/ImageLoaderCurrent.java#L366-L402","documentation":"With the -skipBlocks option, oiv_legacy avoids parsing each block by calling in.skipBytes(bytesToSkip) where bytesToSkip = 24 * numBlocks (three longs per block: blockId, numBytes, generation stamp - exactly the fields the non-skipping loop below the throw reads). DataInputStream.skipBytes returns fewer bytes than requested only when it hits EOF first, so this IOException means the stream ended before all declared block data could be consumed: the fsimage is truncated or internally inconsistent.","triggerScenarios":"`hdfs oiv_legacy -skipBlocks -p FileDistribution|Indented -i <fsimage>` on a file cut short mid-transfer (interrupted scp/rsync, partial NFS read, disk-full during copy), or an image whose header is corrupt so numBlocks is a garbage-huge value; in both cases skipBytes cannot return bytesToSkip.","commonSituations":"Copying fsimages off a NameNode while the checkpoint is still being written; archives that stored only the first N MB; images damaged in transit; occasionally a layout mismatch making the reader misinterpret a block count.","solutions":["Verify integrity against the sibling checksum: md5sum -c fsimage_XXX.md5, and re-fetch the fsimage + .md5 pair from the NameNode if it fails","Fetch a guaranteed-complete image: hdfs dfsadmin -fs <nn> -fetchImage /tmp/fsimage_XXX after a successful checkpoint","Re-run without -skipBlocks to confirm the failure is positional (a full parse of a truncated file fails with a different EOF error)","Check transfer logs and disk space on every hop the image took"],"exampleFix":"# before\nhdfs oiv_legacy -skipBlocks -p FileDistribution -i fsimage_truncated -o dist.txt\n# IOException: Error skipping over blocks\n\n# after\nmd5sum -c fsimage_0000000000000060000.md5   # fails -> re-fetch a complete image\nhdfs dfsadmin -fs nn1.example.com:8020 -fetchImage /tmp/fsimage_0000000000000060000\nhdfs oiv_legacy -skipBlocks -p FileDistribution -i /tmp/fsimage_0000000000000060000 -o dist.txt","handlingStrategy":"try-catch","validationCode":"# Verify the image is complete before any -skipBlocks run\nmd5sum -c fsimage_0000000000000060000.md5 || {\n  echo \"fsimage truncated/corrupt - re-fetch with: hdfs dfsadmin -fetchImage\"; exit 1; }\nhdfs oiv_legacy -skipBlocks -p FileDistribution -i fsimage_0000000000000060000 -o dist.txt","typeGuard":null,"tryCatchPattern":"try (DataInputStream in = new DataInputStream(\n        new BufferedInputStream(new FileInputStream(image)))) {\n  loader.loadImage(in, visitor, /*skipBlocks*/ true);\n} catch (IOException e) {\n  if (e.getMessage().equals(\"Error skipping over blocks\")) {\n    // stream ended mid-image: treat as corrupt/truncated input, re-fetch the file\n    throw new IOException(\"fsimage appears truncated - verify .md5 and re-fetch\", e);\n  }\n  throw e;\n}","preventionTips":["Always move fsimage together with its .md5 sibling and verify before processing","Fetch finalized images via `hdfs dfsadmin -fetchImage` instead of scp'ing files that may still be written","Never parse the newest checkpoint while saveNamespace is in progress"],"tags":["hdfs","fsimage","truncated-file","skipblocks","offline-image-viewer"],"backgroundTag":"truncated-input-file","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}