{"record":{"id":"aa6698a147eb127d","repo":"apache/hadoop","slug":"expected-to-parse-in-parallel-but-parsed-t","errorCode":null,"errorMessage":"Expected to parse {} in parallel, but parsed {}. The image may be corrupt.","messagePattern":"Expected to parse (.+?) in parallel, but parsed (.+?)\\. The image may be corrupt\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageTextWriter.java","lineNumber":775,"sourceCode":"        }\n      });\n    }\n\n    try {\n      latch.await();\n    } catch (InterruptedException e) {\n      LOG.error(\"Interrupted waiting for countdown latch\", e);\n      throw new IOException(e);\n    }\n\n    executorService.shutdown();\n    if (exceptions.size() != 0) {\n      LOG.error(\"Failed to output INode sub-sections, {} exception(s) occurred.\",\n          exceptions.size());\n      throw exceptions.get(0);\n    }\n    if (totalParsed.get() != expectedINodes.get()) {\n      throw new IOException(\"Expected to parse \" + expectedINodes + \" in parallel, \" +\n          \"but parsed \" + totalParsed.get() + \". The image may be corrupt.\");\n    }\n    LOG.info(\"Completed outputting all INode sub-sections to {} tmp files.\",\n        subSections.size());\n\n    try (PrintStream ps = new PrintStream(parallelOutputFile, \"UTF-8\")) {\n      ps.println(getHeader());\n    }\n\n    // merge tmp files\n    long startTime = Time.monotonicNow();\n    mergeFiles(paths, parallelOutputFile);\n    long timeTaken = Time.monotonicNow() - startTime;\n    LOG.info(\"Completed all stages. Time to merge files: {} ms\", timeTaken);\n  }\n\n  protected PermissionStatus getPermission(long perm) {\n    return FSImageFormatPBINode.Loader.loadPermission(perm, stringTable);","sourceCodeStart":757,"sourceCodeEnd":793,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageTextWriter.java#L757-L793","documentation":"With `-m/--multiThread N`, the Delimited processor parses INODE sub-sections on a thread pool and counts every inode parsed; the expected total comes from the section itself. When all workers finish without exceptions but the totals differ, the INODE record stream ended before the declared count — the classic signature of a truncated or bit-corrupted fsimage, which oiv reports rather than emit a partial listing.","triggerScenarios":"A corrupt or truncated fsimage where protobuf records end before the declared inode count (incomplete copy, interrupted transfer, bad disk block), or a section whose declared length/count is internally inconsistent.","commonSituations":"Fetching fsimage over flaky networks or NFS without md5 verification; disk errors on the NameNode data dir; images mangled by text-mode (CRLF) transfers.","solutions":["Verify integrity first: in the directory holding image and sidecar run `md5sum -c fsimage_<txid>.md5`; re-fetch from the NameNode on mismatch","Rerun with `-m 1` (serial mode) to get the exact protobuf parse failure point instead of the count mismatch","If the md5 passes and serial parsing also fails, the image is internally inconsistent — open a HADOOP JIRA attaching the fsimage"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"# verify integrity before running oiv, in the dir holding image + md5 sidecar\nmd5sum -c fsimage_0000000000000123456.md5","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Verify fsimage.md5 after every copy or download, before analysis","Transfer images with binary-safe tools (scp, rsync -p) over stable links","On count-mismatch errors, rerun serially (-m 1) to expose the true parse error"],"tags":["hdfs","offline-image-viewer","fsimage","data-corruption","multithreaded"],"backgroundTag":"data-corruption-detected","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}