apache/hadoop · error · IOException

Expected to parse {} in parallel, but parsed {}. The image m

Error message

Expected to parse {} in parallel, but parsed {}. The image may be corrupt.

What it means

With `-m/--multiThread N`, the Delimited processor parses INODE sub-sections on a thread pool and counts every inode parsed; the expected total comes from the section itself. When all workers finish without exceptions but the totals differ, the INODE record stream ended before the declared count — the classic signature of a truncated or bit-corrupted fsimage, which oiv reports rather than emit a partial listing.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageTextWriter.java:775

        }
      });
    }

    try {
      latch.await();
    } catch (InterruptedException e) {
      LOG.error("Interrupted waiting for countdown latch", e);
      throw new IOException(e);
    }

    executorService.shutdown();
    if (exceptions.size() != 0) {
      LOG.error("Failed to output INode sub-sections, {} exception(s) occurred.",
          exceptions.size());
      throw exceptions.get(0);
    }
    if (totalParsed.get() != expectedINodes.get()) {
      throw new IOException("Expected to parse " + expectedINodes + " in parallel, " +
          "but parsed " + totalParsed.get() + ". The image may be corrupt.");
    }
    LOG.info("Completed outputting all INode sub-sections to {} tmp files.",
        subSections.size());

    try (PrintStream ps = new PrintStream(parallelOutputFile, "UTF-8")) {
      ps.println(getHeader());
    }

    // merge tmp files
    long startTime = Time.monotonicNow();
    mergeFiles(paths, parallelOutputFile);
    long timeTaken = Time.monotonicNow() - startTime;
    LOG.info("Completed all stages. Time to merge files: {} ms", timeTaken);
  }

  protected PermissionStatus getPermission(long perm) {
    return FSImageFormatPBINode.Loader.loadPermission(perm, stringTable);

View on GitHub (pinned to 2add963021)

Solutions

1. Verify integrity first: in the directory holding image and sidecar run `md5sum -c fsimage_<txid>.md5`; re-fetch from the NameNode on mismatch
2. Rerun with `-m 1` (serial mode) to get the exact protobuf parse failure point instead of the count mismatch
3. If the md5 passes and serial parsing also fails, the image is internally inconsistent — open a HADOOP JIRA attaching the fsimage

# verify integrity before running oiv, in the dir holding image + md5 sidecar
md5sum -c fsimage_0000000000000123456.md5

When it happens

Trigger: A corrupt or truncated fsimage where protobuf records end before the declared inode count (incomplete copy, interrupted transfer, bad disk block), or a section whose declared length/count is internally inconsistent.

Common situations: Fetching fsimage over flaky networks or NFS without md5 verification; disk errors on the NameNode data dir; images mangled by text-mode (CRLF) transfers.

Related errors

• File " + url + " computed digest " + computedDigest + " does
• serial id ${id} > ${maxEntryNumber}
• Unrecognized FSImage
• Unrecognized section {s.getName()}
• Unrecognized FSImage

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/aa6698a147eb127d. Report an issue: GitHub.