apache/hadoop · error · IOException

Unrecognized FSImage

Error message

Unrecognized FSImage

What it means

Identical gate to the text writer's: PBImageXmlWriter.visit() (the `hdfs oiv -p XML` processor) checks the 'HDFSIMG1' magic and minimum file length before parsing. Failure means the input is not a protobuf fsimage — an edit log, the .md5 sidecar, a legacy pre-2.x image, a truncated download, or an unrelated file.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageXmlWriter.java:290

  private final SimpleDateFormat isoDateFormat;
  private SerialNumberManager.StringTable stringTable;

  public static SimpleDateFormat createSimpleDateFormat() {
    SimpleDateFormat format =
      new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS");
    format.setTimeZone(TimeZone.getTimeZone("UTC"));
    return format;
  }

  public PBImageXmlWriter(Configuration conf, PrintStream out) {
    this.conf = conf;
    this.out = out;
    this.isoDateFormat = createSimpleDateFormat();
  }

  public void visit(RandomAccessFile file) throws IOException {
    if (!FSImageUtil.checkFileFormat(file)) {
      throw new IOException("Unrecognized FSImage");
    }

    FileSummary summary = FSImageUtil.loadSummary(file);
    try (FileInputStream fin = new FileInputStream(file.getFD())) {
      out.print("<?xml version=\"1.0\"?>\n<fsimage>");

      out.print("<version>");
      o("layoutVersion", summary.getLayoutVersion());
      o("onDiskVersion", summary.getOndiskVersion());
      // Output the version of OIV (which is not necessarily the version of
      // the fsimage file).  This could be helpful in the case where a bug
      // in OIV leads to information loss in the XML-- we can quickly tell
      // if a specific fsimage XML file is affected by this bug.
      o("oivRevision", VersionInfo.getRevision());
      out.print("</version>\n");

      ArrayList<FileSummary.Section> sections = Lists.newArrayList(summary
          .getSectionsList());

View on GitHub (pinned to 2add963021)

Solutions

1. Check the magic: `head -c 8 <file>` must print HDFSIMG1
2. Pass the explicit fsimage_<txid> path rather than a glob
3. Verify with the sibling fsimage.md5 and re-transfer if it mismatches
4. For pre-Hadoop-2 images use `hdfs oiv_legacy -p XML`

Example fix

# before
hdfs oiv -p XML -i /tmp/maybe-image -o out.xml
# IOException: Unrecognized FSImage

# after
head -c 8 /tmp/maybe.xml >/dev/null; head -c 8 /dfs/name/current/fsimage_0000000000000123456  # HDFSIMG1
hdfs oiv -p XML -i /dfs/name/current/fsimage_0000000000000123456 -o out.xml

// Same gate as the text writer: check magic before PBImageXmlWriter.visit()
try (RandomAccessFile f = new RandomAccessFile(path, "r")) {
  byte[] magic = new byte[FSImageUtil.MAGIC_HEADER.length];
  f.readFully(magic);
  if (!Arrays.equals(magic, FSImageUtil.MAGIC_HEADER)) {
    throw new IllegalArgumentException(path + " is not a protobuf fsimage");
  }
}

When it happens

Trigger: Passing a non-fsimage file to `hdfs oiv -p XML -i <file>`: edits logs, fsimage.md5, legacy-format images, truncated or partially downloaded files.

Common situations: Shell glob picking the wrong file from the NameNode directory; interrupted transfers; legacy images run through the PB tool by mistake.

Related errors

• Unrecognized FSImage
• Unrecognized FSImage
• FSImage XML ended prematurely, without including section(s)
• Got unexpected tag end event for {} while looking for sectio
• Unknown or duplicate section found for {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/76121e469c2ca82a. Report an issue: GitHub.