apache/hadoop · error · IOException

Unrecognized FSImage

Error message

Unrecognized FSImage

What it means

FSImageLoader.load() validates its input with FSImageUtil.checkFileFormat(): the file must be at least MINIMUM_FILE_LENGTH bytes and start with the 8-byte magic HDFSIMG1. When either check fails it throws IOException("Unrecognized FSImage") — the file is simply not a protobuf-format fsimage. This loader backs the oiv web server and the XML/lsInteractive-style processors.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FSImageLoader.java:116

  private FSImageLoader(SerialNumberManager.StringTable stringTable,
                        byte[][] inodes, Map<Long, long[]> dirmap) {
    this.stringTable = stringTable;
    this.inodes = inodes;
    this.dirmap = dirmap;
  }

  /**
   * Load fsimage into the memory.
   * @param inputFile the filepath of the fsimage to load.
   * @return FSImageLoader
   * @throws IOException if failed to load fsimage.
   */
  static FSImageLoader load(String inputFile) throws IOException {
    Configuration conf = new Configuration();
    RandomAccessFile file = new RandomAccessFile(inputFile, "r");
    if (!FSImageUtil.checkFileFormat(file)) {
      throw new IOException("Unrecognized FSImage");
    }

    FsImageProto.FileSummary summary = FSImageUtil.loadSummary(file);


    try (FileInputStream fin = new FileInputStream(file.getFD())) {
      // Map to record INodeReference to the referred id
      ImmutableList<Long> refIdList = null;
      SerialNumberManager.StringTable stringTable = null;
      byte[][] inodes = null;
      Map<Long, long[]> dirmap = null;

      ArrayList<FsImageProto.FileSummary.Section> sections =
          Lists.newArrayList(summary.getSectionsList());
      Collections.sort(sections,
          new Comparator<FsImageProto.FileSummary.Section>() {
            @Override
            public int compare(FsImageProto.FileSummary.Section s1,

View on GitHub (pinned to 2add963021)

Solutions

1. Point -i at the fsimage_XXXXXXXXNNNNNNNN file itself — not the .md5, not edits_*.
2. Verify the header: head -c 8 <file> must print HDFSIMG1.
3. For Hadoop 1.x images, use hdfs oiv_legacy or produce a protobuf image via a 2.x+ NameNode first.
4. If the file was copied, compare sizes/checksums with the source to rule out truncation.

Example fix

# before
hdfs oiv -i /nn/current/fsimage_0000000000000021521.md5 -p XML -o out.xml
# -> Unrecognized FSImage

# after
hdfs oiv -i /nn/current/fsimage_0000000000000021521 -p XML -o out.xml

import java.nio.file.*;

static boolean isProtoBufFsImage(Path p) throws IOException {
  if (Files.size(p) < 64) return false;
  try (InputStream in = Files.newInputStream(p)) {
    byte[] magic = new byte[8];
    if (in.read(magic) != 8) return false;
    return new String(magic, StandardCharsets.UTF_8).equals("HDFSIMG1");
  }
}

try {
  FSImageLoader.load(inputFile);
} catch (IOException e) {
  if ("Unrecognized FSImage".equals(e.getMessage())) {
    // wrong or corrupt file: check for .md5, edits_*, truncation, legacy format
    throw new IOException("Not a protobuf fsimage: " + inputFile, e);
  }
  throw e;
}

When it happens

Trigger: hdfs oiv (or the -w web server) pointed at an fsimage.md5 checksum sibling, an edit log (edits_inprogress_...), an empty or truncated file, a pre-protobuf Hadoop 1.x fsimage, or any unrelated file.

Common situations: Globbing the NameNode storage dir and picking the wrong fsimage_* sibling; images truncated during transfer; running a modern oiv against an old cluster's legacy-format images.

Related errors

• Unrecognized FSImage
• Unrecognized section {s.getName()}
• Unrecognized FSImage
• Unrecognized FSImage
• Too many distribution intervals {numIntervals}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3705d7888d081a25. Report an issue: GitHub.