apache/hadoop · error · IOException

Unknown or duplicate section found for {}

Error message

Unknown or duplicate section found for {}

What it means

ReverseXML read a top-level element whose name is either not a registered fsimage section or was already consumed: each section may occur exactly once, and the unprocessedSections set shrinks as sections are processed, so a second occurrence of the same element (or an element name that is not a section at all) fails this check.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageReconstructor.java:1784

        if (ev.asEndElement().getName().getLocalPart().equals("fsimage")) {
          if(unprocessedSections.size() == 1 && unprocessedSections.contains
                  (SnapshotDiffSectionProcessor.NAME)){
            break;
          }
          throw new IOException("FSImage XML ended prematurely, without " +
              "including section(s) " + StringUtils.join(", ",
              unprocessedSections));
        }
        throw new IOException("Got unexpected tag end event for " +
            ev.asEndElement().getName().getLocalPart() + " while looking " +
            "for section header tag.");
      } else if (ev.getEventType() != XMLStreamConstants.START_ELEMENT) {
        throw new IOException("Expected section header START_ELEMENT; " +
            "got event of type " + ev.getEventType());
      }
      String sectionName = ev.asStartElement().getName().getLocalPart();
      if (!unprocessedSections.contains(sectionName)) {
        throw new IOException("Unknown or duplicate section found for " +
            sectionName);
      }
      SectionProcessor sectionProcessor = sections.get(sectionName);
      if (sectionProcessor == null) {
        throw new IOException("Unknown FSImage section " + sectionName +
            ".  Valid section names are [" +
            StringUtils.join(", ", sections.keySet()) + "]");
      }
      unprocessedSections.remove(sectionName);
      sectionProcessor.process();
    }

    // Write the StringTable section to disk.
    // This has to be done after the other sections, since some of them
    // add entries to the string table.
    writeStringTableSection();

    // Write the FileSummary section to disk.

View on GitHub (pinned to 2add963021)

Solutions

1. Search the XML for the element name in the message; keep exactly one occurrence and delete the duplicate
2. Remove any non-section elements from directly under <fsimage>
3. Regenerate the XML from a single fsimage instead of merging two outputs

# each top-level section may appear exactly once
import xml.etree.ElementTree as ET
from collections import Counter
root = ET.parse('fsimage.xml').getroot()
dupes = [tag for tag, n in Counter(c.tag for c in root).items() if n > 1]
if dupes:
    raise SystemExit(f'duplicate section elements: {dupes}')

When it happens

Trigger: The same section element appearing twice (e.g. two <INODE> blocks from merging two XML files), or an element that is not an fsimage section placed as a direct child of <fsimage>.

Common situations: Concatenating or merging OIV XML outputs from two fsimages; copy-paste duplication while editing; inserting wrapper or annotation elements at top level.

Related errors

• FSImage XML ended prematurely, without including section(s)
• Got unexpected tag end event for {} while looking for sectio
• Unrecognized FSImage
• no entry found for {}
• More than one value found for {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7e86e7b48c4b496d. Report an issue: GitHub.