apache/hadoop · error · IOException

FSImage XML ended prematurely, without including section(s)

Error message

FSImage XML ended prematurely, without including section(s) {}

What it means

OfflineImageReconstructor (the `hdfs oiv -p ReverseXML` processor) rebuilds a binary fsimage from the XML that `hdfs oiv -p XML` emits. It walks the top-level children of <fsimage> expecting exactly one element per registered section (SNAPSHOT_DIFF is the only optional one). If it reaches the closing </fsimage> tag while some expected sections were never seen, it aborts and lists the missing sections rather than write an incomplete image.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageReconstructor.java:1771

    LOG.debug("Loading <fsimage>.");
    expectTag("fsimage", false);
    // Read the <version> tag.
    readVersion();
    // Write the HDFSIMG1 magic number which begins the fsimage file.
    out.write(FSImageUtil.MAGIC_HEADER);
    // Write a series of fsimage sections.
    sectionStartOffset = FSImageUtil.MAGIC_HEADER.length;
    final HashSet<String> unprocessedSections =
        new HashSet<>(sections.keySet());
    while (!unprocessedSections.isEmpty()) {
      XMLEvent ev = expectTag("[section header]", true);
      if (ev.getEventType() == XMLStreamConstants.END_ELEMENT) {
        if (ev.asEndElement().getName().getLocalPart().equals("fsimage")) {
          if(unprocessedSections.size() == 1 && unprocessedSections.contains
                  (SnapshotDiffSectionProcessor.NAME)){
            break;
          }
          throw new IOException("FSImage XML ended prematurely, without " +
              "including section(s) " + StringUtils.join(", ",
              unprocessedSections));
        }
        throw new IOException("Got unexpected tag end event for " +
            ev.asEndElement().getName().getLocalPart() + " while looking " +
            "for section header tag.");
      } else if (ev.getEventType() != XMLStreamConstants.START_ELEMENT) {
        throw new IOException("Expected section header START_ELEMENT; " +
            "got event of type " + ev.getEventType());
      }
      String sectionName = ev.asStartElement().getName().getLocalPart();
      if (!unprocessedSections.contains(sectionName)) {
        throw new IOException("Unknown or duplicate section found for " +
            sectionName);
      }
      SectionProcessor sectionProcessor = sections.get(sectionName);
      if (sectionProcessor == null) {
        throw new IOException("Unknown FSImage section " + sectionName +

View on GitHub (pinned to 2add963021)

Solutions

1. Regenerate the XML from the original fsimage with `hdfs oiv -p XML -i <fsimage> -o image.xml` and pass it unmodified to `hdfs oiv -p ReverseXML`
2. Identify what is missing: compare the <Name> entries inside the <SECTIONS> element against the top-level child elements of <fsimage> — the exception message already lists the missing section names
3. If the XML was copied or piped, re-transfer and verify size/checksum; truncation is the most common cause
4. Keep the OIV that produced the XML and the ReverseXML consumer on the same Hadoop release so the expected section sets match

Example fix

# before: reconstruct a truncated/edited XML
hdfs oiv -p ReverseXML -i fsimage.xml -o fsimage.out
# IOException: FSImage XML ended prematurely, without including section(s) INODE

# after: verify every declared section exists at top level first
python3 -c "import xml.etree.ElementTree as ET; r = ET.parse('fsimage.xml').getroot(); p = {c.tag for c in r}; d = {n.text for n in r.iter('Name')}; m = d - p - {'SNAPSHOT_DIFF'}; assert not m, f'missing sections: {sorted(m)}'"
hdfs oiv -p ReverseXML -i fsimage.xml -o fsimage.out

# before reconstructing, every declared section must exist at top level
import xml.etree.ElementTree as ET
root = ET.parse('fsimage.xml').getroot()
present = {child.tag for child in root}
declared = {n.text for n in root.iter('Name')}
missing = declared - present - {'SNAPSHOT_DIFF'}
if missing:
    raise SystemExit(f'XML incomplete, missing sections: {sorted(missing)}')

When it happens

Trigger: Running ReverseXML on XML lacking one or more top-level section elements: a truncated copy of the XML file, section elements removed by hand-editing or a splice script, or XML produced by an OIV build whose section set differs from the reconstructor's (version skew).

Common situations: Round-tripping OIV XML back to a binary fsimage; pipelines that transform the XML between generation and reconstruction; partial file transfers; older OIV output fed to a newer oiv that expects a section the old output lacks.

Related errors

• Expecting {expected}, but got XMLStreamException
• Got unexpected tag end event for {} while looking for sectio
• Unknown or duplicate section found for {}
• Unrecognized FSImage
• no entry found for {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b9d3dfb597b1c6cd. Report an issue: GitHub.