{"record":{"id":"b9d3dfb597b1c6cd","repo":"apache/hadoop","slug":"fsimage-xml-ended-prematurely-without-including-s","errorCode":null,"errorMessage":"FSImage XML ended prematurely, without including section(s) {}","messagePattern":"FSImage XML ended prematurely, without including section\\(s\\) (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageReconstructor.java","lineNumber":1771,"sourceCode":"    LOG.debug(\"Loading <fsimage>.\");\n    expectTag(\"fsimage\", false);\n    // Read the <version> tag.\n    readVersion();\n    // Write the HDFSIMG1 magic number which begins the fsimage file.\n    out.write(FSImageUtil.MAGIC_HEADER);\n    // Write a series of fsimage sections.\n    sectionStartOffset = FSImageUtil.MAGIC_HEADER.length;\n    final HashSet<String> unprocessedSections =\n        new HashSet<>(sections.keySet());\n    while (!unprocessedSections.isEmpty()) {\n      XMLEvent ev = expectTag(\"[section header]\", true);\n      if (ev.getEventType() == XMLStreamConstants.END_ELEMENT) {\n        if (ev.asEndElement().getName().getLocalPart().equals(\"fsimage\")) {\n          if(unprocessedSections.size() == 1 && unprocessedSections.contains\n                  (SnapshotDiffSectionProcessor.NAME)){\n            break;\n          }\n          throw new IOException(\"FSImage XML ended prematurely, without \" +\n              \"including section(s) \" + StringUtils.join(\", \",\n              unprocessedSections));\n        }\n        throw new IOException(\"Got unexpected tag end event for \" +\n            ev.asEndElement().getName().getLocalPart() + \" while looking \" +\n            \"for section header tag.\");\n      } else if (ev.getEventType() != XMLStreamConstants.START_ELEMENT) {\n        throw new IOException(\"Expected section header START_ELEMENT; \" +\n            \"got event of type \" + ev.getEventType());\n      }\n      String sectionName = ev.asStartElement().getName().getLocalPart();\n      if (!unprocessedSections.contains(sectionName)) {\n        throw new IOException(\"Unknown or duplicate section found for \" +\n            sectionName);\n      }\n      SectionProcessor sectionProcessor = sections.get(sectionName);\n      if (sectionProcessor == null) {\n        throw new IOException(\"Unknown FSImage section \" + sectionName +","sourceCodeStart":1753,"sourceCodeEnd":1789,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageReconstructor.java#L1753-L1789","documentation":"OfflineImageReconstructor (the `hdfs oiv -p ReverseXML` processor) rebuilds a binary fsimage from the XML that `hdfs oiv -p XML` emits. It walks the top-level children of <fsimage> expecting exactly one element per registered section (SNAPSHOT_DIFF is the only optional one). If it reaches the closing </fsimage> tag while some expected sections were never seen, it aborts and lists the missing sections rather than write an incomplete image.","triggerScenarios":"Running ReverseXML on XML lacking one or more top-level section elements: a truncated copy of the XML file, section elements removed by hand-editing or a splice script, or XML produced by an OIV build whose section set differs from the reconstructor's (version skew).","commonSituations":"Round-tripping OIV XML back to a binary fsimage; pipelines that transform the XML between generation and reconstruction; partial file transfers; older OIV output fed to a newer oiv that expects a section the old output lacks.","solutions":["Regenerate the XML from the original fsimage with `hdfs oiv -p XML -i <fsimage> -o image.xml` and pass it unmodified to `hdfs oiv -p ReverseXML`","Identify what is missing: compare the <Name> entries inside the <SECTIONS> element against the top-level child elements of <fsimage> — the exception message already lists the missing section names","If the XML was copied or piped, re-transfer and verify size/checksum; truncation is the most common cause","Keep the OIV that produced the XML and the ReverseXML consumer on the same Hadoop release so the expected section sets match"],"exampleFix":"# before: reconstruct a truncated/edited XML\nhdfs oiv -p ReverseXML -i fsimage.xml -o fsimage.out\n# IOException: FSImage XML ended prematurely, without including section(s) INODE\n\n# after: verify every declared section exists at top level first\npython3 -c \"import xml.etree.ElementTree as ET; r = ET.parse('fsimage.xml').getroot(); p = {c.tag for c in r}; d = {n.text for n in r.iter('Name')}; m = d - p - {'SNAPSHOT_DIFF'}; assert not m, f'missing sections: {sorted(m)}'\"\nhdfs oiv -p ReverseXML -i fsimage.xml -o fsimage.out","handlingStrategy":"validation","validationCode":"# before reconstructing, every declared section must exist at top level\nimport xml.etree.ElementTree as ET\nroot = ET.parse('fsimage.xml').getroot()\npresent = {child.tag for child in root}\ndeclared = {n.text for n in root.iter('Name')}\nmissing = declared - present - {'SNAPSHOT_DIFF'}\nif missing:\n    raise SystemExit(f'XML incomplete, missing sections: {sorted(missing)}')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never hand-edit OIV XML between generation and ReverseXML; regenerate it instead","Verify transferred XML with byte size or sha256 before reconstructing","Keep the XML generator and the ReverseXML consumer on the same Hadoop release line"],"tags":["hdfs","offline-image-viewer","fsimage","xml","reversexml"],"backgroundTag":"missing-required-section","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}