apache/hadoop · error · IOException

Expecting {expected}, but got XMLStreamException

Error message

Expecting {expected}, but got XMLStreamException

What it means

OfflineImageReconstructor is the ReverseXML processor (`hdfs oiv -p ReverseXML -i img.xml -o fsimage`, dispatched at OfflineImageViewerPB.java:230-232) that converts an OIV XML dump back into a binary protobuf fsimage. expectTag pulls the next StAX XMLEvent; any XMLStreamException there - unclosed tag, bad entity, encoding break - is wrapped in this IOException together with the tag the parser was waiting for. It almost always means the input is not well-formed XML.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageReconstructor.java:194

  /**
   * Read the next tag start or end event.
   *
   * @param expected     The name of the next tag we expect.
   *                     We will validate that the tag has this name,
   *                     unless this string is enclosed in braces.
   * @param allowEnd     If true, we will also end events.
   *                     If false, end events cause an exception.
   *
   * @return             The next tag start or end event.
   */
  private XMLEvent expectTag(String expected, boolean allowEnd)
      throws IOException {
    XMLEvent ev = null;
    while (true) {
      try {
        ev = events.nextEvent();
      } catch (XMLStreamException e) {
        throw new IOException("Expecting " + expected +
            ", but got XMLStreamException", e);
      }
      switch (ev.getEventType()) {
      case XMLEvent.ATTRIBUTE:
        throw new IOException("Got unexpected attribute: " + ev);
      case XMLEvent.CHARACTERS:
        if (!ev.asCharacters().isWhiteSpace()) {
          throw new IOException("Got unxpected characters while " +
              "looking for " + expected + ": " +
              ev.asCharacters().getData());
        }
        break;
      case XMLEvent.END_ELEMENT:
        if (!allowEnd) {
          throw new IOException("Got unexpected end event " +
              "while looking for " + expected);
        }
        return ev;

View on GitHub (pinned to 2add963021)

Solutions

1. Check well-formedness first: xmllint --noout fsimage.xml and regenerate with `hdfs oiv -p XML -i <fsimage> -o img.xml` if it fails
2. Inspect the wrapped XMLStreamException cause - it carries the exact line and column of the break
3. If edits are unavoidable, use an XML-aware editor or DOM library instead of line-oriented tools
4. Re-run the XML and ReverseXML steps from the same Hadoop version pair

Example fix

<!-- before: unclosed root -->
<fsimage><NameSection><namespaceId>1</namespaceId></NameSection>

<!-- after -->
<fsimage><NameSection><namespaceId>1</namespaceId></NameSection></fsimage>

# Reject non-well-formed XML before ReverseXML
tmp=$(mktemp); xmllint --noout fsimage.xml 2> "$tmp" || {
  cat "$tmp"; echo "regenerate: hdfs oiv -p XML -i <fsimage> -o fsimage.xml"; exit 1; }
hdfs oiv -p ReverseXML -i fsimage.xml -o fsimage_rebuilt

try {
  OfflineImageReconstructor.run("fsimage.xml", "fsimage_rebuilt");
} catch (IOException e) {
  System.err.println("XML rejected while expecting: " + e.getMessage());
  Throwable c = e.getCause();
  if (c instanceof XMLStreamException) {
    System.err.println("at line " + ((XMLStreamException) c).getLocation().getLineNumber()
        + " column " + ((XMLStreamException) c).getLocation().getColumnNumber());
  }
}

When it happens

Trigger: ReverseXML on an XML file that was truncated (a `hdfs oiv -p XML` pipe/redirect that broke), hand-edited and left unbalanced, saved with the wrong charset, or mangled by sed/awk transformations between generation and reconstruction.

Common situations: Streaming XML through unbuffered pipes in CI; operators renaming paths/quotas by hand and breaking well-formedness; XML from a different oiv version whose escaping the parser rejects.

Related errors

• Got unexpected end tag for ${name}
• FSImage XML ended prematurely, without including section(s)
• Unrecognized FSImage
• Unrecognized section {s.getName()}
• Unrecognized FSImage

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/df0e08b8273cc6c6. Report an issue: GitHub.