apache/hadoop · error · IOException

<snapshot> section was missing <id>

Error message

<snapshot> section was missing <id>

What it means

ReverseXML found a <snapshot> element inside <SnapshotSection> whose <id> child is absent. The protobuf Snapshot message requires snapshotId, so after loading the element's children into the Node tree, removeChildInt("id") returned null and reconstruction aborts. Note the required tag is exactly <id>, not <snapshotId>.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageReconstructor.java:1382

      }
      header.verifyNoRemainingKeys("SnapshotSection");
      bld.build().writeDelimitedTo(out);
      int actualNumSnapshots = 0;
      while (actualNumSnapshots < expectedNumSnapshots) {
        try {
          expectTag(SNAPSHOT_SECTION_SNAPSHOT, false);
        } catch (IOException e) {
          throw new IOException("Only read " + actualNumSnapshots +
              " <snapshot> entries out of " + expectedNumSnapshots, e);
        }
        actualNumSnapshots++;
        Node snapshot = new Node();
        loadNodeChildren(snapshot, "snapshot fields");
        FsImageProto.SnapshotSection.Snapshot.Builder s =
            FsImageProto.SnapshotSection.Snapshot.newBuilder();
        Integer snapshotId = snapshot.removeChildInt(SECTION_ID);
        if (snapshotId == null) {
          throw new IOException("<snapshot> section was missing <id>");
        }
        s.setSnapshotId(snapshotId);
        Node snapshotRoot = snapshot.removeChild(SNAPSHOT_SECTION_ROOT);
        INodeSection.INode.Builder inodeBld = processINodeXml(snapshotRoot);
        s.setRoot(inodeBld);
        s.build().writeDelimitedTo(out);
      }
      expectTagEnd(SNAPSHOT_SECTION_NAME);
      recordSectionLength(SectionName.SNAPSHOT.name());
    }
  }

  private class SnapshotDiffSectionProcessor implements SectionProcessor {
    static final String NAME = "SnapshotDiffSection";

    @Override
    public void process() throws IOException {
      // No header for this section type.

View on GitHub (pinned to 2add963021)

Solutions

1. Add <id>NUMBER</id> as a child of each affected <snapshot> element, using the snapshot's numeric ID
2. If the tag was renamed (e.g. <snapshotId>), rename it back to exactly <id> - the parser matches the literal tag name
3. Regenerate the XML from a binary fsimage with a matching-version oiv and reapply edits
4. Diff against a known-good dump to catch other renamed/missing fields

Example fix

<!-- before -->
<snapshot><snapshotId>101</snapshotId><root>...</root></snapshot>
<!-- after -->
<snapshot><id>101</id><root>...</root></snapshot>

# python: every <snapshot> must carry an <id> child
import xml.etree.ElementTree as ET

def snapshots_have_ids(path):
    for ev, el in ET.iterparse(path, events=('end',)):
        if el.tag == 'snapshot' and el.find('id') is None:
            return False
    return True

try {
  int rc = new ProcessBuilder("hdfs","oiv","-processor","ReverseXML",
      "-i", xml, "-o", out).inheritIO().start().waitFor();
  if (rc != 0) throw new IllegalStateException("reconstruction failed: " + xml);
} catch (IOException | InterruptedException e) {
  throw new RuntimeException("cannot run oiv", e);
}

When it happens

Trigger: A <snapshot> element with no <id> child - typically deleted or renamed during hand-editing (e.g. changed to <snapshotId> or <sid>), or XML written from scratch by a script that never emitted it.

Common situations: Hand-edited XML where <id> was renamed for readability; synthetic XML authored without studying the oiv XML schema; field-name drift between Hadoop releases' XML writers.

Related errors

• Only read ${actualNumSnapshots} <snapshot> entries out of ${
• <dirDiffEntry> contained no <inodeId> entry.
• <dirDiffEntry> contained no <count> entry.
• Expected to find <childrenSize> in <dirDiff> section.
• Expected to find <createdListSize> in <dirDiff> section.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d6dd66130832e2d2. Report an issue: GitHub.