apache/hadoop · error · IOException

Only read ${actualDiffs + 1} diffs out of ${expectedDiffs}

Error message

Only read ${actualDiffs + 1} diffs out of ${expectedDiffs}

What it means

While reading the <count>-declared <dirDiff> children of a <dirDiffEntry>, expectTag("dirDiff") failed before the declared number was consumed, and the cause is re-wrapped with this message. Note an off-by-one quirk: the message prints actualDiffs+1, i.e. one more than the number of diffs successfully read, because it counts the failed iteration. The true reason (wrong tag, premature end event, etc.) is in the chained cause.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageReconstructor.java:1448

      Long inodeId = dirDiffHeader.removeChildLong(
          SNAPSHOT_DIFF_SECTION_INODE_ID);
      if (inodeId == null) {
        throw new IOException("<dirDiffEntry> contained no <inodeId> entry.");
      }
      headerBld.setInodeId(inodeId);
      Integer expectedDiffs = dirDiffHeader.removeChildInt(
          SNAPSHOT_DIFF_SECTION_COUNT);
      if (expectedDiffs == null) {
        throw new IOException("<dirDiffEntry> contained no <count> entry.");
      }
      headerBld.setNumOfDiff(expectedDiffs);
      dirDiffHeader.verifyNoRemainingKeys("dirDiffEntry");
      headerBld.build().writeDelimitedTo(out);
      for (int actualDiffs = 0; actualDiffs < expectedDiffs; actualDiffs++) {
        try {
          expectTag(SNAPSHOT_DIFF_SECTION_DIR_DIFF, false);
        } catch (IOException e) {
          throw new IOException("Only read " + (actualDiffs + 1) +
              " diffs out of " + expectedDiffs, e);
        }
        Node dirDiff = new Node();
        loadNodeChildren(dirDiff, "dirDiff fields");
        FsImageProto.SnapshotDiffSection.DirectoryDiff.Builder bld =
            FsImageProto.SnapshotDiffSection.DirectoryDiff.newBuilder();
        Integer snapshotId = dirDiff.removeChildInt(
            SNAPSHOT_DIFF_SECTION_SNAPSHOT_ID);
        if (snapshotId != null) {
          bld.setSnapshotId(snapshotId);
        }
        Integer childrenSize = dirDiff.removeChildInt(
            SNAPSHOT_DIFF_SECTION_CHILDREN_SIZE);
        if (childrenSize == null) {
          throw new IOException("Expected to find <childrenSize> in " +
              "<dirDiff> section.");
        }
        bld.setIsSnapshotRoot(dirDiff.removeChildBool(

View on GitHub (pinned to 2add963021)

Solutions

1. Read the chained cause to see what the parser hit instead of <dirDiff>
2. Make <count> equal the number of <dirDiff> elements actually present in that <dirDiffEntry> (mind the message's off-by-one when interpreting it)
3. Restore any <dirDiff> entries that were deleted or truncated
4. Pre-validate every <dirDiffEntry> with a streaming count check before reconstructing

Example fix

<!-- before: count says 3, entry holds 2 -->
<dirDiffEntry><inodeId>16386</inodeId><count>3</count>
  <dirDiff>...</dirDiff><dirDiff>...</dirDiff>
</dirDiffEntry>
<!-- after -->
<dirDiffEntry><inodeId>16386</inodeId><count>2</count>
  <dirDiff>...</dirDiff><dirDiff>...</dirDiff>
</dirDiffEntry>

# python: <count> must equal number of <dirDiff> children per dirDiffEntry
# (also guards the follow-on 'Only read N diffs' failure)
import xml.etree.ElementTree as ET

def validate(path):
    for el in ET.iterparse(path, events=('end',)):
        e = el if el.tag == 'dirDiffEntry' else None
        if e is not None:
            c = e.find('count')
            if c is None or int(c.text) != len(e.findall('dirDiff')):
                return False
    return True

# on 'Only read N diffs out of M' inspect the chained cause in stderr,
# correct <count> or restore diffs, delete partial output, re-run

When it happens

Trigger: <count> larger than the actual number of <dirDiff> elements in the entry, or a malformed <dirDiff> that makes the stream reader hit an unexpected event mid-loop; also entries truncated by line-based editing.

Common situations: Hand-edited entries where diffs were deleted without updating <count>; scripts that append <dirDiff> blocks but never recompute the count; copy-paste of diff blocks between entries.

Related errors

• <createdListSize> was {}, but there were {} <created> entrie
• Only read {} diffs out of {}
• Only read ${actualNumSnapshots} <snapshot> entries out of ${
• Got unexpected end tag for ${name}
• SnapshotDiffSection contained unexpected tag ${tagName}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9332a795244de241. Report an issue: GitHub.