apache/hadoop · error · IOException

<createdListSize> was {}, but there were {} <created> entrie

Error message

<createdListSize> was {}, but there were {} <created> entries.

What it means

After consuming all <created> entries of a <dirDiff>, the reconstructor compared the running count against the declared <createdListSize> and they differ. The protobuf createdListSize must equal the number of CreatedListEntry records actually serialized, so the mismatch is fatal even though the XML is well-formed.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageReconstructor.java:1522

        while (true) {
          Node created = dirDiff.removeChild(
              SNAPSHOT_DIFF_SECTION_CREATED);
          if (created == null){
            break;
          }
          String cleName = created.removeChildStr(SECTION_NAME);
          if (cleName == null) {
            throw new IOException("Expected <created> entry to have " +
                "a <name> field");
          }
          created.verifyNoRemainingKeys("created");
          FsImageProto.SnapshotDiffSection.CreatedListEntry.newBuilder().
              setName(ByteString.copyFrom(cleName, StandardCharsets.UTF_8)).
              build().writeDelimitedTo(out);
          actualCreatedListSize++;
        }
        if (actualCreatedListSize != expectedCreatedListSize) {
          throw new IOException("<createdListSize> was " +
              expectedCreatedListSize +", but there were " +
              actualCreatedListSize + " <created> entries.");
        }
        dirDiff.verifyNoRemainingKeys("dirDiff");
      }
      expectTagEnd(SNAPSHOT_DIFF_SECTION_DIR_DIFF_ENTRY);
    }

    private void processFileDiffEntry() throws IOException {
      LOG.debug("Processing fileDiffEntry");
      DiffEntry.Builder headerBld = DiffEntry.newBuilder();
      headerBld.setType(DiffEntry.Type.FILEDIFF);
      Node fileDiffHeader = new Node();
      loadNodeChildren(fileDiffHeader, "fileDiffEntry fields", "fileDiff");
      Long inodeId = fileDiffHeader.removeChildLong(
          SNAPSHOT_DIFF_SECTION_INODE_ID);
      if (inodeId == null) {
        throw new IOException("<fileDiffEntry> contained no <inodeid> entry.");

View on GitHub (pinned to 2add963021)

Solutions

1. Count the <created> children of the affected <dirDiff> and set <createdListSize> to exactly that number
2. Alternatively add/remove <created> entries until the list matches the declared size - both sides must agree
3. Re-run a streaming pre-check that validates size vs entry count for every <dirDiff>
4. Regenerate the XML from the original fsimage with a same-version oiv

Example fix

<!-- before: size says 2, only 1 entry -->
<dirDiff>...<createdListSize>2</createdListSize>
  <created><name>part-00000</name></created>
</dirDiff>
<!-- after -->
<dirDiff>...<createdListSize>1</createdListSize>
  <created><name>part-00000</name></created>
</dirDiff>

# python: createdListSize must equal the number of <created> children
import xml.etree.ElementTree as ET

def created_lists_consistent(path):
    for ev, el in ET.iterparse(path, events=('end',)):
        if el.tag == 'dirDiff':
            s = el.find('createdListSize')
            if s is not None and int(s.text) != len(el.findall('created')):
                return False
    return True

// catch the oiv count-mismatch message; it prints both numbers -
// reconcile them, delete partial output, re-run

When it happens

Trigger: <created> entries added or deleted without updating <createdListSize>, or vice versa; the declared size edited while the entry list stayed unchanged.

Common situations: Hand-edits that fix disaster-recovery images by dropping created files; scripts appending entries but never recounting; partial copy-paste of created lists between diffs.

Related errors

• Only read ${actualDiffs + 1} diffs out of ${expectedDiffs}
• Only read {} diffs out of {}
• Only read ${actualNumSnapshots} <snapshot> entries out of ${
• Got unexpected end tag for ${name}
• SnapshotDiffSection contained unexpected tag ${tagName}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/cbffa9f0a43574e2. Report an issue: GitHub.