apache/hadoop · error · IOException

Incorrect data format. Mkdir operation.

Error message

Incorrect data format. Mkdir operation.

What it means

MkdirOp.readFields() enforces a version-dependent field count on legacy records: for logVersion > -17 the length word must be 2, and for logVersion <= -17 (but still without EDITLOG_OP_OPTIMIZATION) it must be 3, because layout -17 added a field to mkdir records. Failing either check means the record is corrupt or is being interpreted with the wrong layout version.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java:1696

      FSImageSerialization.writeLong(timestamp, out); // atime, unused at this
      permissions.write(out);
      AclEditLogUtil.write(aclEntries, out);
      XAttrEditLogProto.Builder b = XAttrEditLogProto.newBuilder();
      b.addAllXAttrs(PBHelperClient.convertXAttrProto(xAttrs));
      b.build().writeDelimitedTo(out);
    }
    
    @Override
    void readFields(DataInputStream in, int logVersion) throws IOException {
      if (!NameNodeLayoutVersion.supports(
          LayoutVersion.Feature.EDITLOG_OP_OPTIMIZATION, logVersion)) {
        this.length = in.readInt();
      }
      if (-17 < logVersion && length != 2 ||
          logVersion <= -17 && length != 3
          && !NameNodeLayoutVersion.supports(
              LayoutVersion.Feature.EDITLOG_OP_OPTIMIZATION, logVersion)) {
        throw new IOException("Incorrect data format. Mkdir operation.");
      }
      if (NameNodeLayoutVersion.supports(
          LayoutVersion.Feature.ADD_INODE_ID, logVersion)) {
        this.inodeId = FSImageSerialization.readLong(in);
      } else {
        // This id should be updated when this editLogOp is applied
        this.inodeId = HdfsConstants.GRANDFATHER_INODE_ID;
      }
      this.path = FSImageSerialization.readString(in);
      if (NameNodeLayoutVersion.supports(
          LayoutVersion.Feature.EDITLOG_OP_OPTIMIZATION, logVersion)) {
        this.timestamp = FSImageSerialization.readLong(in);
      } else {
        this.timestamp = readLong(in);
      }

      // The disk format stores atimes for directories as well.
      // However, currently this is not being updated/used because of

View on GitHub (pinned to 2add963021)

Solutions

1. Run 'hdfs namenode -recover' to skip the record and continue
2. If replaying an archived old cluster, read it with a Hadoop release matching the layout version that wrote it, checkpoint, then upgrade
3. Locate the failing txid with 'hdfs offlineEditsViewer' and restore undamaged copies

hdfs offlineEditsViewer -i <legacy edits segment> -o /dev/null   # use a release matching the segment's layout version

try {
  reader.readOp();
} catch (IOException e) {
  // mkdir field count wrong for the declared layout version:
  // re-read the segment with the matching Hadoop release, then checkpoint and upgrade
}

When it happens

Trigger: A corrupt length word in an old mkdir record; a legacy segment whose layout-version header disagrees with the actual field layout written; misalignment after earlier damage.

Common situations: Edits spanning the Hadoop 0.20-era layout -17 boundary; archives of very old clusters replayed by newer tooling; disk corruption of legacy segments.

Related errors

• The layout version {} supports inodeId but gave bogus inodeI
• Incorrect data format. logVersion is {} but writables.length
• Incorrect data format for ConcatDeleteOp.
• Incorrect data format. ConcatDeleteOp cannot have a negative
• Incorrect data format. Old rename operation.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/147397ad2616d221. Report an issue: GitHub.