apache/hadoop · critical · IOException

Incorrect data format. logVersion is {} but writables.length

Error message

Incorrect data format. logVersion is {} but writables.length is {}. 

What it means

AddCloseOp.readFields validates the legacy length field against the record's own layout version: for old formats (-17 < logVersion) length must be 4, and from -17 onward it must be 5 unless EDITLOG_OP_OPTIMIZATION applies. Any other value means the record shape contradicts its declared version, and readFields throws IOException('Incorrect data format. logVersion is X but writables.length is Y'). Corruption or wrong-version data.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java:615

    @Override
    void readFields(DataInputStream in, int logVersion)
        throws IOException {
      if (!NameNodeLayoutVersion.supports(
          LayoutVersion.Feature.EDITLOG_OP_OPTIMIZATION, logVersion)) {
        this.length = in.readInt();
      }
      if (NameNodeLayoutVersion.supports(
          LayoutVersion.Feature.ADD_INODE_ID, logVersion)) {
        this.inodeId = in.readLong();
      } else {
        // The inodeId should be updated when this editLogOp is applied
        this.inodeId = HdfsConstants.GRANDFATHER_INODE_ID;
      }
      if ((-17 < logVersion && length != 4) ||
          (logVersion <= -17 && length != 5 && !NameNodeLayoutVersion.supports(
              LayoutVersion.Feature.EDITLOG_OP_OPTIMIZATION, logVersion))) {
        throw new IOException("Incorrect data format."  +
                              " logVersion is " + logVersion +
                              " but writables.length is " +
                              length + ". ");
      }
      this.path = FSImageSerialization.readString(in);

      if (NameNodeLayoutVersion.supports(
          LayoutVersion.Feature.EDITLOG_OP_OPTIMIZATION, logVersion)) {
        this.replication = FSImageSerialization.readShort(in);
        this.mtime = FSImageSerialization.readLong(in);
      } else {
        this.replication = readShort(in);
        this.mtime = readLong(in);
      }

      if (NameNodeLayoutVersion.supports(
          LayoutVersion.Feature.FILE_ACCESS_TIME, logVersion)) {
        if (NameNodeLayoutVersion.supports(

View on GitHub (pinned to 2add963021)

Solutions

1. Validate the segment with 'hdfs oev -p xml' to confirm it cannot be parsed
2. Recover with 'hdfs namenode -recover' or restore fsimage plus edits from backup
3. If logs were produced by custom tooling, fix that writer to emit the length field its layout version requires

Example fix

# before: replay fails 'Incorrect data format. logVersion is -32 but writables.length is 7'
hdfs --daemon start namenode

# after: prove the segment is malformed, then recover or restore
hdfs oev -i <segment> -o /tmp/check.xml -p xml   # parser fails at the same record
hdfs namenode -recover

hdfs oev -i <segment> -o /tmp/check.xml -p xml
# a full parse verifies the length field against each record's layout version

try {
  loader.loadFSEdits(storage, 0);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Incorrect data format")) {
    // record shape contradicts its declared logVersion: recover or restore
    // from backup; do not attempt to patch the binary stream by hand
  }
  throw e;
}

When it happens

Trigger: Decoding an OP_ADD or OP_CLOSE record whose serialized shape disagrees with the declared logVersion: corrupted bytes, logs regenerated or patched by external tools, or data written by an incompatible implementation.

Common situations: Corrupt segments after crashes; homegrown tools that rewrite edit logs with wrong length fields.

Related errors

• The layout version {} supports inodeId but gave bogus inodeI
• invalid negative number of blocks
• Error replaying edit log at offset {}. Expected transaction
• Invalid operation read {}
• Mismatched block IDs or generation stamps for the old last b

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/66bca1e339aba334. Report an issue: GitHub.