{"record":{"id":"66bca1e339aba334","repo":"apache/hadoop","slug":"incorrect-data-format-logversion-is-but-writab","errorCode":null,"errorMessage":"Incorrect data format. logVersion is {} but writables.length is {}. ","messagePattern":"Incorrect data format\\. logVersion is (.+?) but writables\\.length is (.+?)\\. ","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java","lineNumber":615,"sourceCode":"\n    @Override\n    void readFields(DataInputStream in, int logVersion)\n        throws IOException {\n      if (!NameNodeLayoutVersion.supports(\n          LayoutVersion.Feature.EDITLOG_OP_OPTIMIZATION, logVersion)) {\n        this.length = in.readInt();\n      }\n      if (NameNodeLayoutVersion.supports(\n          LayoutVersion.Feature.ADD_INODE_ID, logVersion)) {\n        this.inodeId = in.readLong();\n      } else {\n        // The inodeId should be updated when this editLogOp is applied\n        this.inodeId = HdfsConstants.GRANDFATHER_INODE_ID;\n      }\n      if ((-17 < logVersion && length != 4) ||\n          (logVersion <= -17 && length != 5 && !NameNodeLayoutVersion.supports(\n              LayoutVersion.Feature.EDITLOG_OP_OPTIMIZATION, logVersion))) {\n        throw new IOException(\"Incorrect data format.\"  +\n                              \" logVersion is \" + logVersion +\n                              \" but writables.length is \" +\n                              length + \". \");\n      }\n      this.path = FSImageSerialization.readString(in);\n\n      if (NameNodeLayoutVersion.supports(\n          LayoutVersion.Feature.EDITLOG_OP_OPTIMIZATION, logVersion)) {\n        this.replication = FSImageSerialization.readShort(in);\n        this.mtime = FSImageSerialization.readLong(in);\n      } else {\n        this.replication = readShort(in);\n        this.mtime = readLong(in);\n      }\n\n      if (NameNodeLayoutVersion.supports(\n          LayoutVersion.Feature.FILE_ACCESS_TIME, logVersion)) {\n        if (NameNodeLayoutVersion.supports(","sourceCodeStart":597,"sourceCodeEnd":633,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java#L597-L633","documentation":"AddCloseOp.readFields validates the legacy length field against the record's own layout version: for old formats (-17 < logVersion) length must be 4, and from -17 onward it must be 5 unless EDITLOG_OP_OPTIMIZATION applies. Any other value means the record shape contradicts its declared version, and readFields throws IOException('Incorrect data format. logVersion is X but writables.length is Y'). Corruption or wrong-version data.","triggerScenarios":"Decoding an OP_ADD or OP_CLOSE record whose serialized shape disagrees with the declared logVersion: corrupted bytes, logs regenerated or patched by external tools, or data written by an incompatible implementation.","commonSituations":"Corrupt segments after crashes; homegrown tools that rewrite edit logs with wrong length fields.","solutions":["Validate the segment with 'hdfs oev -p xml' to confirm it cannot be parsed","Recover with 'hdfs namenode -recover' or restore fsimage plus edits from backup","If logs were produced by custom tooling, fix that writer to emit the length field its layout version requires"],"exampleFix":"# before: replay fails 'Incorrect data format. logVersion is -32 but writables.length is 7'\nhdfs --daemon start namenode\n\n# after: prove the segment is malformed, then recover or restore\nhdfs oev -i <segment> -o /tmp/check.xml -p xml   # parser fails at the same record\nhdfs namenode -recover","handlingStrategy":"try-catch","validationCode":"hdfs oev -i <segment> -o /tmp/check.xml -p xml\n# a full parse verifies the length field against each record's layout version","typeGuard":null,"tryCatchPattern":"try {\n  loader.loadFSEdits(storage, 0);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Incorrect data format\")) {\n    // record shape contradicts its declared logVersion: recover or restore\n    // from backup; do not attempt to patch the binary stream by hand\n  }\n  throw e;\n}","preventionTips":["Use only released tools to inspect or convert edit logs","Checkpoint before risky maintenance to shrink exposure to corrupt segments"],"tags":["hdfs","namenode","edit-log","corruption","layout-version","data-format"],"backgroundTag":"edit-log-corruption","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}