{"record":{"id":"3acf1ad744ea239d","repo":"apache/hadoop","slug":"transaction-is-corrupt-calculated-checksum-is-ca","errorCode":null,"errorMessage":"Transaction is corrupt. Calculated checksum is {calculatedChecksum} but read checksum {expectedChecksum}","messagePattern":"Transaction is corrupt\\. Calculated checksum is (.+?) but read checksum (.+?)","errorType":"exception","errorClass":"org.apache.hadoop.fs.ChecksumException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java","lineNumber":5269,"sourceCode":"      } else  if (opLength < MIN_OP_LENGTH) {\n        throw new IOException(\"Op \" + (int)opCodeByte + \" has size \" +\n            opLength + \", but the minimum op size is \" + MIN_OP_LENGTH);\n      }\n      long txid = in.readLong();\n      // Verify checksum\n      in.reset();\n      in.mark(maxOpSize);\n      checksum.reset();\n      for (int rem = opLength - CHECKSUM_LENGTH; rem > 0;) {\n        int toRead = Math.min(temp.length, rem);\n        IOUtils.readFully(in, temp, 0, toRead);\n        checksum.update(temp, 0, toRead);\n        rem -= toRead;\n      }\n      int expectedChecksum = in.readInt();\n      int calculatedChecksum = (int)checksum.getValue();\n      if (expectedChecksum != calculatedChecksum) {\n        throw new ChecksumException(\n            \"Transaction is corrupt. Calculated checksum is \" +\n            calculatedChecksum + \" but read checksum \" +\n            expectedChecksum, txid);\n      }\n      return txid;\n    }\n  }\n\n  /**\n   * Read edit logs which have a checksum and a transaction ID, but not a\n   * length.\n   */\n  private static class ChecksummedReader extends Reader {\n    private final Checksum checksum;\n\n    ChecksummedReader(Checksum checksum, DataInputStream in,\n                      StreamLimiter limiter, int logVersion) {\n      super(new DataInputStream(","sourceCodeStart":5251,"sourceCodeEnd":5287,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java#L5251-L5287","documentation":"The definitive corruption signal for modern edits segments: LengthPrefixedReader CRC32s the opLength-minus-4 record bytes and compares against the stored 4-byte checksum int; a mismatch throws ChecksumException whose position field carries the record's txid. Data corruption here means the bytes on disk no longer match what was written.","triggerScenarios":"Bit rot on the journal/name disk; torn writes from a crash before fsync; a truncated tail whose length field still parsed; segments damaged in transfer to a standby or between JournalNodes.","commonSituations":"Failing disks, journal dirs on unreliable storage, crashes mid-transaction, network corruption of segment transfers, tampered segment files.","solutions":["Note the txid from the exception, then run 'hdfs namenode -recover' to skip forward from that transaction and checkpoint immediately","Pull an uncorrupted copy of the same segment from the QJM majority / SecondaryNameNode and replace the bad file instead of skipping","Roll back to the previous fsimage checkpoint if skipping would lose needed transactions","Scrub the storage (smartctl, memory test) before trusting the node with metadata again"],"exampleFix":"# before\nhdfs namenode   # ChecksumException: Transaction is corrupt. Calculated checksum is ... but read checksum ...\n# after\nhdfs namenode -recover   # skip from the reported txid, then: hdfs fsck / && hdfs dfsadmin -saveNamespace","handlingStrategy":"try-catch","validationCode":"# cheap full-CRC sweep before NameNode start / standby bootstrap\nfor f in /dfs/name/current/edits_*; do hdfs offlineEditsViewer -i \"$f\" -o /dev/null || echo \"BAD: $f\"; done","typeGuard":null,"tryCatchPattern":"try {\n  FSEditLogOp op = reader.readOp();\n} catch (org.apache.hadoop.fs.ChecksumException ce) {\n  long corruptTxid = ce.getPos(); // ChecksumException carries the txid here\n  LOG.error(\"CRC mismatch at txid \" + corruptTxid + \" in \" + segment, ce);\n  // prefer restoring a clean QJM copy; else 'hdfs namenode -recover' to skip from corruptTxid\n}","preventionTips":["Run 3+ JournalNodes so CRC-corrupt copies are outvoted by healthy ones","Keep journal/name storage on ECC-memory, healthy-disk nodes; act on media errors","Checkpoint regularly to bound how much replay depends on any one segment","Never ignore a first ChecksumException — it usually marks a failing device"],"tags":["hdfs","edit-log","corruption","checksum","crc32"],"backgroundTag":"checksum-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}