{"record":{"id":"bdd2cc65c9f13b22","repo":"apache/hadoop","slug":"file-did-not-match-stored-md5-checksum-stored","errorCode":null,"errorMessage":"File {} did not match stored MD5 checksum  (stored: {}, computed: {}","messagePattern":"File (.+?) did not match stored MD5 checksum  \\(stored: (.+?), computed: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/MD5FileUtils.java","lineNumber":62,"sourceCode":"public abstract class MD5FileUtils {\n  private static final Logger LOG = LoggerFactory.getLogger(\n      MD5FileUtils.class);\n\n  public static final String MD5_SUFFIX = \".md5\";\n  private static final Pattern LINE_REGEX =\n    Pattern.compile(\"([0-9a-f]{32}) [ \\\\*](.+)\");\n  \n  /**\n   * Verify that the previously saved md5 for the given file matches\n   * expectedMd5.\n   * @throws IOException \n   */\n  public static void verifySavedMD5(File dataFile, MD5Hash expectedMD5)\n      throws IOException {\n    MD5Hash storedHash = readStoredMd5ForFile(dataFile);\n    // Check the hash itself\n    if (!expectedMD5.equals(storedHash)) {\n      throw new IOException(\n          \"File \" + dataFile + \" did not match stored MD5 checksum \" +\n          \" (stored: \" + storedHash + \", computed: \" + expectedMD5);\n    }\n  }\n  \n  /**\n   * Read the md5 file stored alongside the given data file\n   * and match the md5 file content.\n   * @param md5File the file containing md5 data\n   * @return a matcher with two matched groups\n   *   where group(1) is the md5 string and group(2) is the data file path.\n   */\n  private static Matcher readStoredMd5(File md5File) throws IOException {\n    BufferedReader reader =\n        new BufferedReader(new InputStreamReader(\n            Files.newInputStream(md5File.toPath()), StandardCharsets.UTF_8));\n    String md5Line;\n    try {","sourceCodeStart":44,"sourceCodeEnd":80,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/MD5FileUtils.java#L44-L80","documentation":"MD5FileUtils.verifySavedMD5 compares the hash stored in the '.md5' sidecar file against the expected (typically freshly computed) hash of the data file. A mismatch means the file's current bytes are not what they were when the checksum was saved - the canonical signal of corruption, truncation, or a mixed-generation file+sidecar pair. This is how the NameNode detects damaged fsimage/edit files before trusting them.","triggerScenarios":"verifySavedMD5(dataFile, computeMd5ForFile(dataFile)) after the data file changed since saveMD5File ran: truncated or partially downloaded fsimage (get/image transfer interrupted), bit rot on failing disks, interrupted save leaving new bytes with an old sidecar, or a sidecar copied from a different checkpoint generation.","commonSituations":"NameNode failing to load a checkpoint with 'Image file is corrupt'; fetching fsimage via HTTP tools that truncated the transfer; mixed storage directories after a restore; backup scripts copying data file and sidecar from different points in time.","solutions":["Triage both hashes: 'md5sum <datafile>' versus the 32 hex chars inside '<datafile>.md5' - decides whether the data or the sidecar is the stale half.","If the data file is corrupt, restore the file AND its .md5 together from a consistent source: another storage directory, secondary/standby NameNode checkpoint, or backup.","If only the sidecar is stale (file verified good by other means, e.g., it loads with 'hdfs oiv'), regenerate it with MD5FileUtils.saveMD5File / md5sum.","Investigate the root cause: disk SMART errors, truncated transfers (re-fetch with size verification), and re-run hdfs fsck."],"exampleFix":"// before\nMD5FileUtils.verifySavedMD5(imgFile, MD5FileUtils.computeMd5ForFile(imgFile));\n// throws 'did not match stored MD5 checksum' with both hashes buried in the message\n\n// after - explicit triage before failing\nMD5Hash stored = MD5FileUtils.readStoredMd5ForFile(imgFile);\nif (stored != null) {\n  MD5Hash computed = MD5FileUtils.computeMd5ForFile(imgFile);\n  if (!stored.equals(computed)) {\n    throw new IOException(String.format(\n        \"fsimage %s inconsistent (stored=%s, computed=%s): \"\n        + \"restore file and .md5 as a pair from a healthy checkpoint\",\n        imgFile, stored, computed));\n  }\n}","handlingStrategy":"validation","validationCode":"// pre-verify the pair yourself and decide policy before the library throws\nFile md5 = MD5FileUtils.getDigestFileForFile(dataFile);\nif (md5.exists()) {\n  String line = new String(Files.readAllBytes(md5.toPath()), StandardCharsets.UTF_8).split(\"\\\\s+\")[0];\n  String computed = MD5FileUtils.computeMd5ForFile(dataFile).toString();\n  if (!line.equals(computed)) {\n    // decide: restore file+md5 pair from backup, or regenerate sidecar\n    LOG.warn(\"Checksum drift on {}: stored={} computed={}\", dataFile, line, computed);\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  MD5FileUtils.verifySavedMD5(dataFile, expected);\n} catch (IOException e) { // 'did not match stored MD5 checksum'\n  // message carries stored and computed hashes: triage which half is stale,\n  // restore file+sidecar as a consistent pair, never mix generations\n  throw new IOException(\"Integrity failure on \" + dataFile\n      + \" - restore data file and .md5 together: \" + e.getMessage(), e);\n}","preventionTips":["Always move/copy a data file and its .md5 sidecar as an atomic pair.","Verify transfers: compare byte sizes and md5 after downloading fsimages or edit segments.","Monitor disk health (SMART, dmesg I/O errors) in NameNode/JournalNode storage dirs.","Keep a consistent backup of storage directories for restore-with-sidecar scenarios."],"tags":["checksum","md5","corruption","fsimage","integrity"],"backgroundTag":"checksum-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}