{"record":{"id":"cb1e6e736c93a20f","repo":"apache/hadoop","slug":"does-not-exist-cb1e6e","errorCode":null,"errorMessage":"{} does not exist.","messagePattern":"(.+?) does not exist\\.","errorType":"exception","errorClass":"FileNotFoundException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/MD5FileUtils.java","lineNumber":170,"sourceCode":"  private static void saveMD5File(File dataFile, String digestString)\n      throws IOException {\n    File md5File = getDigestFileForFile(dataFile);\n    String md5Line = digestString + \" *\" + dataFile.getName() + \"\\n\";\n\n    AtomicFileOutputStream afos = new AtomicFileOutputStream(md5File);\n    afos.write(md5Line.getBytes(StandardCharsets.UTF_8));\n    afos.close();\n\n    if (LOG.isDebugEnabled()) {\n      LOG.debug(\"Saved MD5 \" + digestString + \" to \" + md5File);\n    }\n  }\n\n  public static void renameMD5File(File oldDataFile, File newDataFile)\n      throws IOException {\n    final File fromFile = getDigestFileForFile(oldDataFile);\n    if (!fromFile.exists()) {\n      throw new FileNotFoundException(fromFile + \" does not exist.\");\n    }\n\n    final String digestString = readStoredMd5(fromFile).group(1);\n    saveMD5File(newDataFile, digestString);\n\n    if (!fromFile.delete()) {\n      LOG.warn(\"deleting  \" + fromFile.getAbsolutePath() + \" FAILED\");\n    }\n  }\n\n  /**\n   * @return a reference to the file with .md5 suffix that will\n   * contain the md5 checksum for the given data file.\n   */\n  public static File getDigestFileForFile(File file) {\n    return new File(file.getParentFile(), file.getName() + MD5_SUFFIX);\n  }\n}","sourceCodeStart":152,"sourceCodeEnd":188,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/MD5FileUtils.java#L152-L188","documentation":"renameMD5File(oldDataFile, newDataFile) moves a checksum sidecar across a data-file rename: it reads oldDataFile's .md5 and re-saves it under newDataFile's name. It throws FileNotFoundException when the source sidecar does not exist - a precondition check, since it cannot recompute a digest for a missing file. Used by FSImage when renaming checkpoint files (FSImage.java:1378).","triggerScenarios":"renameMD5File(a, b) when 'a.md5' is absent - never created (saveMD5File not called), already moved by an earlier invocation (retry after partial failure), deleted by cleanup, or the wrong path passed as oldDataFile.","commonSituations":"Checkpoint promotion retries after a partial failure where the first attempt already consumed the sidecar; code paths that skip checksum saving under some configurations; passing the new name in the old argument slot.","solutions":["Check MD5FileUtils.getDigestFileForFile(oldDataFile).exists() before calling; make the rename conditional.","If the sidecar should exist, locate it (moved earlier? other storage dir?) and restore it to the expected path.","If the data file still exists and you want a sidecar for the new name, generate one directly: MD5FileUtils.saveMD5File(newDataFile, MD5FileUtils.computeMd5ForFile(newDataFile).toString()).","Make multi-step checkpoint promotions idempotent so retries never assume step N-1's output is present."],"exampleFix":"// before\nMD5FileUtils.renameMD5File(fromFile, toFile); // throws '...md5 does not exist.' on retry\n\n// after - conditional rename with regeneration fallback\nFile oldMd5 = MD5FileUtils.getDigestFileForFile(fromFile);\nif (oldMd5.exists()) {\n  MD5FileUtils.renameMD5File(fromFile, toFile);\n} else if (toFile.exists()) {\n  MD5FileUtils.saveMD5File(toFile,\n      MD5FileUtils.computeMd5ForFile(toFile).toString());\n}","handlingStrategy":"validation","validationCode":"File oldMd5 = MD5FileUtils.getDigestFileForFile(oldDataFile);\nif (oldMd5.exists()) {\n  MD5FileUtils.renameMD5File(oldDataFile, newDataFile);\n} else if (newDataFile.exists()) {\n  MD5FileUtils.saveMD5File(newDataFile,\n      MD5FileUtils.computeMd5ForFile(newDataFile).toString());\n}","typeGuard":null,"tryCatchPattern":"try {\n  MD5FileUtils.renameMD5File(oldDataFile, newDataFile);\n} catch (FileNotFoundException e) { // '<old>.md5 does not exist.'\n  // sidecar already moved (retry) or never created - regenerate instead of failing\n  if (newDataFile.exists()) {\n    MD5FileUtils.saveMD5File(newDataFile,\n        MD5FileUtils.computeMd5ForFile(newDataFile).toString());\n  }\n}","preventionTips":["Make checkpoint rename flows idempotent so retries never assume step N-1's outputs exist.","Always conditionally check getDigestFileForFile(old).exists() before renameMD5File.","Ensure the step that produces the data file also saves its sidecar before any rename."],"tags":["filesystem","md5","precondition","file-not-found","checkpoint"],"backgroundTag":"file-not-found","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}