apache/hadoop · error · FileNotFoundException

{} does not exist.

Error message

{} does not exist.

What it means

renameMD5File(oldDataFile, newDataFile) moves a checksum sidecar across a data-file rename: it reads oldDataFile's .md5 and re-saves it under newDataFile's name. It throws FileNotFoundException when the source sidecar does not exist - a precondition check, since it cannot recompute a digest for a missing file. Used by FSImage when renaming checkpoint files (FSImage.java:1378).

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/MD5FileUtils.java:170

  private static void saveMD5File(File dataFile, String digestString)
      throws IOException {
    File md5File = getDigestFileForFile(dataFile);
    String md5Line = digestString + " *" + dataFile.getName() + "\n";

    AtomicFileOutputStream afos = new AtomicFileOutputStream(md5File);
    afos.write(md5Line.getBytes(StandardCharsets.UTF_8));
    afos.close();

    if (LOG.isDebugEnabled()) {
      LOG.debug("Saved MD5 " + digestString + " to " + md5File);
    }
  }

  public static void renameMD5File(File oldDataFile, File newDataFile)
      throws IOException {
    final File fromFile = getDigestFileForFile(oldDataFile);
    if (!fromFile.exists()) {
      throw new FileNotFoundException(fromFile + " does not exist.");
    }

    final String digestString = readStoredMd5(fromFile).group(1);
    saveMD5File(newDataFile, digestString);

    if (!fromFile.delete()) {
      LOG.warn("deleting  " + fromFile.getAbsolutePath() + " FAILED");
    }
  }

  /**
   * @return a reference to the file with .md5 suffix that will
   * contain the md5 checksum for the given data file.
   */
  public static File getDigestFileForFile(File file) {
    return new File(file.getParentFile(), file.getName() + MD5_SUFFIX);
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Check MD5FileUtils.getDigestFileForFile(oldDataFile).exists() before calling; make the rename conditional.
2. If the sidecar should exist, locate it (moved earlier? other storage dir?) and restore it to the expected path.
3. If the data file still exists and you want a sidecar for the new name, generate one directly: MD5FileUtils.saveMD5File(newDataFile, MD5FileUtils.computeMd5ForFile(newDataFile).toString()).
4. Make multi-step checkpoint promotions idempotent so retries never assume step N-1's output is present.

Example fix

// before
MD5FileUtils.renameMD5File(fromFile, toFile); // throws '...md5 does not exist.' on retry

// after - conditional rename with regeneration fallback
File oldMd5 = MD5FileUtils.getDigestFileForFile(fromFile);
if (oldMd5.exists()) {
  MD5FileUtils.renameMD5File(fromFile, toFile);
} else if (toFile.exists()) {
  MD5FileUtils.saveMD5File(toFile,
      MD5FileUtils.computeMd5ForFile(toFile).toString());
}

File oldMd5 = MD5FileUtils.getDigestFileForFile(oldDataFile);
if (oldMd5.exists()) {
  MD5FileUtils.renameMD5File(oldDataFile, newDataFile);
} else if (newDataFile.exists()) {
  MD5FileUtils.saveMD5File(newDataFile,
      MD5FileUtils.computeMd5ForFile(newDataFile).toString());
}

try {
  MD5FileUtils.renameMD5File(oldDataFile, newDataFile);
} catch (FileNotFoundException e) { // '<old>.md5 does not exist.'
  // sidecar already moved (retry) or never created - regenerate instead of failing
  if (newDataFile.exists()) {
    MD5FileUtils.saveMD5File(newDataFile,
        MD5FileUtils.computeMd5ForFile(newDataFile).toString());
  }
}

When it happens

Trigger: renameMD5File(a, b) when 'a.md5' is absent - never created (saveMD5File not called), already moved by an earlier invocation (retry after partial failure), deleted by cleanup, or the wrong path passed as oldDataFile.

Common situations: Checkpoint promotion retries after a partial failure where the first attempt already consumed the sidecar; code paths that skip checksum saving under some configurations; passing the new name in the old argument slot.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• %s not found: %s
• Parent directory doesn't exist: {}
• Non existing file: ${path}. Create option is not specified i
• rename source ${src} not found.
• %s does not exist or is not file.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/cb1e6e736c93a20f. Report an issue: GitHub.