apache/hadoop · error · FileNotFoundException

File does not exist: {}

Error message

File does not exist: {}

What it means

getFileChecksum(src, length) resolves the file's block locations first; a null LocatedBlocks response means the file is not in the namespace, and the client throws FileNotFoundException before contacting any Datanode. So a checksum failure of this shape is a missing-file problem, not a checksum computation problem.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java:1962

   *
   * @param src The file path
   * @param length the length of the range, i.e., the range is [0, length]
   * @return The checksum
   * @see DistributedFileSystem#getFileChecksum(Path)
   */
  public MD5MD5CRC32FileChecksum getFileChecksum(String src, long length)
      throws IOException {
    return (MD5MD5CRC32FileChecksum) getFileChecksumInternal(
        src, length, ChecksumCombineMode.MD5MD5CRC);
  }

  protected LocatedBlocks getBlockLocations(String src,
                                            long length) throws IOException {
    //get block locations for the file range
    LocatedBlocks blockLocations = callGetBlockLocations(namenode,
        src, 0, length);
    if (null == blockLocations) {
      throw new FileNotFoundException("File does not exist: " + src);
    }
    if (blockLocations.isUnderConstruction()) {
      throw new IOException("Fail to get checksum, since file " + src
          + " is under construction.");
    }

    return blockLocations;
  }

  protected IOStreamPair connectToDN(DatanodeInfo dn, int timeout,
                                     Token<BlockTokenIdentifier> blockToken)
      throws IOException {
    return DFSUtilClient.connectToDN(dn, timeout, conf, saslClient,
        socketFactory, getConf().isConnectToDnViaHostname(), this, blockToken);
  }

  /**
   * Infer the checksum type for a replica by sending an OP_READ_BLOCK

View on GitHub (pinned to 2add963021)

Solutions

1. Gate with getFileStatus() immediately before checksumming and re-check existence in the catch to classify the failure.
2. For verify-after-copy flows, checksum the same generation you copied (e.g. source snapshot) rather than a mutating live tree.
3. If concurrent deletes are expected, skip-and-log missing files instead of failing the whole run.

Example fix

// before
FileChecksum cs = fs.getFileChecksum(path);
// FileNotFoundException: File does not exist: <path>

// after
FileStatus st;
try {
  st = fs.getFileStatus(path);
} catch (FileNotFoundException e) {
  return Optional.empty(); // concurrently deleted: skip this file
}
FileChecksum cs = fs.getFileChecksum(path);

if (!fs.exists(path)) {
  // vanished or never existed: skip this record instead of checksumming
}

catch (FileNotFoundException e) {
  // concurrently deleted: skip the file, or fail the transfer record with path context
}

When it happens

Trigger: getFileChecksum on a path deleted between the caller's existence check and the call; checksum-verification pipelines (distcp -checksum, audit tools) walking trees that a retention job is concurrently pruning; verifying output of a job that wrote to a different path.

Common situations: distcp source files cleaned mid-transfer; audit/verification steps racing writers; consumers verifying a file that a writer publishes via rename after the verifier started.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• File not found: {}, likely due to delayed block removal
• Directory does not exist: {}
• File does not exist: {}
• Cannot open filename {}
• key + ": No such file or directory."

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/546ce29b812f0fde. Report an issue: GitHub.