apache/hadoop · error · FileNotFoundException

File is deleted: {} (inode {}) {}

Error message

File is deleted: {} (inode {}) {}

What it means

Thrown as FileNotFoundException when a lease operation targets an under-construction file whose INode is already gone. FSNamesystem considers a file deleted if it was removed from the inode map or if the snapshot feature marks it deleted (isFileDeleted). The NameNode refuses any further modification of a deleted file, so lease recovery, block allocation, and file completion all fail fast.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:3269

    assert hasReadLock(RwLockMode.FS);
    if (inode == null) {
      throw new FileNotFoundException("File does not exist: "
          + leaseExceptionString(src, fileId, holder));
    }
    if (!inode.isFile()) {
      throw new LeaseExpiredException("INode is not a regular file: "
          + leaseExceptionString(src, fileId, holder));
    }
    final INodeFile file = inode.asFile();
    if (!file.isUnderConstruction()) {
      throw new LeaseExpiredException("File is not open for writing: "
          + leaseExceptionString(src, fileId, holder));
    }
    // No further modification is allowed on a deleted file.
    // A file is considered deleted, if it is not in the inodeMap or is marked
    // as deleted in the snapshot feature.
    if (isFileDeleted(file)) {
      throw new FileNotFoundException("File is deleted: "
          + leaseExceptionString(src, fileId, holder));
    }
    final String owner = file.getFileUnderConstructionFeature().getClientName();
    if (holder != null && !owner.equals(holder)) {
      throw new LeaseExpiredException("Client (=" + holder
          + ") is not the lease owner (=" + owner + ": "
          + leaseExceptionString(src, fileId, holder));
    }
    return file;
  }
 
  /**
   * Complete in-progress write to the given file.
   * @return true if successful, false if the client should continue to retry
   *         (e.g if not all blocks have reached minimum replication yet)
   * @throws IOException on error (eg lease mismatch, file not open, file deleted)
   */
  boolean completeFile(final String src, String holder,

View on GitHub (pinned to 2add963021)

Solutions

1. Verify the path still exists (fs.exists / getFileStatus) before closing or recovering the lease; if it is gone, treat the write as failed and recreate the file
2. Audit for concurrent deleters of the same path (other clients, cleaner jobs, committer cleanup) and serialize delete vs. finalize
3. If snapshots are in play, confirm the file was not deleted within a snapshot that still pins it
4. After a failover, do not keep retrying the old file handle; restart the write to a new unique path

Example fix

// before
try { dfs.recoverLease(path); } catch (IOException e) { /* blind retry loop */ }

// after
try { dfs.recoverLease(path); }
catch (FileNotFoundException e) {
  LOG.warn("File was deleted, abandoning write: " + path);
  return; // terminal, do not retry
}

Path p = new Path(src);
if (!fs.exists(p)) {
  throw new IllegalStateException("Refusing lease op; file already deleted: " + p);
}

try {
  dfs.recoverLease(path); // or complete/close
} catch (FileNotFoundException e) {
  // file deleted underneath the writer: terminal, do not retry
  abandonWrite(path);
}

When it happens

Trigger: Calls that validate the write lease (addBlock, completeFile, recoverLease, abandonBlock, updateBlock) on a path deleted by another client, deleted inside a snapshot, or deleted by the same application while a writer still held the file open.

Common situations: A cleanup thread or job committer deletes a temp output while a slow writer is still closing it; MapReduce/Spark speculative tasks racing the output committer; retry loops that keep calling recoverLease on a path already removed; delete-inside-snapshot workflows.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• "File/Directory " + iip.getPath() + " does not exist."
• Client (={}) is not the lease owner (={}: {} (inode {}) {}
• The blockCollection of {} is null, likely because the file o
• File not found: {}, likely due to delayed block removal
• Path {} does not exist

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/96f555c42707e42c. Report an issue: GitHub.