apache/hadoop · error · LeaseExpiredException

Lease mismatch: {} is accessed by a non lease holder {}

Error message

Lease mismatch: {} is accessed by a non lease holder {}

What it means

checkUCBlock throws LeaseExpiredException when the clientName argument is null or differs from the client name in the file's FileUnderConstructionFeature: a non-lease-holder is attempting to update or recover the block. This is the single-writer enforcement at the block level.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:6057

    }
    if (storedBlock.getBlockUCState() != BlockUCState.UNDER_CONSTRUCTION) {
      throw new IOException("Unexpected BlockUCState: " + block
          + " is " + storedBlock.getBlockUCState()
          + " but not " + BlockUCState.UNDER_CONSTRUCTION);
    }
    
    // check file inode
    final INodeFile file = getBlockCollection(storedBlock);
    if (file == null || !file.isUnderConstruction() || isFileDeleted(file)) {
      throw new IOException("The file " + storedBlock + 
          " belonged to does not exist or it is not under construction.");
    }
    
    // check lease
    if (clientName == null
        || !clientName.equals(file.getFileUnderConstructionFeature()
            .getClientName())) {
      throw new LeaseExpiredException("Lease mismatch: " + block + 
          " is accessed by a non lease holder " + clientName); 
    }

    return file;
  }
  
  /**
   * Client is reporting some bad block locations.
   */
  void reportBadBlocks(LocatedBlock[] blocks) throws IOException {
    checkOperation(OperationCategory.WRITE);
    writeLock(RwLockMode.BM);
    try {
      checkOperation(OperationCategory.WRITE);
      for (int i = 0; i < blocks.length; i++) {
        ExtendedBlock blk = blocks[i].getBlock();
        DatanodeInfo[] nodes = blocks[i].getLocations();
        String[] storageIDs = blocks[i].getStorageIDs();

View on GitHub (pinned to 2add963021)

Solutions

1. Keep the lease alive during long pauses: hflush/hsync periodically or DFSClient keepalive
2. On catching LeaseExpiredException, call recoverLease, wait for isFileClosed, then append to continue
3. Use one writer per path with a unique client identity

Example fix

// before
out.write(chunk); // hours later: LeaseExpiredException on next addBlock/close

// after
// renew the lease during idle periods
if (now - lastWrite > leaseSoftLimit) { out.hflush(); } // or dfsClient.renewLease()

try {
  out.close(); // or addBlock path
} catch (LeaseExpiredException e) {
  // not the lease holder anymore: recover lease, wait, append to continue
  if (dfs.recoverLease(path)) {
    while (!dfs.isFileClosed(path)) { Thread.sleep(500); }
    out = dfs.append(path);
  }
}

When it happens

Trigger: updateBlock/nextGenerationStamp invoked by a client that is not the writer: the lease expired (idle past soft/hard limits) and was recovered by someone else, or the writer is a duplicate attempt using a different client name.

Common situations: Writer paused longer than dfs.namenode.lease-hard-limit-sec (default ~1h); readers triggering recoverLease to unblock reads of an open file; speculative duplicate tasks writing the same output.

Related errors

• Client (={}) is not the lease owner (={}: {} (inode {}) {}
• Failed to {} {} for {} on {} because {} is already the curre
• Failed to {} {} for {} on {} because this file lease is curr
• File is deleted: {} (inode {}) {}
• Block (={}) not found

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e3bae66195c8fec7. Report an issue: GitHub.