apache/hadoop · error · IOException

DIR* NameSystem.internalReleaseLease: attempt to release a c

Error message

DIR* NameSystem.internalReleaseLease: attempt to release a create lock on {} but file is already closed.

What it means

internalReleaseLease throws IOException when it is asked to release a create lock but the file's block states violate the open-file invariant: more than the last two blocks are non-COMPLETE, or the penultimate block is neither COMPLETE nor COMMITTED. Because the file looks effectively closed but inconsistent, lease recovery aborts with a DIR* state-change warning instead of corrupting the namespace.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:3817

      finalizeINodeFileUnderConstruction(src, pendingFile,
          iip.getLatestSnapshotId(), false);
      NameNode.stateChangeLog.warn("BLOCK*" +
          " internalReleaseLease: All existing blocks are COMPLETE," +
          " lease removed, file " + src + " closed.");
      return true;  // closed!
    }

    // Only the last and the penultimate blocks may be in non COMPLETE state.
    // If the penultimate block is not COMPLETE, then it must be COMMITTED.
    if(nrCompleteBlocks < nrBlocks - 2 ||
       nrCompleteBlocks == nrBlocks - 2 &&
         curBlock != null &&
         curBlock.getBlockUCState() != BlockUCState.COMMITTED) {
      final String message = "DIR* NameSystem.internalReleaseLease: "
        + "attempt to release a create lock on "
        + src + " but file is already closed.";
      NameNode.stateChangeLog.warn(message);
      throw new IOException(message);
    }

    // The last block is not COMPLETE, and
    // that the penultimate block if exists is either COMPLETE or COMMITTED
    final BlockInfo lastBlock = pendingFile.getLastBlock();
    BlockUCState lastBlockState = lastBlock.getBlockUCState();
    BlockInfo penultimateBlock = pendingFile.getPenultimateBlock();

    // If penultimate block doesn't exist then its minReplication is met
    boolean penultimateBlockMinStorage = penultimateBlock == null ||
        blockManager.hasMinStorage(penultimateBlock);

    switch(lastBlockState) {
    case COMPLETE:
      assert false : "Already checked that the last block is incomplete";
      break;
    case COMMITTED:
      // Close file if committed blocks are minimally replicated

View on GitHub (pinned to 2add963021)

Solutions

1. Run 'hdfs fsck /path -files -blocks -locations' to see the block states of the affected file
2. If the data is dispensable, delete the file (delete also removes its lease) and rewrite it
3. If data must be saved, copy out readable bytes (hdfs dfs -get / distcp), then delete and re-ingest
4. Try 'hdfs debug recoverLease -path <path> -retries N' to force recovery attempts
5. Upgrade Hadoop if the block-state corruption matches a fixed JIRA (check truncate/recovery fixes for your version)

try {
  dfs.recoverLease(path);
} catch (IOException e) {
  if (e.getMessage().contains("attempt to release a create lock")) {
    // invariant-broken file: salvage data then delete and rewrite
    salvageAndRewrite(path);
  } else { throw e; }
}

When it happens

Trigger: Lease expiry or recoverLease on a file where earlier (non-terminal) blocks were never committed: repeated failed pipeline recoveries, a crashed writer with multiple abandoned blocks, or block-state corruption from a buggy truncate/recovery path.

Common situations: Long-running writers killed mid-file whose pipelines repeatedly broke; files hit by known recovery bugs in older Hadoop 2.x releases; namespace restored from a bad checkpoint.

Related errors

• Cannot complete block: block has not been COMMITTED by the c
• Recovery block {b} where it is not under construction.
• Trying to commit inconsistent block: id = {blockId}, expecte
• Commit block with mismatching GS. NN has {block}, client sub
• Commit or complete block {commitBlock}, whereas it is under

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/aa00b8220f978ccc. Report an issue: GitHub.