apache/hadoop · error · IOException

Trying to delete non-existant block {}

Error message

Trying to delete non-existant block {}

What it means

After updateBlocks removes the last block via FSDirWriteFileOp.unprotectedRemoveBlock, a false return means the block was not present in the file. UpdateBlocksOp is exempt (its reapplication is tolerated), but any other op throws IOException('Trying to delete non-existant block <block>'; the spelling is in the source). The op names a block the namespace does not hold.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java:1189

        fsNamesys.getBlockManager().processQueuedMessagesForBlock(newBlock);
      }
    }
    
    if (newBlocks.length < oldBlocks.length) {
      // We're removing a block from the file, e.g. abandonBlock(...)
      if (!file.isUnderConstruction()) {
        throw new IOException("Trying to remove a block from file " +
            path + " which is not under construction.");
      }
      if (newBlocks.length != oldBlocks.length - 1) {
        throw new IOException("Trying to remove more than one block from file "
            + path);
      }
      Block oldBlock = oldBlocks[oldBlocks.length - 1];
      boolean removed = FSDirWriteFileOp.unprotectedRemoveBlock(
          fsDir, path, iip, file, oldBlock);
      if (!removed && !(op instanceof UpdateBlocksOp)) {
        throw new IOException("Trying to delete non-existant block " + oldBlock);
      }
    } else if (newBlocks.length > oldBlocks.length) {
      final boolean isStriped = ecPolicy != null;
      // We're adding blocks
      for (int i = oldBlocks.length; i < newBlocks.length; i++) {
        Block newBlock = newBlocks[i];
        final BlockInfo newBI;
        if (!op.shouldCompleteLastBlock()) {
          // TODO: shouldn't this only be true for the last block?
          // what about an old-version fsync() where fsync isn't called
          // until several blocks in?
          if (isStriped) {
            newBI = new BlockInfoStriped(newBlock, ecPolicy);
          } else {
            newBI = new BlockInfoContiguous(newBlock,
                file.getPreferredBlockReplication());
          }
          newBI.convertToBlockUnderConstruction(

View on GitHub (pinned to 2add963021)

Solutions

1. Run 'hdfs namenode -recover' to skip the inconsistent transaction
2. Restore a consistent fsimage plus edits from one backup epoch
3. Inspect the file's transaction history with 'hdfs oev' to find where the block was already removed

Example fix

# before: replay fails 'Trying to delete non-existant block blk_123'
hdfs --daemon start namenode

# after: recover past the duplicate removal
hdfs namenode -recover

try {
  loader.loadFSEdits(storage, 0);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("non-existant block")) {
    // block already absent: duplicate replay or diverged state; recover or
    // restore a consistent fsimage+edits pair
  }
  throw e;
}

When it happens

Trigger: Replaying a block-removal op whose target block is already gone: duplicate replay, image/edits divergence, or corrupted block fields.

Common situations: Same replay-inconsistency family: mixed restores, duplicated segments, corruption.

Related errors

• Mismatched block IDs or generation stamps for the old last b
• Mismatched block IDs or generation stamps, attempting to rep
• Trying to remove more than one block from file {}
• The layout version {} supports inodeId but gave bogus inodeI
• Trying to remove a block from file {} which is not under con

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ac4d2e2522096b4f. Report an issue: GitHub.