apache/hadoop · error · IOException

"append: lastBlock=" + lastBlock + " of src=" + path + " is

Error message

"append: lastBlock=" + lastBlock + " of src=" + path + " is not sufficiently replicated yet."

What it means

append requires the last complete block to be at least minimally replicated before the file may be extended; blockManager.isSufficientlyReplicated(lastBlock) returned false, so the NameNode throws IOException 'not sufficiently replicated yet'. Unlike the COMMITTED case this is about live replica count, not block finalization state.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirAppendOp.java:139

      if (lpPolicy != null && lpPolicy.getId() == file.getStoragePolicyID()) {
        throw new UnsupportedOperationException(
            "Cannot append to lazy persist file " + path);
      }
      // Opening an existing file for append - may need to recover lease.
      fsn.recoverLeaseInternal(RecoverLeaseOp.APPEND_FILE, iip, path, holder,
          clientMachine, false);

      final BlockInfo lastBlock = file.getLastBlock();
      // Check that the block has at least minimum replication.
      if (lastBlock != null) {
        if (lastBlock.getBlockUCState() == BlockUCState.COMMITTED) {
          throw new RetriableException(
              new NotReplicatedYetException("append: lastBlock="
                  + lastBlock + " of src=" + path
                  + " is COMMITTED but not yet COMPLETE."));
        } else if (lastBlock.isComplete()
          && !blockManager.isSufficientlyReplicated(lastBlock)) {
          throw new IOException("append: lastBlock=" + lastBlock + " of src="
              + path + " is not sufficiently replicated yet.");
        }
      }
      lb = prepareFileForAppend(fsn, iip, holder, clientMachine, newBlock,
          true, logRetryCache);
    } catch (IOException ie) {
      NameNode.stateChangeLog
          .warn("DIR* NameSystem.append: " + ie.getMessage());
      throw ie;
    } finally {
      fsd.writeUnlock();
    }

    HdfsFileStatus stat =
        FSDirStatAndListingOp.getFileInfo(fsd, iip, false, false);
    if (lb != null) {
      NameNode.stateChangeLog.debug(
          "DIR* NameSystem.appendFile: file {} for {} at {} block {} block"

View on GitHub (pinned to 2add963021)

Solutions

1. Wait and retry the append - re-replication usually restores sufficient replication on its own.
2. Run 'hdfs fsck <path> -files -blocks -locations' to see the last block's replica locations and whether replicas are missing or merely pending.
3. Restore dead DataNodes or let under-replication processing fix the block; if replicas are permanently lost, the file needs manual recovery (fsck -repair or re-copy from source).

for (int attempt = 0; attempt < 10; attempt++) {
  try {
    return dfs.append(path, bufferSize);
  } catch (IOException e) {
    if (e.getMessage() != null && e.getMessage().contains("not sufficiently replicated")) {
      Thread.sleep(2000L); // wait for re-replication
      continue;
    }
    throw e;
  }
}

When it happens

Trigger: append() when the file's last block has fewer live replicas than the minimum (dfs.namenode.replication.min): DataNodes holding replicas died or were decommissioned, or replication has not caught up after a node failure.

Common situations: Clusters with DataNodes down or in maintenance; files whose only replicas of recent blocks were on one lost node; small clusters where a single dead DataNode drops last blocks below minimum replication.

Related errors

• "append: lastBlock=" + lastBlock + " of src=" + path + " is
• Cannot append to a replica with unexpected generation stamp
• The new generation stamp {newGS} should be greater than the
• Cannot append to an unfinalized replica {b}
• Corrupted replica {replicaInfo} with a length of {numBytes}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d17148f9604a4bda. Report an issue: GitHub.