{"record":{"id":"f3a99fad4c96b7ec","repo":"apache/hadoop","slug":"append-lastblock-lastblock-of-src-pa","errorCode":null,"errorMessage":"\"append: lastBlock=\" + lastBlock + \" of src=\" + path + \" is COMMITTED but not yet COMPLETE.\"","messagePattern":"\"append: lastBlock=\" \\+ lastBlock \\+ \" of src=\" \\+ path \\+ \" is COMMITTED but not yet COMPLETE\\.\"","errorType":"exception","errorClass":"RetriableException","httpStatus":null,"severity":"warning","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirAppendOp.java","lineNumber":133,"sourceCode":"                + CreateFlag.NEW_BLOCK + \" create flag while appending file.\");\n      }\n\n      BlockManager blockManager = fsd.getBlockManager();\n      final BlockStoragePolicy lpPolicy = blockManager\n          .getStoragePolicy(\"LAZY_PERSIST\");\n      if (lpPolicy != null && lpPolicy.getId() == file.getStoragePolicyID()) {\n        throw new UnsupportedOperationException(\n            \"Cannot append to lazy persist file \" + path);\n      }\n      // Opening an existing file for append - may need to recover lease.\n      fsn.recoverLeaseInternal(RecoverLeaseOp.APPEND_FILE, iip, path, holder,\n          clientMachine, false);\n\n      final BlockInfo lastBlock = file.getLastBlock();\n      // Check that the block has at least minimum replication.\n      if (lastBlock != null) {\n        if (lastBlock.getBlockUCState() == BlockUCState.COMMITTED) {\n          throw new RetriableException(\n              new NotReplicatedYetException(\"append: lastBlock=\"\n                  + lastBlock + \" of src=\" + path\n                  + \" is COMMITTED but not yet COMPLETE.\"));\n        } else if (lastBlock.isComplete()\n          && !blockManager.isSufficientlyReplicated(lastBlock)) {\n          throw new IOException(\"append: lastBlock=\" + lastBlock + \" of src=\"\n              + path + \" is not sufficiently replicated yet.\");\n        }\n      }\n      lb = prepareFileForAppend(fsn, iip, holder, clientMachine, newBlock,\n          true, logRetryCache);\n    } catch (IOException ie) {\n      NameNode.stateChangeLog\n          .warn(\"DIR* NameSystem.append: \" + ie.getMessage());\n      throw ie;\n    } finally {\n      fsd.writeUnlock();\n    }","sourceCodeStart":115,"sourceCodeEnd":151,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirAppendOp.java#L115-L151","documentation":"The file's last block is in COMMITTED state: the writer requested finalization but the NameNode has not yet seen COMPLETE reports from enough DataNodes, so append cannot safely extend or add a block after it. The NameNode wraps NotReplicatedYetException in RetriableException, explicitly telling the client to retry the same append later instead of failing.","triggerScenarios":"Calling append immediately after another writer closes or hflushes the same file while block finalization is still in flight; appending right after lease recovery; DataNodes slow to process blockReceivedAndDeleted reports.","commonSituations":"Two pipeline stages appending to one file back-to-back; append after recovering the lease of a dead client; overloaded or GC-pausing DataNodes delaying incremental block reports.","solutions":["Retry the append with short backoff - RetriableException is designed for retry and typically clears within seconds once the block reaches COMPLETE.","Ensure the previous writer fully closes its output stream before the next stage starts appending.","If it persists, check DataNode liveness ('hdfs dfsadmin -report') and NameNode metrics for pending block-completion backlogs."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"for (int attempt = 0; attempt < 5; attempt++) {\n  try {\n    return dfs.append(path, bufferSize);\n  } catch (RemoteException re) {\n    if (\"org.apache.hadoop.ipc.RetriableException\".equals(re.getClassName())) {\n      Thread.sleep(500L << attempt); // exponential backoff, block is finalizing\n      continue;\n    }\n    throw re;\n  }\n}\nthrow new IOException(\"append still retriable after retries: \" + path);","preventionTips":["Fully close the previous writer before the next stage appends.","Treat RetriableException as transient in monitoring; alert only on sustained repeats."],"tags":["hdfs","append","block-state","retry","replication"],"backgroundTag":"block-not-yet-complete","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}