{"record":{"id":"d17148f9604a4bda","repo":"apache/hadoop","slug":"append-lastblock-lastblock-of-src-pa-d17148","errorCode":null,"errorMessage":"\"append: lastBlock=\" + lastBlock + \" of src=\" + path + \" is not sufficiently replicated yet.\"","messagePattern":"\"append: lastBlock=\" \\+ lastBlock \\+ \" of src=\" \\+ path \\+ \" is not sufficiently replicated yet\\.\"","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirAppendOp.java","lineNumber":139,"sourceCode":"      if (lpPolicy != null && lpPolicy.getId() == file.getStoragePolicyID()) {\n        throw new UnsupportedOperationException(\n            \"Cannot append to lazy persist file \" + path);\n      }\n      // Opening an existing file for append - may need to recover lease.\n      fsn.recoverLeaseInternal(RecoverLeaseOp.APPEND_FILE, iip, path, holder,\n          clientMachine, false);\n\n      final BlockInfo lastBlock = file.getLastBlock();\n      // Check that the block has at least minimum replication.\n      if (lastBlock != null) {\n        if (lastBlock.getBlockUCState() == BlockUCState.COMMITTED) {\n          throw new RetriableException(\n              new NotReplicatedYetException(\"append: lastBlock=\"\n                  + lastBlock + \" of src=\" + path\n                  + \" is COMMITTED but not yet COMPLETE.\"));\n        } else if (lastBlock.isComplete()\n          && !blockManager.isSufficientlyReplicated(lastBlock)) {\n          throw new IOException(\"append: lastBlock=\" + lastBlock + \" of src=\"\n              + path + \" is not sufficiently replicated yet.\");\n        }\n      }\n      lb = prepareFileForAppend(fsn, iip, holder, clientMachine, newBlock,\n          true, logRetryCache);\n    } catch (IOException ie) {\n      NameNode.stateChangeLog\n          .warn(\"DIR* NameSystem.append: \" + ie.getMessage());\n      throw ie;\n    } finally {\n      fsd.writeUnlock();\n    }\n\n    HdfsFileStatus stat =\n        FSDirStatAndListingOp.getFileInfo(fsd, iip, false, false);\n    if (lb != null) {\n      NameNode.stateChangeLog.debug(\n          \"DIR* NameSystem.appendFile: file {} for {} at {} block {} block\"","sourceCodeStart":121,"sourceCodeEnd":157,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirAppendOp.java#L121-L157","documentation":"append requires the last complete block to be at least minimally replicated before the file may be extended; blockManager.isSufficientlyReplicated(lastBlock) returned false, so the NameNode throws IOException 'not sufficiently replicated yet'. Unlike the COMMITTED case this is about live replica count, not block finalization state.","triggerScenarios":"append() when the file's last block has fewer live replicas than the minimum (dfs.namenode.replication.min): DataNodes holding replicas died or were decommissioned, or replication has not caught up after a node failure.","commonSituations":"Clusters with DataNodes down or in maintenance; files whose only replicas of recent blocks were on one lost node; small clusters where a single dead DataNode drops last blocks below minimum replication.","solutions":["Wait and retry the append - re-replication usually restores sufficient replication on its own.","Run 'hdfs fsck <path> -files -blocks -locations' to see the last block's replica locations and whether replicas are missing or merely pending.","Restore dead DataNodes or let under-replication processing fix the block; if replicas are permanently lost, the file needs manual recovery (fsck -repair or re-copy from source)."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"for (int attempt = 0; attempt < 10; attempt++) {\n  try {\n    return dfs.append(path, bufferSize);\n  } catch (IOException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"not sufficiently replicated\")) {\n      Thread.sleep(2000L); // wait for re-replication\n      continue;\n    }\n    throw e;\n  }\n}","preventionTips":["Monitor under-replicated blocks and DataNode liveness so appends are not attempted during degraded states.","Keep dfs.namenode.replication.min satisfied; investigate dead DataNodes before scheduling append-heavy jobs."],"tags":["hdfs","append","replication","retry","datanode"],"backgroundTag":"insufficient-replication","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}