{"record":{"id":"c21105e80fcd92bd","repo":"apache/hadoop","slug":"not-replicated-yet-src","errorCode":null,"errorMessage":"Not replicated yet: <src>","messagePattern":"Not replicated yet: <src>","errorType":"exception","errorClass":"NotReplicatedYetException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java","lineNumber":183,"sourceCode":"      ExtendedBlock previous, LocatedBlock[] onRetryBlock) throws IOException {\n    final long blockSize;\n    final short numTargets;\n    final byte storagePolicyID;\n    String clientMachine;\n    final BlockType blockType;\n\n    INodesInPath iip = fsn.dir.resolvePath(pc, src, fileId);\n    FileState fileState = analyzeFileState(fsn, iip, fileId, clientName,\n                                           previous, onRetryBlock);\n    if (onRetryBlock[0] != null && onRetryBlock[0].getLocations().length > 0) {\n      // This is a retry. No need to generate new locations.\n      // Use the last block if it has locations.\n      return null;\n    }\n\n    final INodeFile pendingFile = fileState.inode;\n    if (!fsn.checkFileProgress(src, pendingFile, false)) {\n      throw new NotReplicatedYetException(\"Not replicated yet: \" + src);\n    }\n    if (pendingFile.getBlocks().length >= fsn.maxBlocksPerFile) {\n      throw new IOException(\"File has reached the limit on maximum number of\"\n          + \" blocks (\" + DFSConfigKeys.DFS_NAMENODE_MAX_BLOCKS_PER_FILE_KEY\n          + \"): \" + pendingFile.getBlocks().length + \" >= \"\n          + fsn.maxBlocksPerFile);\n    }\n    blockSize = pendingFile.getPreferredBlockSize();\n    clientMachine = pendingFile.getFileUnderConstructionFeature()\n        .getClientMachine();\n    blockType = pendingFile.getBlockType();\n    ErasureCodingPolicy ecPolicy = null;\n    if (blockType == BlockType.STRIPED) {\n      ecPolicy =\n          FSDirErasureCodingOp.unprotectedGetErasureCodingPolicy(fsn, iip);\n      numTargets = (short) (ecPolicy.getSchema().getNumDataUnits()\n          + ecPolicy.getSchema().getNumParityUnits());\n    } else {","sourceCodeStart":165,"sourceCodeEnd":201,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java#L165-L201","documentation":"Before allocating a new block for a file under construction, the NameNode requires every block except the last to be at least minimally replicated (dfs.namenode.replication.min, default 1). analyzeFileState calls checkFileProgress and throws NotReplicatedYetException when the penultimate block has not reached that level: the client is asking for the next block faster than datanodes have reported the previous one.","triggerScenarios":"ClientProtocol.addBlock() for the next block while the previous block still has fewer than min-replication live replicas: right after a pipeline error, when datanodes are dead, full, or restarting, or when block reports have not arrived yet.","commonSituations":"Dead or overloaded datanodes; cluster too small for the file's replication factor; custom clients calling addBlock in a tight retry loop; rolling datanode restarts during writes.","solutions":["Let the standard DFSOutputStream handle it: it retries NotReplicatedYetException with backoff and usually self-heals once datanodes report the block","Check cluster health: 'hdfs dfsadmin -report' for live nodes and 'hdfs fsck <path>' for under-replicated blocks; fix dead disks or full datanodes","Ensure enough healthy datanodes exist for the file's replication factor before large writes","Only as a last resort, lower dfs.namenode.replication.min, accepting weaker durability"],"exampleFix":"// before: raw addBlock call fails fast\nLocatedBlock lb = namenode.addBlock(src, clientName, previous, fileId, null, 0);\n\n// after: honor the transient backoff signal\nlong delay = 500L;\nfor (int i = 0; i < 30; i++) {\n  try {\n    lb = namenode.addBlock(src, clientName, previous, fileId, null, 0);\n    break;\n  } catch (NotReplicatedYetException e) {\n    Thread.sleep(delay);\n    delay = Math.min(delay * 2, 10_000L);\n  }\n}","handlingStrategy":"retry","validationCode":"DistributedFileSystem dfs = (DistributedFileSystem) fs;\nint live = dfs.getDataNodeStats(DatanodeReportType.LIVE).length;\nif (live < fileReplication) {\n  throw new IOException(\"Only \" + live + \" live datanodes for replication \" + fileReplication);\n}","typeGuard":null,"tryCatchPattern":"long delay = 500L;\nwhile (true) {\n  try {\n    return namenode.addBlock(src, clientName, previous, fileId, null, 0);\n  } catch (NotReplicatedYetException e) {\n    Thread.sleep(delay);\n    delay = Math.min(delay * 2, 10_000L);\n  }\n}","preventionTips":["Use DFSOutputStream instead of raw ClientProtocol; it already retries this exception with backoff","Keep enough healthy datanodes for the replication factor during bulk writes","Avoid rolling datanode restarts while large writes are in flight"],"tags":["hdfs","block-allocation","replication","datanode-health","transient"],"backgroundTag":"insufficient-replication","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}