{"record":{"id":"0ae6016720f7f10f","repo":"apache/hadoop","slug":"request-looked-like-a-retry-to-allocate-block-las","errorCode":null,"errorMessage":"Request looked like a retry to allocate block <lastBlockInFile> but it already contains <numBytes> bytes","messagePattern":"Request looked like a retry to allocate block <lastBlockInFile> but it already contains <numBytes> bytes","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java","lineNumber":663,"sourceCode":"      //    the second attempts in Part I, because the first one hasn't\n      //    changed the namesystem state yet.\n      //    We run this analysis again in Part II where case 4 is impossible.\n\n      BlockInfo penultimateBlock = file.getPenultimateBlock();\n      if (previous == null &&\n          lastBlockInFile != null &&\n          lastBlockInFile.getNumBytes() >= file.getPreferredBlockSize() &&\n          lastBlockInFile.isComplete()) {\n        // Case 1\n        if (NameNode.stateChangeLog.isDebugEnabled()) {\n           NameNode.stateChangeLog.debug(\n               \"BLOCK* NameSystem.allocateBlock: handling block allocation\" +\n               \" writing to a file with a complete previous block: src=\" +\n               src + \" lastBlock=\" + lastBlockInFile);\n        }\n      } else if (Block.matchingIdAndGenStamp(penultimateBlock, previousBlock)) {\n        if (lastBlockInFile.getNumBytes() != 0) {\n          throw new IOException(\n              \"Request looked like a retry to allocate block \" +\n              lastBlockInFile + \" but it already contains \" +\n              lastBlockInFile.getNumBytes() + \" bytes\");\n        }\n\n        // Case 2\n        // Return the last block.\n        NameNode.stateChangeLog.info(\"BLOCK* allocateBlock: caught retry for \" +\n            \"allocation of a new block in \" + src + \". Returning previously\" +\n            \" allocated block \" + lastBlockInFile);\n        long offset = file.computeFileSize();\n        BlockUnderConstructionFeature uc =\n            lastBlockInFile.getUnderConstructionFeature();\n        onRetryBlock[0] = makeLocatedBlock(fsn, lastBlockInFile,\n            uc.getExpectedStorageLocations(), offset);\n        return new FileState(file, src, iip);\n      } else {\n        // Case 3","sourceCodeStart":645,"sourceCodeEnd":681,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java#L645-L681","documentation":"analyzeFileState classifies addBlock retries: when the client's previous block matches the file's penultimate block, the NameNode wants to hand back the still-empty last block. If that last block already contains bytes, the 'retry' is inconsistent with NameNode state: the client is replaying an RPC from before data was written. It throws IOException rather than hand one block to two writers.","triggerScenarios":"ClientProtocol.addBlock(previous=B) where B matches the penultimate block but the last block is non-empty: duplicate or replayed RPCs after the original succeeded and data flowed, or custom clients caching and re-sending block allocations.","commonSituations":"Application retry wrappers replaying captured RPCs; at-least-once delivery combined with manual ClientProtocol use; rare failover edge cases in old client versions.","solutions":["Abort the write and restart from consistent state: recover/close the lease, then append or write a new file","Fix the client so it never re-sends addBlock after the returned block has been written to","Use DFSOutputStream semantics: never cache and replay block allocations by hand"],"exampleFix":"// before: replaying a stale allocation\nLocatedBlock lb = namenode.addBlock(src, clientName, stalePrevious, fileId, null, 0); // throws\n\n// after: re-establish the writer from authoritative state\ntry (FSDataOutputStream out = fs.append(path)) {\n  out.write(payload);\n}","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  lb = namenode.addBlock(src, clientName, previous, fileId, null, 0);\n} catch (IOException e) { // \"Request looked like a retry ...\"\n  // client state diverged from the NameNode: abandon this writer\n  writer.abort();\n  try (FSDataOutputStream out = fs.append(path)) {\n    /* rewrite the tail from a consistent offset */\n  }\n}","preventionTips":["Never cache and replay addBlock RPCs; use DFSOutputStream","Make application-level retries idempotent against NameNode state","On any block-allocation IOException, re-sync from the NameNode before writing more data"],"tags":["hdfs","addblock","retry","client-state","data-integrity"],"backgroundTag":"stale-client-retry","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}