{"record":{"id":"5d4794471c004e03","repo":"apache/hadoop","slug":"cannot-append-to-an-unfinalized-replica-block","errorCode":null,"errorMessage":"Cannot append to an unfinalized replica {block}","messagePattern":"Cannot append to an unfinalized replica (.+?)","errorType":"exception","errorClass":"ReplicaNotFoundException","httpStatus":null,"severity":"warning","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java","lineNumber":1088,"sourceCode":"          + dstMeta, e);\n    }\n    LOG.debug(\"Linked {} to {} . Dest meta file: {}\", srcReplicaUri, dstFile,\n        dstMeta);\n    return new File[]{dstMeta, dstFile};\n  }\n\n  /**\n   * Move block files from one storage to another storage.\n   * @return Returns the Old replicaInfo\n   * @throws IOException\n   */\n  @Override\n  public ReplicaInfo moveBlockAcrossStorage(ExtendedBlock block,\n      StorageType targetStorageType, String targetStorageId)\n      throws IOException {\n    ReplicaInfo replicaInfo = getReplicaInfo(block);\n    if (replicaInfo.getState() != ReplicaState.FINALIZED) {\n      throw new ReplicaNotFoundException(\n          ReplicaNotFoundException.UNFINALIZED_REPLICA + block);\n    }\n    if (replicaInfo.getNumBytes() != block.getNumBytes()) {\n      throw new IOException(\"Corrupted replica \" + replicaInfo\n          + \" with a length of \" + replicaInfo.getNumBytes()\n          + \" expected length is \" + block.getNumBytes());\n    }\n    if (replicaInfo.getVolume().getStorageType() == targetStorageType) {\n      throw new ReplicaAlreadyExistsException(\"Replica \" + replicaInfo\n          + \" already exists on storage \" + targetStorageType);\n    }\n\n    if (replicaInfo.isOnTransientStorage()) {\n      // Block movement from RAM_DISK will be done by LazyPersist mechanism\n      throw new IOException(\"Replica \" + replicaInfo\n          + \" cannot be moved from storageType : \"\n          + replicaInfo.getVolume().getStorageType());\n    }","sourceCodeStart":1070,"sourceCodeEnd":1106,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java#L1070-L1106","documentation":"Thrown as ReplicaNotFoundException (UNFINALIZED_REPLICA prefix, 'Cannot append to an unfinalized replica') from FsDatasetImpl.moveBlockAcrossStorage(ExtendedBlock, StorageType, String) when the resolved replica's state is not FINALIZED — i.e. it is RBW (being written), RWR (recovered write), TEMP, or RUR. Storage-type moves only apply to finalized replicas, because unfinalized files are still owned by an active writer.","triggerScenarios":"NameNode/mover requests a storage-policy move (setStoragePolicy + Mover, or fast move during block scheduling) for a block whose local replica is mid-write: an open file's block, a block under lease recovery, or a replica still in RBW after a pipeline failure.","commonSituations":"Mover run while a job is actively writing the file; setStoragePolicy applied to open files; HDFS-9913-era fast move racing a replication recovery; block scanner-triggered moves of recently recovered RWR replicas.","solutions":["Close/wait for writers of the file (lease expiry or app completion) and re-run the mover — finalized replicas move fine.","Check replica state on the DN: the exception text includes the replica (toString carries state, e.g. RBW/RWR); confirm with the DN's block report for that block.","If the replica is stuck RWR/RUR after a dead writer, recover the lease (hdfs debug recoverLease) to finalize it, then move.","This is transient for the mover — it logs and retries later; no data is at risk."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Check replica state before requesting a cross-storage move.\nReplicaInfo info = fsDataset.getReplica(\n    block.getBlockPoolId(), block.getBlockId());\nif (info == null || info.getState() != ReplicaState.FINALIZED) {\n  // skip this block; the mover will revisit once writes finish\n  continue;\n}","typeGuard":null,"tryCatchPattern":"// Treat unfinalized as 'try later', matching mover semantics.\ntry {\n  fsDataset.moveBlockAcrossStorage(block, StorageType.ARCHIVE, null);\n} catch (ReplicaNotFoundException e) {\n  if (e.getMessage().contains(\"unfinalized\")) {\n    LOG.debug(\"Block {} not finalized yet; deferring move\", block);\n    deferForNextPass(block);\n  } else {\n    throw e;\n  }\n}","preventionTips":["Run mover against files that are closed (no active lease): hdfs fsck -openforwrite to find blockers.","Skip replicas whose state is not FINALIZED before issuing move calls.","Expect and tolerate these exceptions in mover loops — they are scheduling noise, not failures."],"tags":["hadoop","hdfs","datanode","fsdataset","mover","storage-policy","replica-state"],"backgroundTag":"replica-not-finalized","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}