{"record":{"id":"6c862a9702a3ed0a","repo":"apache/hadoop","slug":"corrupted-replica-replicainfo-with-a-length-of","errorCode":null,"errorMessage":"Corrupted replica {replicaInfo} with a length of {numBytes} expected length is {numBytes}","messagePattern":"Corrupted replica (.+?) with a length of (.+?) expected length is (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java","lineNumber":1092,"sourceCode":"    return new File[]{dstMeta, dstFile};\n  }\n\n  /**\n   * Move block files from one storage to another storage.\n   * @return Returns the Old replicaInfo\n   * @throws IOException\n   */\n  @Override\n  public ReplicaInfo moveBlockAcrossStorage(ExtendedBlock block,\n      StorageType targetStorageType, String targetStorageId)\n      throws IOException {\n    ReplicaInfo replicaInfo = getReplicaInfo(block);\n    if (replicaInfo.getState() != ReplicaState.FINALIZED) {\n      throw new ReplicaNotFoundException(\n          ReplicaNotFoundException.UNFINALIZED_REPLICA + block);\n    }\n    if (replicaInfo.getNumBytes() != block.getNumBytes()) {\n      throw new IOException(\"Corrupted replica \" + replicaInfo\n          + \" with a length of \" + replicaInfo.getNumBytes()\n          + \" expected length is \" + block.getNumBytes());\n    }\n    if (replicaInfo.getVolume().getStorageType() == targetStorageType) {\n      throw new ReplicaAlreadyExistsException(\"Replica \" + replicaInfo\n          + \" already exists on storage \" + targetStorageType);\n    }\n\n    if (replicaInfo.isOnTransientStorage()) {\n      // Block movement from RAM_DISK will be done by LazyPersist mechanism\n      throw new IOException(\"Replica \" + replicaInfo\n          + \" cannot be moved from storageType : \"\n          + replicaInfo.getVolume().getStorageType());\n    }\n\n    FsVolumeReference volumeRef = null;\n    boolean shouldConsiderSameMountVolume =\n        shouldConsiderSameMountVolume(replicaInfo.getVolume(),","sourceCodeStart":1074,"sourceCodeEnd":1110,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java#L1074-L1110","documentation":"Thrown as plain IOException from FsDatasetImpl.moveBlockAcrossStorage when replicaInfo.getNumBytes() differs from block.getNumBytes(). The NameNode-supplied ExtendedBlock is the authority on expected length; a local replica with a different on-disk length indicates the replica is stale, truncated, or from a different write generation, and copying/moving it would propagate corruption.","triggerScenarios":"A storage-policy move requested for a replica whose recorded length disagrees with the NameNode's block length: truncated by a dying disk, stale after append recovery bumped length elsewhere, replica resurrected from a bad volume, or race between an updateBlock and the move request.","commonSituations":"Bit-rot/truncation on one DN while other replicas are healthy; move racing an append/lease recovery that changed the block length; restored-from-backup data dirs with older replica generations.","solutions":["Validate the file: hdfs fsck <file> -files -blocks -locations; if other replicas are healthy, delete/evict the mismatched replica (hdfs debug evacuateBlock or let the scanner invalidate it) and let re-replication replace it.","Compare lengths: the message prints replica length vs expected — a smaller local length is classic truncation.","Do not retry the move for this replica; the mismatch will not self-heal and retrying just re-throws.","If all replicas mismatch, restore the file from snapshot/backup — the NameNode's expected length is unrecoverable locally."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Compare lengths before moving; mismatch means corruption, never retry.\nReplicaInfo info = fsDataset.getReplicaInfo(\n    block.getBlockPoolId(), block.getBlockId());\nif (info.getNumBytes() != block.getNumBytes()) {\n  reportCorruptReplica(block, info.getNumBytes());\n  continue; // do NOT move a length-mismatched replica\n}","typeGuard":null,"tryCatchPattern":"// Length mismatch is non-retryable: quarantine replica, re-replicate.\ntry {\n  fsDataset.moveBlockAcrossStorage(block, target, null);\n} catch (IOException e) {\n  if (String.valueOf(e.getMessage()).startsWith(\"Corrupted replica\")) {\n    reportCorruptToNameNode(dfsClient, block, e);\n    return; // re-replication replaces it; retrying the move is wrong\n  }\n  throw e;\n}","preventionTips":["Always validate replica length against the NameNode's expected length before storage moves.","On mismatch, report a corrupt replica (dfsClient.reportCorruptReplica) instead of retrying.","Monitor disk health (SMART, DN volume metrics) — truncation is usually a dying-disk symptom."],"tags":["hadoop","hdfs","datanode","fsdataset","mover","corruption","block-length"],"backgroundTag":"block-length-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}