{"record":{"id":"dca9a9e912d5947f","repo":"apache/hadoop","slug":"numbytes-numbytes-visible-visible-temp","errorCode":null,"errorMessage":"{numBytes} = numBytes < visible = {visible}, temp={temp}","messagePattern":"(.+?) = numBytes < visible = (.+?), temp=(.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java","lineNumber":1843,"sourceCode":"              \"r.getState() != ReplicaState.TEMPORARY, r=\" + r);\n        }\n        temp = r;\n      }\n      // check generation stamp\n      if (temp.getGenerationStamp() != expectedGs) {\n        throw new ReplicaAlreadyExistsException(\n            \"temp.getGenerationStamp() != expectedGs = \" + expectedGs\n                + \", temp=\" + temp);\n      }\n\n      // TODO: check writer?\n      // set writer to the current thread\n      // temp.setWriter(Thread.currentThread());\n\n      // check length\n      final long numBytes = temp.getNumBytes();\n      if (numBytes < visible) {\n        throw new IOException(numBytes + \" = numBytes < visible = \"\n            + visible + \", temp=\" + temp);\n      }\n      // check volume\n      final FsVolumeImpl v = (FsVolumeImpl) temp.getVolume();\n      if (v == null) {\n        throw new IOException(\"r.getVolume() = null, temp=\" + temp);\n      }\n\n      final ReplicaInPipeline rbw = v.convertTemporaryToRbw(b, temp);\n\n      if(rbw.getState() != ReplicaState.RBW) {\n        throw new IOException(\"Expected replica state: \" + ReplicaState.RBW\n            + \" obtained \" + rbw.getState() + \" for converting block \"\n            + b);\n      }\n      // overwrite the RBW in the volume map\n      volumeMap.add(b.getBlockPoolId(), rbw.getReplicaInfo());\n      return rbw;","sourceCodeStart":1825,"sourceCodeEnd":1861,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java#L1825-L1861","documentation":"The conversion checks that the temporary replica holds at least the visible length advertised for the block (numBytes >= visible). If the temp replica holds fewer bytes, its data is incomplete relative to what the pipeline assumes readers can already see, so an IOException naming numBytes, visible and the replica is thrown.","triggerScenarios":"convertTemporaryToRbw where the on-disk temp replica is shorter than the ExtendedBlock's visible length - an interrupted replication transfer, a partially copied block, or a corrupted/truncated temp file.","commonSituations":"Interrupted datanode-to-datanode replication leaving a short temp replica; disk corruption trimming the block file; located blocks advertising a visible length beyond what this DN received.","solutions":["Retry the transfer/replication - the short temp replica is discarded and recopied from a healthy source","Invalidate the short temp replica so re-replication heals it","Run 'hdfs fsck' to verify the visible length against other replicas","Check DN disk health if short temp replicas recur"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"Replica r = fsDataset.getReplica(b.getBlockPoolId(), b.getBlockId());\nif (r != null && r.getNumBytes() < visible) {\n  discardTempAndRecopyFromSource(b); // local copy cannot cover the visible length\n  return;\n}\nfsDataset.convertTemporaryToRbw(b, visible);","typeGuard":"boolean coversVisibleLength(Replica r, long visible) {\n  return r != null && r.getNumBytes() >= visible;\n}","tryCatchPattern":"catch (IOException ioe) {\n  if (ioe.getMessage() != null && ioe.getMessage().contains(\"numBytes < visible\")) {\n    recopyReplicaFromHealthySource(b); // failed transfer: re-copy, never convert\n  } else { throw ioe; }\n}","preventionTips":["Verify replication transfers fully complete (byte counts match) before replicas join pipelines","Treat short temp replicas as failed transfers: re-copy rather than convert","Monitor DataNode disk health so truncated files are caught before they surface here"],"tags":["hdfs","datanode","block-write","length-mismatch","temporary-replica"],"backgroundTag":"hdfs-replica-length-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}