apache/hadoop · error · IOException

{numBytes} = numBytes < visible = {visible}, temp={temp}

Error message

{numBytes} = numBytes < visible = {visible}, temp={temp}

What it means

The conversion checks that the temporary replica holds at least the visible length advertised for the block (numBytes >= visible). If the temp replica holds fewer bytes, its data is incomplete relative to what the pipeline assumes readers can already see, so an IOException naming numBytes, visible and the replica is thrown.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java:1843

              "r.getState() != ReplicaState.TEMPORARY, r=" + r);
        }
        temp = r;
      }
      // check generation stamp
      if (temp.getGenerationStamp() != expectedGs) {
        throw new ReplicaAlreadyExistsException(
            "temp.getGenerationStamp() != expectedGs = " + expectedGs
                + ", temp=" + temp);
      }

      // TODO: check writer?
      // set writer to the current thread
      // temp.setWriter(Thread.currentThread());

      // check length
      final long numBytes = temp.getNumBytes();
      if (numBytes < visible) {
        throw new IOException(numBytes + " = numBytes < visible = "
            + visible + ", temp=" + temp);
      }
      // check volume
      final FsVolumeImpl v = (FsVolumeImpl) temp.getVolume();
      if (v == null) {
        throw new IOException("r.getVolume() = null, temp=" + temp);
      }

      final ReplicaInPipeline rbw = v.convertTemporaryToRbw(b, temp);

      if(rbw.getState() != ReplicaState.RBW) {
        throw new IOException("Expected replica state: " + ReplicaState.RBW
            + " obtained " + rbw.getState() + " for converting block "
            + b);
      }
      // overwrite the RBW in the volume map
      volumeMap.add(b.getBlockPoolId(), rbw.getReplicaInfo());
      return rbw;

View on GitHub (pinned to 2add963021)

Solutions

1. Retry the transfer/replication - the short temp replica is discarded and recopied from a healthy source
2. Invalidate the short temp replica so re-replication heals it
3. Run 'hdfs fsck' to verify the visible length against other replicas
4. Check DN disk health if short temp replicas recur

Replica r = fsDataset.getReplica(b.getBlockPoolId(), b.getBlockId());
if (r != null && r.getNumBytes() < visible) {
  discardTempAndRecopyFromSource(b); // local copy cannot cover the visible length
  return;
}
fsDataset.convertTemporaryToRbw(b, visible);

boolean coversVisibleLength(Replica r, long visible) {
  return r != null && r.getNumBytes() >= visible;
}

catch (IOException ioe) {
  if (ioe.getMessage() != null && ioe.getMessage().contains("numBytes < visible")) {
    recopyReplicaFromHealthySource(b); // failed transfer: re-copy, never convert
  } else { throw ioe; }
}

When it happens

Trigger: convertTemporaryToRbw where the on-disk temp replica is shorter than the ExtendedBlock's visible length - an interrupted replication transfer, a partially copied block, or a corrupted/truncated temp file.

Common situations: Interrupted datanode-to-datanode replication leaving a short temp replica; disk corruption trimming the block file; located blocks advertising a visible length beyond what this DN received.

Related errors

• Replica does not exist {b}
• r.getState() != ReplicaState.TEMPORARY, r={r}
• temp.getGenerationStamp() != expectedGs = {expectedGs}, temp
• Failed to create temporary file for {}. File {} should not
• Failed to create temporary file for {}. File {} should be c

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/dca9a9e912d5947f. Report an issue: GitHub.