apache/hadoop · error · IOException

r.getVolume() = null, temp={temp}

Error message

r.getVolume() = null, temp={temp}

What it means

Last guard before the physical conversion: the temp replica must belong to a volume. If ReplicaInfo.getVolume() returns null - a state that should be impossible for a volumeMap entry - the DataNode throws IOException('r.getVolume() = null') instead of taking an NPE deeper in the conversion.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java:1849

        throw new ReplicaAlreadyExistsException(
            "temp.getGenerationStamp() != expectedGs = " + expectedGs
                + ", temp=" + temp);
      }

      // TODO: check writer?
      // set writer to the current thread
      // temp.setWriter(Thread.currentThread());

      // check length
      final long numBytes = temp.getNumBytes();
      if (numBytes < visible) {
        throw new IOException(numBytes + " = numBytes < visible = "
            + visible + ", temp=" + temp);
      }
      // check volume
      final FsVolumeImpl v = (FsVolumeImpl) temp.getVolume();
      if (v == null) {
        throw new IOException("r.getVolume() = null, temp=" + temp);
      }

      final ReplicaInPipeline rbw = v.convertTemporaryToRbw(b, temp);

      if(rbw.getState() != ReplicaState.RBW) {
        throw new IOException("Expected replica state: " + ReplicaState.RBW
            + " obtained " + rbw.getState() + " for converting block "
            + b);
      }
      // overwrite the RBW in the volume map
      volumeMap.add(b.getBlockPoolId(), rbw.getReplicaInfo());
      return rbw;
    } finally {
      if (dataNodeMetrics != null) {
        long convertTemporaryToRbwMs = Time.monotonicNow() - startTimeMs;
        dataNodeMetrics.addConvertTemporaryToRbwOp(convertTemporaryToRbwMs);
      }
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Check DataNode storage-directory health and logs - a failed/unwritable volume is the usual root cause
2. Restart the DataNode so the volumeMap is rebuilt from on-disk state
3. If a custom fsdataset is in use, ensure every replica added to the volumeMap carries its volume
4. If volumes keep failing, retire/replace the disk before HDFS marks replicas missing (dfs.datanode.failed.volumes.tolerated)

Replica r = fsDataset.getReplica(b.getBlockPoolId(), b.getBlockId());
if (r == null || r.getVolume() == null) {
  // volumeMap is unhealthy: fail over to other replicas and alert on the storage
  failOverToOtherReplicas();
  alertDatanodeStorageFailure();
  return;
}
fsDataset.convertTemporaryToRbw(b, visible);

boolean hasVolume(Replica r) {
  return r != null && r.getVolume() != null;
}

catch (IOException ioe) {
  if (ioe.getMessage() != null && ioe.getMessage().contains("getVolume() = null")) {
    failOverToOtherReplicas();
    alertDatanodeStorageFailure(); // in-memory replica state is inconsistent
  } else { throw ioe; }
}

When it happens

Trigger: A volumeMap entry whose replica lost its volume reference: a volume removed or failed underneath the replica, corrupted in-memory state after crash recovery, or a custom dataset inserting volume-less replicas.

Common situations: Storage-directory failure or disk loss mid-operation; volumeMap inconsistency after an unclean DN restart; custom FsDataset implementations not wiring volumes into ReplicaInfo.

Related errors

• CreateRBW returned a replica of state {state} for block {blo
• Failed to create temporary file for {}. File {} should not
• Failed to create temporary file for {}. File {} should be c
• Append on block {blockId} returned a replica of state {state
• Block {b} already exists in state {state} and thus cannot be

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1ecb1dd8a604155b. Report an issue: GitHub.