apache/hadoop · error · IOException

{} does not support multiple targets {}

Error message

{} does not support multiple targets {}

What it means

IOException from DataXceiver.opWriteBlock when a block write arrives with a pipeline stage that is a datanode-to-datanode transfer (PIPELINE_SETUP_TRANSFER / TRANSFER_RBW / TRANSFER_FINALIZED, i.e. isTransfer == true) and the request carries more than zero downstream targets. Block transfers between DataNodes (used by rebalancing, stripe/mover operations, and replication recovery moves) move data to exactly one destination; a real client append/create pipeline is the only mode allowed to specify multiple targets.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java:737

    // To support older clients, we don't pass in empty storageIds
    final int nsi = targetStorageIds.length;
    final String[] storageIds;
    if (nsi > 0) {
      storageIds = new String[nsi + 1];
      storageIds[0] = storageId;
      if (targetStorageTypes.length > 0) {
        System.arraycopy(targetStorageIds, 0, storageIds, 1, nsi);
      }
    } else {
      storageIds = new String[0];
    }
    checkAccess(replyOut, isClient, block, blockToken, Op.WRITE_BLOCK,
        BlockTokenIdentifier.AccessMode.WRITE,
        storageTypes, storageIds);

    // check single target for transfer-RBW/Finalized
    if (isTransfer && targets.length > 0) {
      throw new IOException(stage + " does not support multiple targets "
          + Arrays.asList(targets));
    }

    if (LOG.isDebugEnabled()) {
      LOG.debug("opWriteBlock: stage={}, clientname={}\n  " +
              "block  ={}, newGs={}, bytesRcvd=[{}, {}]\n  " +
              "targets={}; pipelineSize={}, srcDataNode={}, pinning={}",
          stage, clientname, block, latestGenerationStamp, minBytesRcvd,
          maxBytesRcvd, Arrays.asList(targets), pipelineSize, srcDataNode,
          pinning);
      LOG.debug("isDatanode={}, isClient={}, isTransfer={}",
          isDatanode, isClient, isTransfer);
      LOG.debug("writeBlock receive buf size {} tcp no delay {}",
          peer.getReceiveBufferSize(), peer.getTcpNoDelay());
    }

    // We later mutate block's generation stamp and length, but we need to
    // forward the original version of the block to downstream mirrors, so

View on GitHub (pinned to 2add963021)

Solutions

1. Check for version mismatch between the source DataNode initiating the transfer and the rejecting DataNode; complete the rolling upgrade so both run the same release
2. Capture the stage and target list from the DataNode debug log (the LOG.debug right after the check prints stage, targets, pipelineSize) and compare against the sender's code path
3. If a custom client/balancer is in play, fix it to send an empty targets array for TRANSFER_RBW/TRANSFER_FINALIZED/PIPELINE_SETUP_TRANSFER stages
4. Retry the operation (balancer/mover) once versions are aligned; single-target transfer is retried per-block automatically

Example fix

// before (custom client / DN sending a transfer): targets populated -> DN throws
//   opWriteBlock(stage=TRANSFER_RBW, targets=[dn2, dn3])
new Sender(out).writeBlock(block, STORAGE_TYPE, STORAGE_ID, tokens,
    "source-datanode", 0, BLOCK_SIZE, 0, 0,
    new DatanodeInfo[]{dn2, dn3},      // WRONG on transfer
    new StorageType[]{DISK, DISK}, null, Stage.TRANSFER_RBW, ...);

// after: transfers carry no downstream targets
new Sender(out).writeBlock(block, STORAGE_TYPE, STORAGE_ID, tokens,
    "source-datanode", 0, BLOCK_SIZE, 0, 0,
    new DatanodeInfo[]{},               // transfer = zero targets
    new StorageType[]{}, null, Stage.TRANSFER_RBW, ...);

// Sender-side guard: only client stages may carry pipeline targets
boolean isTransfer = stage == BlockConstructionStage.TRANSFER_RBW
    || stage == BlockConstructionStage.TRANSFER_FINALIZED
    || stage == BlockConstructionStage.PIPELINE_SETUP_TRANSFER;
DatanodeInfo[] targetsToSend = isTransfer ? new DatanodeInfo[0] : targets;
// pass targetsToSend to Sender.writeBlock(...)

try {
  sendWriteBlock(stage, targets);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("does not support multiple targets")) {
    // protocol bug or version mismatch: retry the single-target transfer,
    // and align sender/receiver Hadoop versions
    sendWriteBlock(stage, new DatanodeInfo[0]);
  } else { throw e; }
}

When it happens

Trigger: A WRITE_BLOCK request with stage == PIPELINE_SETUP_TRANSFER (or TRANSFER_RBW/TRANSFER_FINALIZED) whose targets array is non-empty. Produced by a misbehaving or version-mismatched upstream DataNode/client that populates pipeline targets on a transfer request, e.g.dfsclient replication code sending a chain of targets where the transfer protocol expects none.

Common situations: Interoperability bugs between DataNode versions during a rolling upgrade (older sender includes targets the newer receiver rejects); patched or third-party clients (Erasure coding prototypes, balancer forks) that reuse the client write path for transfers; extremely rare on stock matching versions.

Related errors

• Unsupported protocol found when creating the proxy connectio
• Version Mismatch (Expected: {}, Received: {} )
• Unknown op {} in data stream
• Missing storageIDs: It is likely that the HDFS client, who m
• You cannot pass file descriptors over anything but a UNIX do

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/9e9db62387f5656c. Report an issue: GitHub.