apache/hadoop · error · java.io.IOException

The destination cluster must be specified.

Error message

The destination cluster must be specified.

What it means

RouterFedBalance's BalanceJobBuilder.build() validates the fedbalance destination before constructing the job: new Path(inputDst).toUri().getAuthority() must be non-null, i.e. the destination must be a fully-qualified URI carrying a nameservice authority (hdfs://ns1/dest). A destination without an authority leaves FedBalance unable to tell which cluster receives the data, so build() throws this IOException at job construction time.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/rbfbalance/RouterFedBalance.java:167

    /**
     * Specify the threshold of diff entries.
     * @param value the threshold of a fast distcp.
     */
    public Builder setDiffThreshold(int value) {
      this.diffThreshold = value;
      return this;
    }

    /**
     * Build the balance job.
     */
    public BalanceJob build() throws IOException {
      // Construct job context.
      FedBalanceContext context;
      Path dst = new Path(inputDst);
      if (dst.toUri().getAuthority() == null) {
        throw new IOException("The destination cluster must be specified.");
      }
      Path src = getSrcPath(inputSrc);
      String mount = inputSrc;
      context = new FedBalanceContext.Builder(src, dst, mount, getConf())
          .setForceCloseOpenFiles(forceCloseOpen).setUseMountReadOnly(true)
          .setMapNum(map).setBandwidthLimit(bandwidth).setTrash(trashOpt)
          .setDelayDuration(delayDuration).setDiffThreshold(diffThreshold)
          .build();

      LOG.info(context.toString());
      // Construct the balance job.
      BalanceJob.Builder<BalanceProcedure> builder = new BalanceJob.Builder<>();
      RouterDistCpProcedure dcp =
          new RouterDistCpProcedure(DISTCP_PROCEDURE, null, delayDuration,
              context);
      builder.nextProcedure(dcp);
      MountTableProcedure mtp =
          new MountTableProcedure(MOUNT_TABLE_PROCEDURE, null, delayDuration,

View on GitHub (pinned to 2add963021)

Solutions

1. Specify the destination with an explicit nameservice authority: hdfs fedbalance -s /mount -d hdfs://ns1/target/path
2. Confirm the nameservice id matches one in the federation (hdfs dfsrouteradmin -list shows destination namespaces)
3. Re-run the command and confirm the job submits and the balance proceeds

Example fix

# before: destination has no authority → IOException("The destination cluster must be specified.")
hdfs fedbalance -s /apps/data -d /archive/data

# after: fully-qualified destination with nameservice authority
hdfs fedbalance -s /apps/data -d hdfs://ns1/archive/data

// Validate the fedbalance destination before building/submitting the job
Path dst = new Path(inputDst);
if (dst.toUri().getAuthority() == null) {
  throw new IllegalArgumentException(
      "Destination must include a nameservice, e.g. hdfs://ns1/path: " + inputDst);
}

When it happens

Trigger: Invoking `hdfs fedbalance -s /mount -d <dst>` (or the router balance client) with a destination like /dest, dest, or hdfs:///dest — any URI whose authority component is null.

Common situations: Users omit the nameservice on -d because the source is a router path and they assume the same namespace applies; scripts ported from DistCp invocations using unqualified paths.

Related errors

• The mount point doesn't exist. path=" + fedPath
• The mount point has more than one destination. path={}
• Mount table " + mount + " doesn't exist
• Failed update mount table " + mount
• Failed update mount table " + mount + " with readonly=" + re

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/c29b386928d8388c. Report an issue: GitHub.