apache/hadoop · error · IOException

The source cluster must be specified.

Error message

The source cluster must be specified.

What it means

Mirror of error 5049 in FedBalance.BalanceJob.Builder.build(): after validating dst, it parses inputSrc and requires a non-null authority, because fedbalance must know which cluster to snapshot, disable writes on, and read from. A src without scheme/authority (e.g. /data instead of hdfs://ns1/data) is rejected at build time.

Source

Thrown at hadoop-tools/hadoop-federation-balance/src/main/java/org/apache/hadoop/tools/fedbalance/FedBalance.java:162

     */
    public Builder setDiffThreshold(int value) {
      this.diffThreshold = value;
      return this;
    }

    /**
     * Build the balance job.
     */
    public BalanceJob build() throws IOException {
      // Construct job context.
      FedBalanceContext context;
      Path dst = new Path(inputDst);
      if (dst.toUri().getAuthority() == null) {
        throw new IOException("The destination cluster must be specified.");
      }
      Path src = new Path(inputSrc);
      if (src.toUri().getAuthority() == null) {
        throw new IOException("The source cluster must be specified.");
      }
      context = new FedBalanceContext.Builder(src, dst, NO_MOUNT, getConf())
          .setForceCloseOpenFiles(forceCloseOpen).setUseMountReadOnly(false)
          .setMapNum(map).setBandwidthLimit(bandwidth).setTrash(trashOpt)
          .setDiffThreshold(diffThreshold).build();

      LOG.info(context.toString());
      // Construct the balance job.
      BalanceJob.Builder<BalanceProcedure> builder = new BalanceJob.Builder<>();
      DistCpProcedure dcp =
          new DistCpProcedure(DISTCP_PROCEDURE, null, delayDuration, context);
      builder.nextProcedure(dcp);
      TrashProcedure tp =
          new TrashProcedure(TRASH_PROCEDURE, null, delayDuration, context);
      builder.nextProcedure(tp);
      return builder.build();
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Set -src to a fully-qualified URI: hdfs://<nameservice-or-namenode>:<port>/path.
2. Verify the src nameservice resolves from this client (hdfs getconf -namenodes or a listing of hdfs://ns1/).
3. Keep both -src and -dst fully qualified in scripts so neither check fires.

Example fix

# before
hdfs fedbalance -src /data -dst hdfs://ns2/balance/data

# after
hdfs fedbalance -src hdfs://ns1/data -dst hdfs://ns2/balance/data

String srcStr = "hdfs://ns1/data";
if (new Path(srcStr).toUri().getAuthority() == null) {
  throw new IllegalArgumentException("fedbalance -src must include the cluster authority, e.g. hdfs://ns1/path");
}

When it happens

Trigger: Passing -src /data or any Path whose toUri().getAuthority() is null to the fedbalance CLI; note dst is validated first, so if you see this message the dst URI was already well-formed.

Common situations: Habit from 'hdfs dfs' commands where fs.defaultFS supplies the cluster; scripts that build src from a relative config value; single-cluster testing reused for fedbalance.

Related errors

• The destination cluster must be specified.
• The destination cluster must be specified.
• {src} should be a directory.
• {dst} already exists.
• {src} shouldn't enable snapshot.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d543c27891947a7b. Report an issue: GitHub.