apache/hadoop · error · IOException

Rename of {} to {} is not allowed. The number of remote loca

Error message

Rename of {} to {} is not allowed. The number of remote locations for both source and target should be same.

What it means

In RouterAsyncClientProtocol.rename (async mode), when the destination is a multi-destination directory (isMultiDestDirectory true, i.e. the mount is 'ALL' ordered and the dir exists in the subclusters), the Router fans the rename out to every destination. It first asserts locs.size() == srcLocations.size(); if the destination mount and source mount have different numbers of remote locations, the fan-out cannot be paired one-to-one and IOException is thrown with both paths. This protects against partial renames across mismatched multi-destination mounts.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/async/RouterAsyncClientProtocol.java:263

        rpcServer.getLocationsForPath(src, true, false);
    final List<RemoteLocation> dstLocations =
        rpcServer.getLocationsForPath(dst, false, false);
    // srcLocations may be trimmed by getRenameDestinations()
    final List<RemoteLocation> locs = new LinkedList<>(srcLocations);
    RemoteParam dstParam = getRenameDestinations(locs, dstLocations);
    if (locs.isEmpty()) {
      asyncComplete(
          rbfRename.routerFedRename(src, dst, srcLocations, dstLocations));
      return asyncReturn(Boolean.class);
    }
    RemoteMethod method = new RemoteMethod("rename",
        new Class<?>[] {String.class, String.class},
        new RemoteParam(), dstParam);
    isMultiDestDirectory(src);
    asyncApply((AsyncApplyFunction<Boolean, Boolean>) isMultiDestDirectory -> {
      if (isMultiDestDirectory) {
        if (locs.size() != srcLocations.size()) {
          throw new IOException("Rename of " + src + " to " + dst + " is not"
              + " allowed. The number of remote locations for both source and"
              + " target should be same.");
        }
        rpcClient.invokeAll(locs, method);
      } else {
        rpcClient.invokeSequential(locs, method, Boolean.class,
            Boolean.TRUE);
      }
    });
    return asyncReturn(Boolean.class);
  }

  @Override
  public void rename2(
      final String src, final String dst,
      final Options.Rename... options) throws IOException {
    rpcServer.checkOperation(NameNode.OperationCategory.WRITE);

View on GitHub (pinned to 2add963021)

Solutions

  1. Align the mount table entries so source and destination mounts list the same number (and order) of destinations: hdfs dfsrouteradmin -add /src ns1,ns2 and /dst ns1,ns2, then refresh
  2. Rename to a destination under a single-destination mount, or do a two-step rename through such a path
  3. Audit with hdfs dfsrouteradmin -ls to see the destination lists of both mounts before retrying
Defensive patterns

Strategy: try-catch

Validate before calling

// Before rename across multi-dest mounts, compare destination counts:
// hdfs dfsrouteradmin -ls /srcMount  and  /dstMount
// must show the same number of destinations in the same order.

Try / catch

catch (IOException e) {
  if (e.getMessage().contains("number of remote locations")) {
    // align mount entries or route the rename through a single-dest path
  }
}

Prevention

When it happens

Trigger: rename(src, dst) through an async-mode Router where dst's parent is a multi-destination mount (e.g. 3 destinations) and src's mount has a different destination count (e.g. 1), or vice versa.

Common situations: Mount table evolution: one multi-dest mount extended with another nameservice while the related mount was not; ops teams adding standby namespaces to only one side of a mirrored pair.

Related errors


AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0bf20a422fd8ffe5. Report an issue: GitHub.