apache/hadoop · error · IOException

Rename of {} to {} is not allowed. The remote location shoul

Error message

Rename of {} to {} is not allowed. The remote location should be exactly one.

What it means

Router federation rename requires exactly one remote location for each side of the rename, because it builds a single distcp job from one source nameservice to one destination nameservice. If the mount table yields multiple RemoteLocations for src or dst (multi-destination mount entries, e.g. HASH-based or ordered fallback lists), RouterFederationRename.routerFedRename throws this IOException.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterFederationRename.java:105

   * Router federation rename across namespaces.
   *
   * @param src the source path. There is no mount point under the src path.
   * @param dst the dst path.
   * @param srcLocations the remote locations of src.
   * @param dstLocations the remote locations of dst.
   * @throws IOException if rename fails.
   * @return true if rename succeeds.
   */
  public boolean routerFedRename(final String src, final String dst,
      final List<RemoteLocation> srcLocations,
      final List<RemoteLocation> dstLocations) throws IOException {
    if (!rpcServer.isEnableRenameAcrossNamespace()) {
      throw new IOException("Rename of " + src + " to " + dst
          + " is not allowed, no eligible destination in the same namespace was"
          + " found");
    }
    if (srcLocations.size() != 1 || dstLocations.size() != 1) {
      throw new IOException("Rename of " + src + " to " + dst + " is not"
          + " allowed. The remote location should be exactly one.");
    }
    RemoteLocation srcLoc = srcLocations.get(0);
    RemoteLocation dstLoc = dstLocations.get(0);
    checkSnapshotPath(srcLoc, dstLoc);
    checkPermission(srcLoc, dstLoc);

    UserGroupInformation routerUser = UserGroupInformation.getLoginUser();

    try {
      // as router user with saveJournal and task submission privileges
      return routerUser.doAs((PrivilegedExceptionAction<Boolean>) () -> {
        // Build and submit router federation rename job.
        BalanceJob job = buildRouterRenameJob(srcLoc.getNameserviceId(),
            dstLoc.getNameserviceId(), srcLoc.getDest(), dstLoc.getDest());
        BalanceProcedureScheduler scheduler = rpcServer.getFedRenameScheduler();
        countIncrement();
        try {

View on GitHub (pinned to 2add963021)

Solutions

  1. Check the mount table: hdfs dfsrouteradmin -listMountTable, and find entries producing >1 location for the src/dst paths
  2. Make the relevant mount entries single-destination (one -ns/-dst pair) or rename from a path that maps to exactly one namespace
  3. Restructure nested mounts so only the most specific entry matches the paths involved in the rename

Example fix

# before: multi-destination mount produces several RemoteLocations
hdfs dfsrouteradmin -add /data -ns ns0,ns1 -dst /data,\/data -order HASH
# after: single destination per mount
hdfs dfsrouteradmin -rm /data
hdfs dfsrouteradmin -add /data -ns ns0 -dst /data
Defensive patterns

Strategy: validation

Validate before calling

// Reject paths that map to multiple destinations before calling rename
List<RemoteLocation> srcLocs = resolver.getLocationsForPath(src, false);
List<RemoteLocation> dstLocs = resolver.getLocationsForPath(dst, false);
if (srcLocs.size() != 1 || dstLocs.size() != 1) {
  throw new IllegalArgumentException(
      "Rename requires exactly one remote location per side; got src="
      + srcLocs.size() + " dst=" + dstLocs.size());
}

Type guard

boolean isSingleDestination(SubclusterResolver resolver, String path) throws IOException {
  return resolver.getLocationsForPath(path, false).size() == 1;
}

Try / catch

try {
  boolean ok = dfs.rename(src, dst);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("remote location should be exactly one")) {
    // pick a path under a single-destination mount or reduce the mount entry's destinations
  }
  throw e;
}

Prevention

When it happens

Trigger: routerFedRename is invoked (cross-namespace rename enabled) and subclusterResolver.getLocationsForPath(src,dst) returns a list whose size != 1 for either side: the path maps to a mount entry with multiple destinations, or overlapping mount entries produce several locations.

Common situations: Mount table entries configured with multiple destination namespaces for load balancing/failover; nested mount entries where a parent and child entry both match; attempting fed rename on a path under a HASH-based multi-destination mount.

Related errors


AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/c5799212dc05dc48. Report an issue: GitHub.