apache/hadoop · error · HadoopIllegalArgumentException

Can't rename across file systems.

Error message

Can't rename across file systems.

What it means

ViewDistributedFileSystem.renameWithOptions resolves the mount path info for both src and dst and throws HadoopIllegalArgumentException("Can't rename across file systems.") when the target filesystem URIs differ. Rename can only be delegated atomically to a single cluster, so cross-cluster renames are rejected up front.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/ViewDistributedFileSystem.java:550

  @SuppressWarnings("deprecation")
  @Override
  public void rename(Path src, Path dst, final Options.Rename... options)
      throws IOException {
    if (this.vfs == null) {
      super.rename(src, dst, options);
      return;
    }

    ViewFileSystemOverloadScheme.MountPathInfo<FileSystem> mountSrcPathInfo =
        this.vfs.getMountPathInfo(src, getConf());

    ViewFileSystemOverloadScheme.MountPathInfo<FileSystem> mountDstPathInfo =
        this.vfs.getMountPathInfo(dst, getConf());

    //Check both in same cluster.
    if (!mountSrcPathInfo.getTargetFs().getUri()
        .equals(mountDstPathInfo.getTargetFs().getUri())) {
      throw new HadoopIllegalArgumentException(
          "Can't rename across file systems.");
    }

    FileUtil.rename(mountSrcPathInfo.getTargetFs(),
        mountSrcPathInfo.getPathOnTarget(), mountDstPathInfo.getPathOnTarget(),
        options);
  }

  @Override
  public boolean truncate(final Path f, final long newLength)
      throws IOException {
    if (this.vfs == null) {
      return super.truncate(f, newLength);
    }
    return this.vfs.truncate(f, newLength);
  }

  public boolean delete(final Path f, final boolean recursive)

View on GitHub (pinned to 2add963021)

Solutions

1. Keep rename source and destination under the same mount (same target cluster)
2. For cross-cluster moves use distcp, or FileUtil.copy followed by delete, and accept the non-atomic window
3. Pre-check both paths with getMountPathInfo and compare target URIs before renaming

Example fix

// before
vfs.rename(src, dst, Rename.OVERWRITE);

// after: explicit cross-cluster move
ViewFileSystemOverloadScheme.MountPathInfo<FileSystem> s =
    ((ViewDistributedFileSystem) vfs).getMountPathInfo(src, conf);
ViewFileSystemOverloadScheme.MountPathInfo<FileSystem> d =
    ((ViewDistributedFileSystem) vfs).getMountPathInfo(dst, conf);
FileUtil.copy(s.getTargetFs(), s.getPathOnTarget(),
    d.getTargetFs(), d.getPathOnTarget(), true, true, conf);
s.getTargetFs().delete(s.getPathOnTarget(), true);

ViewDistributedFileSystem vdfs = (ViewDistributedFileSystem) vfs;
URI srcFs = vdfs.getMountPathInfo(src, conf).getTargetFs().getUri();
URI dstFs = vdfs.getMountPathInfo(dst, conf).getTargetFs().getUri();
if (!srcFs.equals(dstFs)) {
  // cross-cluster: fall back to copy + delete
  FileUtil.copy(...);
}

static boolean sameTargetCluster(FileSystem vfs, Path a, Path b,
    Configuration conf) {
  if (!(vfs instanceof ViewDistributedFileSystem)) return true;
  ViewDistributedFileSystem v = (ViewDistributedFileSystem) vfs;
  return v.getMountPathInfo(a, conf).getTargetFs().getUri()
      .equals(v.getMountPathInfo(b, conf).getTargetFs().getUri());
}

try {
  vfs.rename(src, dst, Rename.OVERWRITE);
} catch (HadoopIllegalArgumentException e) {
  if ("Can't rename across file systems.".equals(e.getMessage())) {
    copyThenDelete(src, dst); // non-atomic fallback
  } else {
    throw e;
  }
}

When it happens

Trigger: Renaming between mounts that resolve to different clusters (hdfs://cluster1/user/a to hdfs://cluster2/user/b), including clusters with the same content but different addressed namespaces.

Common situations: Federated viewfs namespaces; scripts moving data between clusters through the view; destination paths that silently resolve onto another mount due to mount-table overlap.

Related errors

• Rename of {} to {} is not allowed, no eligible destination i
• Rename of {} to {} is not allowed. The remote location shoul
• Rename of {} to {} failed.
• Permission denied rename {}({}) to {}({}) Reason={}
• Router Federation Rename is interrupted while checking permi

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/33f51be7630caa51. Report an issue: GitHub.