apache/hadoop · error · PathIOException

Does not match target filesystem

Error message

Does not match target filesystem

What it means

Rename (the 'mv' command) processPath (MoveCommands.java:118) builds 'scheme://host' strings for the source and destination FileSystems and throws PathIOException('Does not match target filesystem') when they differ. Rename is a metadata-only operation, so it cannot cross filesystem or cluster boundaries; the check prevents attempting it.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/MoveCommands.java:118

      "Move files that match the specified file pattern <src> " +
      "to a destination <dst>.  When moving multiple files, the " +
      "destination must be a directory.";

    @Override
    protected void processOptions(LinkedList<String> args) throws IOException {
      CommandFormat cf = new CommandFormat(2, Integer.MAX_VALUE);
      cf.parse(args);
      getRemoteDestination(args);
    }

    @Override
    protected void processPath(PathData src, PathData target) throws IOException {
      String srcUri = src.fs.getUri().getScheme() + "://" +
          src.fs.getUri().getHost();
      String dstUri = target.fs.getUri().getScheme() + "://" +
          target.fs.getUri().getHost();
      if (!srcUri.equals(dstUri)) {
        throw new PathIOException(src.toString(),
            "Does not match target filesystem");
      }
      if (target.exists) {
        throw new PathExistsException(target.toString());
      }
      if (!target.fs.rename(src.path, target.path)) {
        // we have no way to know the actual error...
        throw new PathIOException(src.toString());
      }
    }
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Use copy-then-delete for cross-filesystem moves: 'hadoop fs -cp -f <src> <dst> && hadoop fs -rm <src>'
2. Use DistCp (hadoop distcp) for cross-cluster or large-scale moves
3. Ensure both sides of a same-cluster mv resolve to the same scheme://host — drop explicit URIs and rely on the default filesystem, or use identical HA nameservice identifiers

Example fix

# before
hadoop fs -mv /staging/part-0007 s3a://archive/bucket/part-0007
# mv: `/staging/part-0007': Does not match target filesystem

# after
hadoop fs -cp -f /staging/part-0007 s3a://archive/bucket/part-0007 \
  && hadoop fs -rm /staging/part-0007

// same-filesystem guard before rename
URI s = srcFs.getUri(), d = dstFs.getUri();
boolean sameFs = Objects.equals(s.getScheme(), d.getScheme())
    && Objects.equals(s.getHost(), d.getHost());
if (!sameFs) { /* copy + delete instead of rename */ }

catch (PathIOException e) when 'Does not match target filesystem' -> switch to copyThenDelete(src, dst) (fs.copy + fs.delete) or DistCp for bulk

When it happens

Trigger: 'hadoop fs -mv /path file:///local/path', 'hadoop fs -mv hdfs://ns1/path hdfs://ns2/path' (different nameservices/HA namespaces), moving between the default FS and an explicitly-qualified different cluster URI, or between S3A/scheme-mounted mounts and HDFS. The comparison uses scheme+host only, so two mounts of the same cluster via different HA aliases also mismatch.

Common situations: Data migration scripts that parameterize source/dest URIs and occasionally span clusters; views/mount-table configurations (HDFS Router Federation) where src and dst resolve to different namespaces; mixing relative paths (default fs) with fully-qualified URIs of another cluster.

Related errors

• Input/output error
• Source path and dest path refer the same file or directory
• It is not allowed to rename a parent directory: %s to its su
• File: %s already exists
• Cannot rename %s to %s, %s is a file

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/5d38b08d1c6eb073. Report an issue: GitHub.