apache/hadoop · error · PathOperationException

Operation not supported

Error message

Operation not supported

What it means

Thrown by CommandWithDestination.processPath(src, dst) when the source's FileStatus is a symlink. FsShell's copy pipeline has no strategy for symlinks (the TODO in the source says it must either copy the link or dereference it once FileContext is supported), so PathOperationException is thrown with its built-in message 'src: Operation not supported'.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java:291

    super.processPathArgument(src);
  }

  @Override
  protected void processPath(PathData src) throws IOException {
    processPath(src, getTargetPath(src));
  }
  
  /**
   * Called with a source and target destination pair
   * @param src for the operation
   * @param dst for the operation
   * @throws IOException if anything goes wrong
   */
  protected void processPath(PathData src, PathData dst) throws IOException {
    if (src.stat.isSymlink()) {
      // TODO: remove when FileContext is supported, this needs to either
      // copy the symlink or deref the symlink
      throw new PathOperationException(src.toString());        
    } else if (src.stat.isFile()) {
      copyFileToTarget(src, dst);
    } else if (src.stat.isDirectory() && !isRecursive()) {
      throw new PathIsDirectoryException(src.toString());
    }
  }

  @Override
  protected void recursePath(PathData src) throws IOException {
    PathData savedDst = dst;
    try {
      // modify dst as we descend to append the basename of the
      // current directory being processed
      dst = getTargetPath(src);
      final boolean preserveRawXattrs =
          checkPathsForReservedRaw(src.path, dst.path);
      if (dst.exists) {
        if (!dst.stat.isDirectory()) {

View on GitHub (pinned to 2add963021)

Solutions

1. Resolve the symlink to its target path ('hdfs dfs -ls <dir>' shows 'link -> target') and copy the target directly
2. Exclude symlinks from the source set (filter in the driving script) before copying trees
3. For raw xattr-aware copies note /.reserved/raw paths have their own restrictions (see checkPathsForReservedRaw)

Example fix

# before
hdfs dfs -cp /warehouse/current /backup/
# after (current -> /warehouse/dt=2026-08-22)
hdfs dfs -cp /warehouse/dt=2026-08-22 /backup/

// dereference symlinks before handing paths to the copy pipeline
FileStatus st = fs.getFileStatus(src);
if (st.isSymlink()) { // FileContext-based check; with FileSystem use:
  // Path tgt = fs.resolvePath(src); // or read the link target via FileContext
  src = fs.resolvePath(src);
}

static boolean isSymlinkSource(FileSystem fs, Path p) throws IOException {
  FileStatus st = fs.getFileStatus(p);
  // FileStatus from FileSystem dereferences; use FileContext.lstat-style check or
  // heuristics: compare resolvePath(p) with p to detect traversal through a link
  return !fs.resolvePath(p).equals(fs.makeQualified(p));
}

try {
  shellRun("-cp", src, dst);
} catch (PathOperationException e) {
  // "Operation not supported" on symlink source: resolve link target and copy that
  Path target = fs.resolvePath(src);
  shellRun("-cp", target.toString(), dst);
}

When it happens

Trigger: 'hdfs dfs -cp hdfs://nn/user/me/link /dest' where 'link' is an HDFS symlink; copying a directory tree with -r that contains symlinks; getmerge/put-style flows whose source expansion resolves to a symlink.

Common situations: Symlinks created for 'latest' partition pointers or convenience aliases inside source trees; porting workflows from local rsync/scp which dereference symlinks by default.

Related errors

• No such file or directory
• Too many matches
• Is not a directory
• File exists
• Is a directory

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/62eeeeb8c2b5dce0. Report an issue: GitHub.