{"record":{"id":"62eeeeb8c2b5dce0","repo":"apache/hadoop","slug":"operation-not-supported","errorCode":null,"errorMessage":"Operation not supported","messagePattern":"Operation not supported","errorType":"exception","errorClass":"PathOperationException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java","lineNumber":291,"sourceCode":"    super.processPathArgument(src);\n  }\n\n  @Override\n  protected void processPath(PathData src) throws IOException {\n    processPath(src, getTargetPath(src));\n  }\n  \n  /**\n   * Called with a source and target destination pair\n   * @param src for the operation\n   * @param dst for the operation\n   * @throws IOException if anything goes wrong\n   */\n  protected void processPath(PathData src, PathData dst) throws IOException {\n    if (src.stat.isSymlink()) {\n      // TODO: remove when FileContext is supported, this needs to either\n      // copy the symlink or deref the symlink\n      throw new PathOperationException(src.toString());        \n    } else if (src.stat.isFile()) {\n      copyFileToTarget(src, dst);\n    } else if (src.stat.isDirectory() && !isRecursive()) {\n      throw new PathIsDirectoryException(src.toString());\n    }\n  }\n\n  @Override\n  protected void recursePath(PathData src) throws IOException {\n    PathData savedDst = dst;\n    try {\n      // modify dst as we descend to append the basename of the\n      // current directory being processed\n      dst = getTargetPath(src);\n      final boolean preserveRawXattrs =\n          checkPathsForReservedRaw(src.path, dst.path);\n      if (dst.exists) {\n        if (!dst.stat.isDirectory()) {","sourceCodeStart":273,"sourceCodeEnd":309,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/CommandWithDestination.java#L273-L309","documentation":"Thrown by CommandWithDestination.processPath(src, dst) when the source's FileStatus is a symlink. FsShell's copy pipeline has no strategy for symlinks (the TODO in the source says it must either copy the link or dereference it once FileContext is supported), so PathOperationException is thrown with its built-in message 'src: Operation not supported'.","triggerScenarios":"'hdfs dfs -cp hdfs://nn/user/me/link /dest' where 'link' is an HDFS symlink; copying a directory tree with -r that contains symlinks; getmerge/put-style flows whose source expansion resolves to a symlink.","commonSituations":"Symlinks created for 'latest' partition pointers or convenience aliases inside source trees; porting workflows from local rsync/scp which dereference symlinks by default.","solutions":["Resolve the symlink to its target path ('hdfs dfs -ls <dir>' shows 'link -> target') and copy the target directly","Exclude symlinks from the source set (filter in the driving script) before copying trees","For raw xattr-aware copies note /.reserved/raw paths have their own restrictions (see checkPathsForReservedRaw)"],"exampleFix":"# before\nhdfs dfs -cp /warehouse/current /backup/\n# after (current -> /warehouse/dt=2026-08-22)\nhdfs dfs -cp /warehouse/dt=2026-08-22 /backup/","handlingStrategy":"validation","validationCode":"// dereference symlinks before handing paths to the copy pipeline\nFileStatus st = fs.getFileStatus(src);\nif (st.isSymlink()) { // FileContext-based check; with FileSystem use:\n  // Path tgt = fs.resolvePath(src); // or read the link target via FileContext\n  src = fs.resolvePath(src);\n}","typeGuard":"static boolean isSymlinkSource(FileSystem fs, Path p) throws IOException {\n  FileStatus st = fs.getFileStatus(p);\n  // FileStatus from FileSystem dereferences; use FileContext.lstat-style check or\n  // heuristics: compare resolvePath(p) with p to detect traversal through a link\n  return !fs.resolvePath(p).equals(fs.makeQualified(p));\n}","tryCatchPattern":"try {\n  shellRun(\"-cp\", src, dst);\n} catch (PathOperationException e) {\n  // \"Operation not supported\" on symlink source: resolve link target and copy that\n  Path target = fs.resolvePath(src);\n  shellRun(\"-cp\", target.toString(), dst);\n}","preventionTips":["Resolve 'latest'-style symlink aliases to real partition paths before copying","Filter symlink entries when walking trees with listStatus isSymlink-aware logic","Remember FsShell copy refuses symlinks by design (see the TODO in CommandWithDestination.processPath)"],"tags":["hadoop","hdfs","symlink","unsupported-operation","copy","shell"],"backgroundTag":"operation-not-supported","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}