apache/hadoop · error · PathExistsException

File exists

Error message

File exists

What it means

MoveFromLocal.processPath (MoveCommands.java:65) throws PathExistsException ('File exists' is its default message) when the destination exists and is a directory. Unlike copy (put), moveFromLocal does not merge sources into an existing destination directory — the comment reads 'unlike copy, don't merge existing dirs during move'.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/MoveCommands.java:65

        "[-f] [-p] [-l] [-d] <localsrc> ... <dst>";
    public static final String DESCRIPTION = 
        "Same as -put, except that the source is " +
        "deleted after it's copied\n" +
        "and -t option has not yet implemented.";

    @Override
    protected void processOptions(LinkedList<String> args) throws IOException {
      if(args.contains("-t")) {
        throw new CommandFormat.UnknownOptionException("-t");
      }
      super.processOptions(args);
    }

    @Override
    protected void processPath(PathData src, PathData target) throws IOException {
      // unlike copy, don't merge existing dirs during move
      if (target.exists && target.stat.isDirectory()) {
        throw new PathExistsException(target.toString());
      }
      super.processPath(src, target);
    }
    
    @Override
    protected void postProcessPath(PathData src) throws IOException {
      if (!src.fs.delete(src.path, false)) {
        // we have no way to know the actual error...
        PathIOException e = new PathIOException(src.toString());
        e.setOperation("remove");
        throw e;
      }
    }
  }

  /**
   *  Move remote files to a local filesystem
   */

View on GitHub (pinned to 2add963021)

Solutions

1. Target a full destination file path instead of the directory: 'hadoop fs -moveFromLocal localsrc /existing/dir/newname'
2. Delete or rename the existing destination directory first if a fresh move is intended
3. Switch to 'hadoop fs -put localsrc /existing/dir' (merges) and delete the local source afterwards

Example fix

# before
hadoop fs -moveFromLocal app.log /data/logs   # /data/logs exists -> File exists

# after
hadoop fs -moveFromLocal app.log /data/logs/app.log

# resolve a concrete destination file path before moving
DEST_DIR=/data/logs
DEST="$DEST_DIR/$(basename "$LOCAL_SRC")"
hadoop fs -test -d "$DEST_DIR" && hadoop fs -moveFromLocal "$LOCAL_SRC" "$DEST"

catch (PathExistsException e) { pick a unique destination name (suffix timestamp/UUID) or delete the existing target, then retry }

When it happens

Trigger: Running 'hadoop fs -moveFromLocal localsrc /existing/dir' where /existing/dir already exists as a directory. Passing a destination that a previous run of the same script already created.

Common situations: Re-running an idempotent upload script whose second run finds the destination directory created by the first; operators assuming -moveFromLocal behaves like -put (which merges into existing dirs); destination directories pre-created by provisioning tooling.

Related errors

• Illegal option -t
• A directory with that name exists: %s
• {} already exists
• Failed to rename since the dest path %s already exists.
• File already exists: ${path}. Append or overwrite option mus

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b6ecb2913720a25b. Report an issue: GitHub.