apache/hadoop · error · IOException

Source ${src} Destination ${dst} both should be either file

Error message

Source ${src} Destination ${dst} both should be either file or directory

What it means

Inside the deprecated options-aware rename: the destination exists but srcStatus.isDirectory() != dstStatus.isDirectory(), so the operation is rejected with IOException('Source ... Destination ... both should be either file or directory'). Overwriting cannot change the node type, so this check runs before the OVERWRITE flag is even honored.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java:1695

    boolean overwrite = false;
    if (null != options) {
      for (Rename option : options) {
        if (option == Rename.OVERWRITE) {
          overwrite = true;
        }
      }
    }

    FileStatus dstStatus;
    try {
      dstStatus = getFileLinkStatus(dst);
    } catch (IOException e) {
      dstStatus = null;
    }
    if (dstStatus != null) {
      if (srcStatus.isDirectory() != dstStatus.isDirectory()) {
        throw new IOException("Source " + src + " Destination " + dst
            + " both should be either file or directory");
      }
      if (!overwrite) {
        throw new FileAlreadyExistsException("rename destination " + dst
            + " already exists.");
      }
      // Delete the destination that is a file or an empty directory
      if (dstStatus.isDirectory()) {
        FileStatus[] list = listStatus(dst);
        if (list != null && list.length != 0) {
          throw new IOException(
              "rename cannot overwrite non empty destination directory " + dst);
        }
      }
      delete(dst, false);
    } else {
      final Path parent = dst.getParent();
      final FileStatus parentStatus = getFileStatus(parent);

View on GitHub (pinned to 2add963021)

Solutions

1. Make the types agree: delete the mismatched destination first (delete(dst, true) for directories)
2. Verify dst's type before relying on OVERWRITE for file-replaces-file semantics
3. Branch on the two statuses yourself so the mismatch is a handled case, not an exception

Example fix

// before
fs.rename(srcDir, dstFile, Rename.OVERWRITE); // type mismatch

// after
FileStatus d = fs.exists(dst) ? fs.getFileStatus(dst) : null;
if (d != null && d.isDirectory() != fs.getFileStatus(src).isDirectory()) {
  fs.delete(dst, true);
}
fs.rename(src, dst, Rename.OVERWRITE);

FileStatus s = fs.getFileStatus(src);
if (fs.exists(dst)) {
  FileStatus d = fs.getFileStatus(dst);
  if (d.isDirectory() != s.isDirectory()) {
    fs.delete(dst, true); // types must agree before OVERWRITE can apply
  }
}
fs.rename(src, dst, Rename.OVERWRITE);

When it happens

Trigger: rename(file -> existing directory) or rename(directory -> existing file), with or without Rename.OVERWRITE.

Common situations: Job reruns where a previous run's output changed shape (single marker file vs directory of part-files); publish code renaming a directory onto a path that used to hold a file.

Related errors

• Cannot overwrite an existing file: %s
• Cannot rename because path does not exist: %s
• Rename unsuccessful : '%s' to '%s'
• The source {src} and destination {dst} are the same
• rename source ${src} not found.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e98ea5897c40f8fb. Report an issue: GitHub.