apache/hadoop · error · IOException

Failed to rename %s to %s, %s is a file

Error message

Failed to rename %s to %s, %s is a file

What it means

Thrown by AliyunOSSFileSystem.rename(): the destination path itself does not exist, so the code stats the destination's parent and requires it to be a directory. When that parent exists but is a file, the renamed object would land 'inside' a file key, which OSS cannot represent, so it throws IOException("Failed to rename %s to %s, %s is a file").

Source

Thrown at hadoop-tools/hadoop-aliyun/src/main/java/org/apache/hadoop/fs/aliyun/oss/AliyunOSSFileSystem.java:647

    Path parent = dstPath.getParent();
    while (parent != null && !srcPath.equals(parent)) {
      parent = parent.getParent();
    }
    if (parent != null) {
      return false;
    }
    FileStatus srcStatus = getFileStatus(srcPath);
    FileStatus dstStatus;
    try {
      dstStatus = getFileStatus(dstPath);
    } catch (FileNotFoundException fnde) {
      dstStatus = null;
    }
    if (dstStatus == null) {
      // If dst doesn't exist, check whether dst dir exists or not
      dstStatus = getFileStatus(dstPath.getParent());
      if (!dstStatus.isDirectory()) {
        throw new IOException(String.format(
            "Failed to rename %s to %s, %s is a file", srcPath, dstPath,
            dstPath.getParent()));
      }
    } else {
      if (srcStatus.getPath().equals(dstStatus.getPath())) {
        return !srcStatus.isDirectory();
      } else if (dstStatus.isDirectory()) {
        // If dst is a directory
        dstPath = new Path(dstPath, srcPath.getName());
        FileStatus[] statuses;
        try {
          statuses = listStatus(dstPath);
        } catch (FileNotFoundException fnde) {
          statuses = null;
        }
        if (statuses != null && statuses.length > 0) {
          // If dst exists and not a directory / not empty
          throw new FileAlreadyExistsException(String.format(

View on GitHub (pinned to 2add963021)

Solutions

  1. Delete or rename the file object at the parent path so the directory structure can exist, then retry the rename
  2. Correct the destination path if the parent was mistyped
  3. Create the destination directory first (fs.mkdirs(dst.getParent())) after resolving the conflict

Example fix

// before
fs.rename(new Path("oss://b/f"), new Path("oss://b/a/g")); // 'a' is a file

// after
// resolve 'a': remove the file, then ensure the directory exists
fs.delete(new Path("oss://b/a"), false);
fs.mkdirs(new Path("oss://b/a"));
fs.rename(new Path("oss://b/f"), new Path("oss://b/a/g"));
Defensive patterns

Strategy: validation

Validate before calling

Path parent = dst.getParent();
if (!fs.exists(dst)) {
  if (!fs.exists(parent) || !fs.getFileStatus(parent).isDirectory()) {
    throw new IOException("rename dst parent is not a directory: " + parent);
  }
}
fs.rename(src, dst);

Try / catch

catch (IOException e) { if (e.getMessage().contains("is a file")) { /* clean the file at dst parent or fix dst path */ } throw e; }

Prevention

When it happens

Trigger: Calling rename(src, dst) where dst is absent and dst.getParent() exists as a file object — e.g., rename(oss://bucket/f, oss://bucket/a/g) while 'a' is a file; typically from committers or Hive moving files into partition paths.

Common situations: Partition directories whose prefix is occupied by an old data file; staging/commit moves into paths whose parents were written as files by other tools; renaming to a mistyped destination under a file-like prefix.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/48faef9e9e8732dc. Report an issue: GitHub.