apache/hadoop · error · ParentNotDirectoryException

{} is not a directory

Error message

{} is not a directory

What it means

In the POSIX-protocol rename path, an ObsException with code CONFLICT is caught and explained: the code stats the destination parent and throws ParentNotDirectoryException('<parent> is not a directory') when that parent exists as a file. Hadoop requires every destination ancestor to be a directory, and the OBS POSIX rename refused the operation for exactly that reason.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSPosixBucketUtils.java:160

                  + "file or directory: {}", dstPath);
          return true;
        } else {
          LOG.error("rename: failed to rename " + src + " to "
              + dstPath
              + " because destination exists");
          return false;
        }
      }
    } catch (FileNotFoundException e) {
      // if destination does not exist, do not change the
      // destination key, and just do rename.
      LOG.debug("rename: dest [{}] does not exist", dstPath);
    } catch (FileConflictException e) {
      Path parent = dstPath.getParent();
      if (!OBSCommonUtils.pathToKey(owner, parent).isEmpty()) {
        FileStatus dstParentStatus = owner.getFileStatus(parent);
        if (!dstParentStatus.isDirectory()) {
          throw new ParentNotDirectoryException(
              parent + " is not a directory");
        }
      }
    }

    if (dstKey.startsWith(srcKey) && (dstKey.equals(srcKey)
        || dstKey.charAt(srcKey.length()) == Path.SEPARATOR_CHAR)) {
      LOG.error("rename: dest [{}] cannot be a descendant of src [{}]",
          dstPath, src);
      return false;
    }

    return innerFsRenameWithRetry(owner, src, dstPath, srcKey, dstKey);
  }

  private static boolean innerFsRenameWithRetry(final OBSFileSystem owner,
      final Path src,
      final Path dst, final String srcKey, final String dstKey)

View on GitHub (pinned to 2add963021)

Solutions

1. Validate that every ancestor of the destination is a directory before rename
2. Delete the file blocking the path, then create the directory chain with mkdirs
3. Adopt a layout rule that never reuses a file name as a directory component

Example fix

// before
fs.rename(src, new Path("/marker/out")); // /marker is a file

// after
Path parent = new Path("/marker");
if (fs.exists(parent) && !fs.getFileStatus(parent).isDirectory()) {
  fs.delete(parent, false);
  fs.mkdirs(parent);
}
fs.rename(src, new Path("/marker/out"));

Path parent = dst.getParent();
if (parent != null && !parent.isRoot() && fs.exists(parent)
    && !fs.getFileStatus(parent).isDirectory()) {
  throw new ParentNotDirectoryException("Parent is a file: " + parent);
}

static boolean isDirectoryPath(FileSystem fs, Path p) throws IOException {
  return !fs.exists(p) || fs.getFileStatus(p).isDirectory();
}

try {
  fs.rename(src, dst);
} catch (ParentNotDirectoryException e) {
  // an ancestor of dst exists as a file: delete it or choose another layout
}

When it happens

Trigger: rename to /plainfile/out where /plainfile is a regular object; target paths where one component was previously written as a file; mixing marker-file conventions with directory-based layouts in one POSIX bucket.

Common situations: File and folder name collisions in POSIX-layout buckets; tools that create zero-byte markers and later treat the same names as directories; data restored or copied with a flattened structure.

Related errors

• destination parent [{}] is not a directory
• destination has no parent
• No such file or directory: {}
• File conflicts during rename, {}
• {} is root directory

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3247741aa01e5e96. Report an issue: GitHub.