apache/hadoop · error · ParentNotDirectoryException

is not a directory

Error message

{} is not a directory

What it means

In the POSIX-protocol rename path, an ObsException with code CONFLICT is caught and explained: the code stats the destination parent and throws ParentNotDirectoryException('<parent> is not a directory') when that parent exists as a file. Hadoop requires every destination ancestor to be a directory, and the OBS POSIX rename refused the operation for exactly that reason.

Solutions

1. Validate that every ancestor of the destination is a directory before rename
2. Delete the file blocking the path, then create the directory chain with mkdirs
3. Adopt a layout rule that never reuses a file name as a directory component

Example fix

// before
fs.rename(src, new Path("/marker/out")); // /marker is a file

// after
Path parent = new Path("/marker");
if (fs.exists(parent) && !fs.getFileStatus(parent).isDirectory()) {
  fs.delete(parent, false);
  fs.mkdirs(parent);
}
fs.rename(src, new Path("/marker/out"));

Path parent = dst.getParent();
if (parent != null && !parent.isRoot() && fs.exists(parent)
    && !fs.getFileStatus(parent).isDirectory()) {
  throw new ParentNotDirectoryException("Parent is a file: " + parent);
}

static boolean isDirectoryPath(FileSystem fs, Path p) throws IOException {
  return !fs.exists(p) || fs.getFileStatus(p).isDirectory();
}

try {
  fs.rename(src, dst);
} catch (ParentNotDirectoryException e) {
  // an ancestor of dst exists as a file: delete it or choose another layout
}

When it happens

Trigger: rename to /plainfile/out where /plainfile is a regular object; target paths where one component was previously written as a file; mixing marker-file conventions with directory-based layouts in one POSIX bucket.

Common situations: File and folder name collisions in POSIX-layout buckets; tools that create zero-byte markers and later treat the same names as directories; data restored or copied with a flattened structure.

Related errors

• destination has no parent
• destination parent [ ] is not a directory
• File conflicts during rename
• No such file or directory
• destination is an existed file

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3247741aa01e5e96. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSPosixBucketUtils.java:160

                  + "file or directory: {}", dstPath);
          return true;
        } else {
          LOG.error("rename: failed to rename " + src + " to "
              + dstPath
              + " because destination exists");
          return false;
        }
      }
    } catch (FileNotFoundException e) {
      // if destination does not exist, do not change the
      // destination key, and just do rename.
      LOG.debug("rename: dest [{}] does not exist", dstPath);
    } catch (FileConflictException e) {
      Path parent = dstPath.getParent();
      if (!OBSCommonUtils.pathToKey(owner, parent).isEmpty()) {
        FileStatus dstParentStatus = owner.getFileStatus(parent);
        if (!dstParentStatus.isDirectory()) {
          throw new ParentNotDirectoryException(
              parent + " is not a directory");
        }
      }
    }

    if (dstKey.startsWith(srcKey) && (dstKey.equals(srcKey)
        || dstKey.charAt(srcKey.length()) == Path.SEPARATOR_CHAR)) {
      LOG.error("rename: dest [{}] cannot be a descendant of src [{}]",
          dstPath, src);
      return false;
    }

    return innerFsRenameWithRetry(owner, src, dstPath, srcKey, dstKey);
  }

  private static boolean innerFsRenameWithRetry(final OBSFileSystem owner,
      final Path src,
      final Path dst, final String srcKey, final String dstKey)

View on GitHub (pinned to 2add963021)