What causes "Path is a file: {}" in apache/hadoop?

Pick a different path for the directory, or delete/rename the conflicting file first after confirming it is disposable.

apache/hadoop · error · FileAlreadyExistsException

Path is a file: {}

Error message

Path is a file: {}

What it means

OBSCommonUtils.innerMkdirs starts by stat-ing the requested path; if a FILE already exists at exactly that path, it throws FileAlreadyExistsException('Path is a file: <path>'). OBS has no real directories — the connector refuses to overwrite a key with a directory marker, because that would destroy data. This mirrors HDFS mkdirs semantics where mkdir over an existing file also fails.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSCommonUtils.java:720

   *
   * @param owner the owner OBSFileSystem instance
   * @param path  path to create
   * @return true if a directory was created
   * @throws FileAlreadyExistsException there is a file at the path specified
   * @throws IOException                other IO problems
   * @throws ObsException               on failures inside the OBS SDK
   */
  static boolean innerMkdirs(final OBSFileSystem owner, final Path path)
      throws IOException, FileAlreadyExistsException, ObsException {
    LOG.debug("Making directory: {}", path);
    FileStatus fileStatus;
    try {
      fileStatus = owner.getFileStatus(path);

      if (fileStatus.isDirectory()) {
        return true;
      } else {
        throw new FileAlreadyExistsException("Path is a file: " + path);
      }
    } catch (FileNotFoundException e) {
      Path fPart = path.getParent();
      do {
        try {
          fileStatus = owner.getFileStatus(fPart);
          if (fileStatus.isDirectory()) {
            break;
          }
          if (fileStatus.isFile()) {
            throw new FileAlreadyExistsException(
                String.format("Can't make directory for path '%s'"
                    + " since it is a file.", fPart));
          }
        } catch (FileNotFoundException fnfe) {
          LOG.debug("file {} not fount, but ignore.", path);
        }
        fPart = fPart.getParent();

View on GitHub (pinned to 2add963021)

Solutions

Pick a different path for the directory, or delete/rename the conflicting file first after confirming it is disposable.
Standardize your layout: never let the same key prefix be both a file and a directory across code paths.
Pre-check with fs.getFileStatus(path).isFile() before mkdirs and surface a clear error naming the file.
Clean stale outputs (mapreduce.output.fileoutputformat.outputdir) before job start instead of relying on mkdirs to overwrite.

Example fix

// before
fs.mkdirs(new Path("obs://mybucket/logs")); // object 'logs' already exists as file

// after
Path p = new Path("obs://mybucket/logs");
if (fs.exists(p) && !fs.getFileStatus(p).isDirectory()) {
  throw new IOException(p + " exists as a FILE; move or delete it first");
}
fs.mkdirs(p);

Defensive patterns

Strategy: validation

Validate before calling

static void ensureMkdirsSafe(FileSystem fs, Path p) throws IOException {
  if (fs.exists(p) && !fs.getFileStatus(p).isDirectory()) {
    throw new FileAlreadyExistsException("cannot mkdirs over file " + p);
  }
  fs.mkdirs(p);
}

Try / catch

try {
  fs.mkdirs(path);
} catch (FileAlreadyExistsException e) {
  if (String.valueOf(e.getMessage()).startsWith("Path is a file:")) {
    throw new LayoutConflictException("file/dir conflict at " + path + " — relocate the object", e);
  }
  throw e;
}

Prevention

Enforce one layout rule: any key is only ever a file or only ever a directory prefix.
Clean stale job output dirs before submission.
Add layout linting (ancestor/descendant file checks) to CI for generated path schemas.

When it happens

Trigger: mkdirs() on a path occupied by an existing object, e.g. object 'obs://b/logs' exists (a file) and code calls mkdirs('obs://b/logs'); mapreduce/Spark staging dirs colliding with a previously written file of the same name; file/dir type flips between runs (yesterday a file, today used as directory).

Common situations: Partition writers that sometimes write a file directly at the partition path and sometimes treat it as a directory; leftover files from crashed jobs blocking new directory creation; distcp copying a directory onto a destination that already has a file with the dir's name; case-insensitive-ish path typos colliding with existing keys.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/752e4e2039c49299. Report an issue: GitHub.