apache/hadoop · error · FileAlreadyExistsException

{} is a directory

Error message

{} is a directory

What it means

RawFileSystem.create stats the target first; when an existing directory sits at the path it throws FileAlreadyExistsException('<path> is a directory'). This happens regardless of the overwrite flag, because removing a directory tree to satisfy a file create would destroy data.

Source

Thrown at hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/RawFileSystem.java:163

    Preconditions.checkArgument(rangeSize > 0, "Object storage range size must be positive.");

    FSInputStream fsIn = new ObjectMultiRangeInputStream(taskThreadPool, storage, path,
        status.getLen(), rangeSize, status.checksum());
    return new FSDataInputStream(fsIn);
  }

  public FSDataInputStream open(Path path, byte[] expectedChecksum, Range range) {
    return new FSDataInputStream(
        new ObjectRangeInputStream(storage, path, range, expectedChecksum));
  }

  @Override
  public FSDataOutputStream create(Path path, FsPermission permission, boolean overwrite,
      int bufferSize, short replication, long blockSize, Progressable progress) throws IOException {
    FileStatus fileStatus = getFileStatusOrNull(path);
    if (fileStatus != null) {
      if (fileStatus.isDirectory()) {
        throw new FileAlreadyExistsException(path + " is a directory");
      }

      if (!overwrite) {
        throw new FileAlreadyExistsException(path + " already exists");
      }
      LOG.debug("Overwriting file {}", path);
    }

    if (MagicOutputStream.isMagic(path)) {
      return new FSDataOutputStream(
          new MagicOutputStream(this, storage, uploadThreadPool, getConf(), makeQualified(path)),
          null);
    } else {
      ObjectOutputStream out =
          new ObjectOutputStream(storage, uploadThreadPool, getConf(), makeQualified(path), true);

      if (fileStatus == null && FuseUtils.fuseEnabled()) {
        // The fuse requires the file to be visible when accessing getFileStatus once we created

View on GitHub (pinned to 2add963021)

Solutions

1. Choose an output path or file name that does not collide with the directory
2. Delete the directory explicitly with fs.delete(path, true) only when its removal is intended
3. Guard with a status check before create and fail with a clear message

Example fix

// before
fs.create(new Path("/outputs/data"), true); // /outputs/data is a directory

// after
Path out = new Path("/outputs/data");
if (fs.exists(out) && fs.getFileStatus(out).isDirectory()) {
  fs.delete(out, true); // deliberate removal
}
try (FSDataOutputStream os = fs.create(out, true)) {
  ... // write
}

FileStatus st = fs.getFileStatusOrNull(path);
if (st != null && st.isDirectory()) {
  throw new FileAlreadyExistsException("Output path is a directory: " + path
      + "; pick a file name or delete the directory first");
}

static boolean canCreateFile(FileSystem fs, Path p) throws IOException {
  FileStatus st = fs.getFileStatusOrNull(p);
  return st == null || st.isFile();
}

try {
  return fs.create(path, overwrite);
} catch (FileAlreadyExistsException e) {
  if (e.getMessage() != null && e.getMessage().endsWith("is a directory")) {
    // overwrite cannot fix a directory: choose a new name or delete(dir, true) deliberately
  } else {
    throw e;
  }
}

When it happens

Trigger: create(path, ..., overwrite=true) where path exists as a directory with or without children; a file name equal to an existing partition directory; output path configured to the same location as an input directory.

Common situations: Reusing an input directory as the job output; Hive external table locations overlapping plain file writes; leftover directories from previous runs sharing the intended file name.

Related errors

• {} already exists
• {} already exists
• / is not a file. The directory / already exist at: <theInter
• Not a directory: {}
• Path is a file: {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/bb7afb05aeb51666. Report an issue: GitHub.