apache/hadoop · error · FileAlreadyExistsException

A directory with that name exists: %s

Error message

A directory with that name exists: %s

What it means

create() throws FileAlreadyExistsException when a directory with the target name exists. The check runs when createOptions.isEnsureNoDirectoryConflict() and uses getFileInfoInternal(dirId, inferImplicitDirectories=true), so BOTH an explicit directory-marker object (gs://b/foo/) and an implicit directory (any child gs://b/foo/... exists) count as a conflict. It fires after the file/directory name check and the files-conflicting-with-dirs check.

Source

Thrown at hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleCloudStorageFileSystem.java:140

    //
    // For example, for a new `gs://bucket/c/d/f` file:
    // - files `gs://bucket/c` and `gs://bucket/c/d` should not exist
    // - directory `gs://bucket/c/d/f/` should not exist
    if (configuration.isEnsureNoConflictingItems()) {
      // Check if a directory with the same name exists.
      StorageResourceId dirId = resourceId.toDirectoryId();
      Boolean conflictingDirExist = false;
      if (createOptions.isEnsureNoDirectoryConflict()) {
        // TODO: Do this concurrently
        conflictingDirExist =
            getFileInfoInternal(dirId, /* inferImplicitDirectories */ true).exists();
      }

      checkNoFilesConflictingWithDirs(resourceId);

      // Check if a directory with the same name exists.
      if (conflictingDirExist) {
        throw new FileAlreadyExistsException("A directory with that name exists: " + path);
      }
    }

    if (createOptions.getOverwriteGenerationId() != StorageResourceId.UNKNOWN_GENERATION_ID) {
      resourceId = new StorageResourceId(resourceId.getBucketName(), resourceId.getObjectName(),
          createOptions.getOverwriteGenerationId());
    }

    return gcs.create(resourceId, createOptions);
  }

  void close() {
    if (gcs == null) {
      return;
    }
    LOG.trace("close()");
    try {
      gcs.close();

View on GitHub (pinned to 2add963021)

Solutions

1. Pick a different file name, or remove/rename the conflicting directory tree before creating the file.
2. Clean up stale children from previous runs before the job starts.
3. Only if your semantics tolerate it, disable the conflict check (ensureNoDirectoryConflict=false in CreateFileOptions) — note GCS would then allow a file and a same-named prefix to coexist, which confuses later listings.

Example fix

// before
gcsFs.create(URI.create("gs://b/foo"), CreateFileOptions.DEFAULT); // FileAlreadyExistsException

// after: clear the conflicting directory first
gcsFs.delete(URI.create("gs://b/foo/"), /*recursive=*/ true);
gcsFs.create(URI.create("gs://b/foo"), CreateFileOptions.DEFAULT);

// Pre-check: both explicit marker and implicit (children) directories conflict
FileInfo dirInfo = gcsFs.getFileInfoInternal(
    resourceId.toDirectoryId(), /* inferImplicitDirectories */ true);
if (dirInfo.exists()) {
  throw new OutputNameConflictException(resourceId); // decide: pick new name or clean
}
gcsFs.create(path, CreateFileOptions.DEFAULT);

catch (FileAlreadyExistsException e) {
  // a directory (explicit marker or implicit via children) owns this name
  // either choose a new file name or delete/rename the directory tree first
}

When it happens

Trigger: create(gs://bucket/foo) when gs://bucket/foo/ exists as a directory marker OR any object under the gs://bucket/foo/ prefix exists — e.g. a partition directory already written by a previous job.

Common situations: Writing a file where a Hive/Spark partition directory of the same name lives; output collisions between a file writer and directory-creating code; leftover data from failed jobs occupying the name as a prefix.

Related errors

• Cannot create a file whose name looks like a directory: '%s'
• Can not create '%s' file, because parent folder does not exi
• {} already exists
• File already exists: " + f
• File already exists: %s

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8bb71ff248a422b3. Report an issue: GitHub.