apache/hadoop · error · FileAlreadyExistsException

{} already exists

Error message

{} already exists

What it means

The sibling check in OBSFileSystem.create(...): the path exists as a file and the caller did not ask for overwrite, so the connector throws FileAlreadyExistsException(f + ' already exists'). This is the standard Hadoop FileSystem contract — create without overwrite must fail on an existing file — implemented before the OBSBlockOutputStream is constructed. Note the connector treats a FileNotFoundException from getFileStatus as 'safe to create', so the error strictly means an existing object was found.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSFileSystem.java:709

      final short replication,
      final long blkSize,
      final Progressable progress)
      throws IOException {
    String key = OBSCommonUtils.pathToKey(this, f);
    FileStatus status;
    long objectLen = 0;
    try {
      // get the status or throw an exception
      status = getFileStatus(f);
      objectLen = status.getLen();
      // if the thread reaches here, there is something at the path
      if (status.isDirectory()) {
        // path references a directory: automatic error
        throw new FileAlreadyExistsException(f + " is a directory");
      }
      if (!overwrite) {
        // path references a file and overwrite is disabled
        throw new FileAlreadyExistsException(f + " already exists");
      }
      LOG.debug("create: Overwriting file {}", f);
    } catch (FileNotFoundException e) {
      // this means the file is not found
      LOG.debug("create: Creating new file {}", f);
    }
    return new FSDataOutputStream(
        new OBSBlockOutputStream(
            this,
            key,
            objectLen,
            new SemaphoredDelegatingExecutor(
                boundedMultipartUploadThreadPool,
                blockOutputActiveBlocks, true),
            false),
        null);
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Pass overwrite=true when the semantics allow replacing the file: fs.create(path, true)
2. Or delete the existing object first: if (fs.exists(path)) fs.delete(path, false)
3. Or write to a unique path (append attempt id / timestamp) and rename atomically afterwards
4. Audit callers of FileSystem.create(...) helpers that default overwrite to false

Example fix

// before
FSDataOutputStream out = fs.create(path); // existing file, default overwrite=false -> FileAlreadyExistsException

// after
FSDataOutputStream out = fs.create(path, true);

if (!overwrite && fs.exists(f) && fs.getFileStatus(f).isFile()) {
  throw new FileAlreadyExistsException(f + " already exists and overwrite is disabled");
}
FSDataOutputStream out = fs.create(f, overwrite);

try {
  out = fs.create(f, false);
} catch (FileAlreadyExistsException e) {
  if (String.valueOf(e.getMessage()).endsWith("already exists")) {
    out = fs.create(f, true); // only if replace-on-rerun is acceptable
  } else {
    throw e;
  }
}

When it happens

Trigger: fs.create(path, false) or create(path) with default overwrite=false when the object already exists; re-running a job with output committed by an operation that bypasses the overwrite flag; passing overwrite=false implicitly via some FileSystem.create overloads or FileSystem.create(FileSystem,Path) helpers.

Common situations: Re-running pipelines without cleaning output; downstream consumers that write result files to deterministic paths; enablement of exactly-once semantics where the writer intentionally disallows overwrite but the file from attempt #1 survived.

Related errors

• File already exists: " + f
• File already exists: %s
• A directory with that name exists: %s
• {} is a directory
• Not a directory: {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/5eac2d0b2abc576c. Report an issue: GitHub.