apache/hadoop · error · FileAlreadyExistsException

{} already exists

Error message

{} already exists

What it means

RawFileSystem (the tos:// connector for Volcengine TOS object storage) throws FileAlreadyExistsException from create() when the target path already exists as a file/object and the overwrite flag is not set. This mirrors the Hadoop FileSystem contract (O_CREAT|O_EXCL semantics): creating over an existing file must fail loudly unless the caller explicitly asked to overwrite. The sibling branch throws the same exception with 'is a directory' when the path is a directory.

Source

Thrown at hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/RawFileSystem.java:167

    return new FSDataInputStream(fsIn);
  }

  public FSDataInputStream open(Path path, byte[] expectedChecksum, Range range) {
    return new FSDataInputStream(
        new ObjectRangeInputStream(storage, path, range, expectedChecksum));
  }

  @Override
  public FSDataOutputStream create(Path path, FsPermission permission, boolean overwrite,
      int bufferSize, short replication, long blockSize, Progressable progress) throws IOException {
    FileStatus fileStatus = getFileStatusOrNull(path);
    if (fileStatus != null) {
      if (fileStatus.isDirectory()) {
        throw new FileAlreadyExistsException(path + " is a directory");
      }

      if (!overwrite) {
        throw new FileAlreadyExistsException(path + " already exists");
      }
      LOG.debug("Overwriting file {}", path);
    }

    if (MagicOutputStream.isMagic(path)) {
      return new FSDataOutputStream(
          new MagicOutputStream(this, storage, uploadThreadPool, getConf(), makeQualified(path)),
          null);
    } else {
      ObjectOutputStream out =
          new ObjectOutputStream(storage, uploadThreadPool, getConf(), makeQualified(path), true);

      if (fileStatus == null && FuseUtils.fuseEnabled()) {
        // The fuse requires the file to be visible when accessing getFileStatus once we created
        // the file, so here we close and commit the file to be visible explicitly for fuse, and
        // then reopen the file output stream for further data bytes writing.
        out.close();
        out =

View on GitHub (pinned to 2add963021)

Solutions

  1. Pass overwrite=true: fs.create(path, true) or include CreateFlag.OVERWRITE in the EnumSet overload
  2. Delete or move the existing object first: fs.delete(path, false) before create when you own the destination
  3. Check the destination before writing: fs.exists(path) && !fs.getFileStatus(path).isFile() to fail with your own clearer error
  4. Use unique per-run output paths (job id / timestamp suffix) so runs never collide

Example fix

// before
FSDataOutputStream out = fs.create(path); // throws FileAlreadyExistsException if path exists

// after
FSDataOutputStream out = fs.create(path, true); // overwrite existing object
Defensive patterns

Strategy: validation

Validate before calling

Path p = path.makeQualified(fs.getUri(), fs.getWorkingDirectory());
if (fs.exists(p) && !fs.getFileStatus(p).isDirectory()) {
  // decide: overwrite, delete, or fail with a clear message BEFORE calling create()
}

Try / catch

try {
  out = fs.create(p, /*overwrite*/ false);
} catch (FileAlreadyExistsException e) {
  // deliberately clobber or surface a job-level 'output exists' error
}

Prevention

When it happens

Trigger: Calling fs.create(path) (overwrite defaults to false), or the CreateFlag overload without CreateFlag.OVERWRITE, on a tos:// path whose object key already exists (including zero-byte files left by a previous run or a pending magic-commit destination).

Common situations: Re-running a MapReduce/Spark job into the same output directory without cleanup; two concurrent writers targeting the same object key; leftover output from a crashed earlier attempt; a directory marker existing at the file path.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/18e323054dfa29e7. Report an issue: GitHub.