apache/hadoop · error · FileAlreadyExistsException

File already exists: " + f

Error message

File already exists: " + f

What it means

RawLocalFileSystem.create (private overload with createParent) throws FileAlreadyExistsException when the target exists and overwrite is false. This is the standard create-semantics guard across Hadoop filesystems: create() is destructive only when the caller opts in. On the local filesystem the existence check is a direct stat of the underlying file.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java:641

    }
    return new FSDataOutputStream(new BufferedOutputStream(
        createOutputStreamWithMode(f, true, null), bufferSize), statistics,
        status.getLen());
  }

  @Override
  public FSDataOutputStream create(Path f, boolean overwrite, int bufferSize,
    short replication, long blockSize, Progressable progress)
    throws IOException {
    return create(f, overwrite, true, bufferSize, replication, blockSize,
        progress, null);
  }

  private FSDataOutputStream create(Path f, boolean overwrite,
      boolean createParent, int bufferSize, short replication, long blockSize,
      Progressable progress, FsPermission permission) throws IOException {
    if (exists(f) && !overwrite) {
      throw new FileAlreadyExistsException("File already exists: " + f);
    }
    Path parent = f.getParent();
    if (parent != null && !mkdirs(parent)) {
      throw new IOException("Mkdirs failed to create " + parent.toString());
    }
    return new FSDataOutputStream(new BufferedIOStatisticsOutputStream(
        createOutputStreamWithMode(f, false, permission), bufferSize, true),
        statistics);
  }
  
  protected OutputStream createOutputStream(Path f, boolean append) 
      throws IOException {
    return createOutputStreamWithMode(f, append, null);
  }

  protected OutputStream createOutputStreamWithMode(Path f, boolean append,
      FsPermission permission) throws IOException {
    return new LocalFSFileOutputStream(f, append, permission);

View on GitHub (pinned to 2add963021)

Solutions

1. Pass overwrite=true when clobbering is intended: fs.create(path, true).
2. Delete the stale target before creating: if (fs.exists(path)) fs.delete(path, false);
3. For unique outputs use FileContext/FileSystem temp files or names with timestamps/UUIDs instead of fixed names.
4. If this is MapReduce output, set output.overwrite semantics or clean the output directory in the driver.

Example fix

// before
FSDataOutputStream out = fs.create(new Path("/tmp/report.csv")); // exists -> throws

// after
FSDataOutputStream out = fs.create(new Path("/tmp/report.csv"), true);

boolean overwrite = true; // or prompt/clean beforehand
if (!overwrite && fs.exists(p)) {
  throw new FileAlreadyExistsException(p + " exists; delete it or enable overwrite");
}
FSDataOutputStream out = fs.create(p, overwrite);

try {
  out = fs.create(p, false);
} catch (FileAlreadyExistsException e) {
  // decide policy: clobber, fail, or pick a new name
  out = fs.create(p, true);
}

When it happens

Trigger: Calling fs.create(path, false, ...) (or the short overloads that default overwrite=false) when the local file already exists: re-running a job without cleaning output, writing to a fixed temp filename, or two concurrent tasks writing the same local path.

Common situations: Re-running a MapReduce/local pipeline whose previous output was not removed, hard-coded scratch files (/tmp/myapp.dat) left behind by an earlier run, or dev iteration where output directories persist between runs.

Related errors

• File already exists: %s
• {} already exists
• Parent directory doesn't exist: {}
• File already exists: ${path}. Append or overwrite option mus
• Mkdirs failed to create " + parent.toString()

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/581f7260d9d27c4f. Report an issue: GitHub.