apache/iceberg · error · AlreadyExistsException

Path already exists

Error message

Path already exists: %s

What it means

HadoopOutputFile.create() opens a stream with createOrOverwrite=false, so Hadoop's FileAlreadyExistsException is converted to Iceberg's AlreadyExistsException with this message. It signals the caller requested a strictly-new file but the path exists.

Solutions

1. Use createOrOverwrite() if replacing the file is intended
2. Generate unique file names per attempt (Iceberg does this via UUIDs/counter in OutputFileFactory)
3. Check file existence first or clean stale files from the staging directory before retrying
4. Handle AlreadyExistsException and fall back to a new path

Example fix

// before
OutputFile out = io.newOutputFile(path);
PositionOutputStream s = out.create(); // throws if exists
// after
PositionOutputStream s = (overwrite)
    ? io.newOutputFile(path).createOrOverwrite()
    : io.newOutputFile(path + "." + UUID.randomUUID()).create();

Path p = new Path(location);
if (fs.exists(p)) {
  throw new org.apache.iceberg.exceptions.AlreadyExistsException("Path already exists: %s", location);
}

try { return out.create(); }
catch (AlreadyExistsException e) {
  OutputFile alt = io.newOutputFile(location + "." + UUID.randomUUID());
  return alt.create();
}

When it happens

Trigger: Calling create() when the target path already exists on the FileSystem (Hadoop fs.create with overwrite=false fails).

Common situations: Writing data/manifest files into a location where a same-named file already exists (rerun of a job without cleaning staging, non-unique file names, concurrent writers producing the same path).

Understand the failure class

Background: "already exists" / EEXIST / FileAlreadyExistsException: what the 'file already exists' error means and how to fix it — this error's family across 37 libraries.

Related errors

• Error adding Hadoop resource
• Already exists
• ALTER TABLE contains multiple distribution clauses
• ALTER TABLE contains multiple ordering clauses
• Cannot apply non-unique WAP ID. Found multiple snapshots…

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/7a33b0fc81199888. Report an issue: GitHub.

Appendix: source

Thrown at core/src/main/java/org/apache/iceberg/hadoop/HadoopOutputFile.java:76

    return fromPath(path, fs, fs.getConf());
  }

  public static OutputFile fromPath(Path path, FileSystem fs, Configuration conf) {
    return new HadoopOutputFile(fs, path, conf);
  }

  private HadoopOutputFile(FileSystem fs, Path path, Configuration conf) {
    this.fs = fs;
    this.path = path;
    this.conf = conf;
  }

  @Override
  public PositionOutputStream create() {
    try {
      return HadoopStreams.wrap(fs.create(path, false /* createOrOverwrite */));
    } catch (FileAlreadyExistsException e) {
      throw new AlreadyExistsException(e, "Path already exists: %s", path);
    } catch (IOException e) {
      throw new RuntimeIOException(e, "Failed to create file: %s", path);
    }
  }

  @Override
  public PositionOutputStream createOrOverwrite() {
    try {
      return HadoopStreams.wrap(fs.create(path, true /* createOrOverwrite */));
    } catch (IOException e) {
      throw new RuntimeIOException(e, "Failed to create file: %s", path);
    }
  }

  public Path getPath() {
    return path;
  }

View on GitHub (pinned to 86d9c8fc54)