apache/beam · error · FileAlreadyExistsException

Target object already exists and strategy is FAIL_IF_EXISTS

Error message

Target object already exists and strategy is FAIL_IF_EXISTS

What it means

In GcsUtilV2.rewriteHelper (used by copy and move), when the destination object already exists and the OverwriteStrategy is FAIL_IF_EXISTS, a FileAlreadyExistsException is thrown naming source, destination, and the reason. It is the strict, no-clobber copy mode.

Source

Thrown at sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/util/GcsUtilV2.java:385

        } catch (StorageException e) {
          throw translateStorageException(dstPath, e);
        }

        if (existingTarget == null) {
          copyRequestBuilder.setTarget(dstId, Storage.BlobTargetOption.doesNotExist());
        } else {
          switch (dstOverwrite) {
            case SKIP_IF_EXISTS:
              LOG.warn("Ignoring rewriting from {} to {} because target exists.", srcPath, dstPath);
              continue; // Skip to next file in for-loop

            case SAFE_OVERWRITE:
              copyRequestBuilder.setTarget(
                  dstId, Storage.BlobTargetOption.generationMatch(existingTarget.getGeneration()));
              break;

            case FAIL_IF_EXISTS:
              throw new FileAlreadyExistsException(
                  srcPath.toString(),
                  dstPath.toString(),
                  "Target object already exists and strategy is FAIL_IF_EXISTS");
            default:
              throw new IllegalStateException("Unknown OverwriteStrategy: " + dstOverwrite);
          }
        }
      }

      try {
        CopyWriter copyWriter = storage.copy(copyRequestBuilder.build());
        copyWriter.getResult();

        if (deleteSrc) {
          if (!storage.delete(srcId)) {
            // This may happen if the source file is deleted by another process after copy.
            LOG.warn(
                "Source file {} could not be deleted after move to {}. It may not have existed.",

View on GitHub (pinned to 12126d8942)

Solutions

Switch to OverwriteStrategy.SAFE_OVERWRITE (generationMatch) or OVERWRITE if clobbering is acceptable.
Include a unique run ID/timestamp in destination paths so re-runs don't collide.
Pre-check destination existence and skip or rename before copying.
Catch FileAlreadyExistsException and treat as skip for idempotent pipelines.

Example fix

// before
gcsUtil.copy(src, dst, CreateOptions.withOverwrite(OverwriteStrategy.FAIL_IF_EXISTS));
// after: unique per-run destination
GcsPath dst = GcsPath.fromUri(String.format("gs://bucket/out/%s/", runId));

Defensive patterns

Strategy: try-catch

Validate before calling

// pre-check destination
if (gcsUtil.getBlob(dstPath, Storage.BlobGetOption.fields()) != null) {
  // destination exists: pick new path or choose an overwrite strategy
}

Try / catch

try {
  gcsUtil.copy(src, dst, FAIL_IF_EXISTS_options);
} catch (FileAlreadyExistsException e) {
  // skip (idempotent re-run) or retry with a unique destination
}

Prevention

Embed unique run IDs in destination paths
Choose SAFE_OVERWRITE/OVERWRITE deliberately instead of FAIL_IF_EXISTS when clobbering is fine
Pre-check destinations before bulk copies

When it happens

Trigger: Calling copy/move with OverwriteStrategy.FAIL_IF_EXISTS (e.g. Beam CreateOptions with StandardCreateOptions.IGNORE_EXISTING_FILES semantics inverted, or explicit FAIL_IF_EXISTS) where the destination object's generation already exists.

Common situations: Re-running a job that copies outputs to a versionless destination path; two concurrent jobs writing the same destination filename; migrating data into a bucket that already contains the target objects.

Understand the failure class

Background: "already exists" / EEXIST / FileAlreadyExistsException: what the 'file already exists' error means and how to fix it — this error's family across 37 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/79fef25f015e64a7. Report an issue: GitHub.