apache/seatunnel · warning

Post-sync backup skipped because target already exists; sour

Error message

Post-sync backup skipped because target already exists; source is retained: splitId={}, source={}, target={}, checkpointId={}

What it means

This WARN is logged when the backup target already exists while the source file is still present. The enumerator deliberately refuses to treat the existing target as proof the backup succeeded (it may belong to a previous attempt while a writer recreated the source path) and returns STALE_SKIPPED, retaining the source. It is a safety guard against deleting freshly-written data based on a stale target.

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/ContinuousMultipleTableFileSourceSplitEnumerator.java:877

                return OpCommitResult.FAILED_RETRYABLE;
            }
            if (stagingStatus == null) {
                log.warn(
                        "Post-sync backup cannot determine completion because source, staging, and "
                                + "backup target are absent; operation will be retried: splitId={}, "
                                + "source={}, target={}, checkpointId={}",
                        op.getSplitId(),
                        maskUriUserInfo(op.getSourcePath()),
                        maskUriUserInfo(op.getBackupTargetPath()),
                        checkpointId);
                return OpCommitResult.FAILED_RETRYABLE;
            }
        }

        if (targetStatus != null && sourceStatus != null) {
            // Never use an existing target as proof that this source can be deleted: it may belong
            // to a previous attempt while a writer has recreated the source path.
            log.warn(
                    "Post-sync backup skipped because target already exists; source is retained: "
                            + "splitId={}, source={}, target={}, checkpointId={}",
                    op.getSplitId(),
                    maskUriUserInfo(op.getSourcePath()),
                    maskUriUserInfo(op.getBackupTargetPath()),
                    checkpointId);
            return OpCommitResult.STALE_SKIPPED;
        }

        if (sourceStatus != null
                && stagingStatus == null
                && !isSinkTargetCommitted(ctx, op, checkpointId, op.getSourcePath())) {
            // Keep the source visible until the sink target reaches its final committed location.
            // This avoids unnecessary source-side rename/restore churn on file systems such as FTP
            // where a staged move can temporarily hide the discovery root before the sink commit is
            // actually durable.
            return OpCommitResult.FAILED_RETRYABLE;
        }

View on GitHub (pinned to cf67b549a7)

Solutions

1. Remove or archive the pre-existing backup target file if it is from an old run, then let the operation retry
2. Make the backup target path unique per run/file (include run id, checkpoint id, or content hash) to avoid collisions
3. Avoid resuming from stale savepoints that reference already-completed operations; use the latest checkpoint
4. Confirm whether retaining the source is acceptable; the operation is marked STALE_SKIPPED and will not re-fire for the same state

Example fix

// before
backup-path = hdfs:///backup/data.csv
// after
backup-path = hdfs:///backup/${jobId}/data.csv

// Before enabling post-sync backup, ensure the backup target directory does not
// contain leftovers from previous runs
FileStatus[] stale = fs.listStatus(backupDir);
if (stale.length > 0) { archiveOrDelete(stale); }

When it happens

Trigger: At commit time both op.getSourcePath() and op.getBackupTargetPath() exist on the source filesystem, and no content-fingerprint/version check links the existing target to the current operation.

Common situations: Re-running a job from an older savepoint where the backup target from a previous run still exists; a concurrent writer recreated the source file after a prior backup; backup target collisions caused by non-unique target path templates.

Understand the failure class

Background: "already exists" / EEXIST / FileAlreadyExistsException: what the 'file already exists' error means and how to fix it — this error's family across 37 libraries.

Related errors

• Post-sync backup failed: backup target path is empty, splitI
• Post-sync backup cannot determine completion because source,
• Post-sync backup promotion target is absent after rename; op
• Post-sync backup promoted an unexpected target version; oper
• The file does not support this archive compress type: {}

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/f1596d39239069f6. Report an issue: GitHub.