{"record":{"id":"f1596d39239069f6","repo":"apache/seatunnel","slug":"post-sync-backup-skipped-because-target-already-ex","errorCode":null,"errorMessage":"Post-sync backup skipped because target already exists; source is retained: splitId={}, source={}, target={}, checkpointId={}","messagePattern":"Post-sync backup skipped because target already exists; source is retained: splitId=(.+?), source=(.+?), target=(.+?), checkpointId=(.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/ContinuousMultipleTableFileSourceSplitEnumerator.java","lineNumber":877,"sourceCode":"                return OpCommitResult.FAILED_RETRYABLE;\n            }\n            if (stagingStatus == null) {\n                log.warn(\n                        \"Post-sync backup cannot determine completion because source, staging, and \"\n                                + \"backup target are absent; operation will be retried: splitId={}, \"\n                                + \"source={}, target={}, checkpointId={}\",\n                        op.getSplitId(),\n                        maskUriUserInfo(op.getSourcePath()),\n                        maskUriUserInfo(op.getBackupTargetPath()),\n                        checkpointId);\n                return OpCommitResult.FAILED_RETRYABLE;\n            }\n        }\n\n        if (targetStatus != null && sourceStatus != null) {\n            // Never use an existing target as proof that this source can be deleted: it may belong\n            // to a previous attempt while a writer has recreated the source path.\n            log.warn(\n                    \"Post-sync backup skipped because target already exists; source is retained: \"\n                            + \"splitId={}, source={}, target={}, checkpointId={}\",\n                    op.getSplitId(),\n                    maskUriUserInfo(op.getSourcePath()),\n                    maskUriUserInfo(op.getBackupTargetPath()),\n                    checkpointId);\n            return OpCommitResult.STALE_SKIPPED;\n        }\n\n        if (sourceStatus != null\n                && stagingStatus == null\n                && !isSinkTargetCommitted(ctx, op, checkpointId, op.getSourcePath())) {\n            // Keep the source visible until the sink target reaches its final committed location.\n            // This avoids unnecessary source-side rename/restore churn on file systems such as FTP\n            // where a staged move can temporarily hide the discovery root before the sink commit is\n            // actually durable.\n            return OpCommitResult.FAILED_RETRYABLE;\n        }","sourceCodeStart":859,"sourceCodeEnd":895,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/ContinuousMultipleTableFileSourceSplitEnumerator.java#L859-L895","documentation":"This WARN is logged when the backup target already exists while the source file is still present. The enumerator deliberately refuses to treat the existing target as proof the backup succeeded (it may belong to a previous attempt while a writer recreated the source path) and returns STALE_SKIPPED, retaining the source. It is a safety guard against deleting freshly-written data based on a stale target.","triggerScenarios":"At commit time both op.getSourcePath() and op.getBackupTargetPath() exist on the source filesystem, and no content-fingerprint/version check links the existing target to the current operation.","commonSituations":"Re-running a job from an older savepoint where the backup target from a previous run still exists; a concurrent writer recreated the source file after a prior backup; backup target collisions caused by non-unique target path templates.","solutions":["Remove or archive the pre-existing backup target file if it is from an old run, then let the operation retry","Make the backup target path unique per run/file (include run id, checkpoint id, or content hash) to avoid collisions","Avoid resuming from stale savepoints that reference already-completed operations; use the latest checkpoint","Confirm whether retaining the source is acceptable; the operation is marked STALE_SKIPPED and will not re-fire for the same state"],"exampleFix":"// before\nbackup-path = hdfs:///backup/data.csv\n// after\nbackup-path = hdfs:///backup/${jobId}/data.csv","handlingStrategy":"validation","validationCode":"// Before enabling post-sync backup, ensure the backup target directory does not\n// contain leftovers from previous runs\nFileStatus[] stale = fs.listStatus(backupDir);\nif (stale.length > 0) { archiveOrDelete(stale); }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use unique, run-scoped backup target paths (jobId/checkpointId in the template)","Resume from the latest checkpoint/savepoint, not stale ones","Treat STALE_SKIPPED logs as a signal to clean legacy target files"],"tags":["hadoop","file-source","backup","file-already-exists"],"backgroundTag":"file-already-exists","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}