{"record":{"id":"61cc7d2f3c058549","repo":"apache/seatunnel","slug":"post-sync-backup-cannot-determine-completion-becau","errorCode":null,"errorMessage":"Post-sync backup cannot determine completion because source, staging, and backup target are absent; operation will be retried: splitId={}, source={}, target={}, checkpointId={}","messagePattern":"Post-sync backup cannot determine completion because source, staging, and backup target are absent; operation will be retried: splitId=(.+?), source=(.+?), target=(.+?), checkpointId=(.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/ContinuousMultipleTableFileSourceSplitEnumerator.java","lineNumber":862,"sourceCode":"                            op.getSourceLength(),\n                            op.getSourceModificationTime());\n                    return OpCommitResult.SUCCESS;\n                }\n                log.error(\n                        \"Post-sync backup recovery found an inconsistent target; operation will be \"\n                                + \"retried without deleting data: splitId={}, source={}, target={}, \"\n                                + \"capturedLen={}, capturedMtime={}, actualLen={}, actualMtime={}\",\n                        op.getSplitId(),\n                        maskUriUserInfo(op.getSourcePath()),\n                        maskUriUserInfo(op.getBackupTargetPath()),\n                        op.getSourceLength(),\n                        op.getSourceModificationTime(),\n                        targetStatus.getLen(),\n                        targetStatus.getModificationTime());\n                return OpCommitResult.FAILED_RETRYABLE;\n            }\n            if (stagingStatus == null) {\n                log.warn(\n                        \"Post-sync backup cannot determine completion because source, staging, and \"\n                                + \"backup target are absent; operation will be retried: splitId={}, \"\n                                + \"source={}, target={}, checkpointId={}\",\n                        op.getSplitId(),\n                        maskUriUserInfo(op.getSourcePath()),\n                        maskUriUserInfo(op.getBackupTargetPath()),\n                        checkpointId);\n                return OpCommitResult.FAILED_RETRYABLE;\n            }\n        }\n\n        if (targetStatus != null && sourceStatus != null) {\n            // Never use an existing target as proof that this source can be deleted: it may belong\n            // to a previous attempt while a writer has recreated the source path.\n            log.warn(\n                    \"Post-sync backup skipped because target already exists; source is retained: \"\n                            + \"splitId={}, source={}, target={}, checkpointId={}\",\n                    op.getSplitId(),","sourceCodeStart":844,"sourceCodeEnd":880,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/ContinuousMultipleTableFileSourceSplitEnumerator.java#L844-L880","documentation":"This WARN fires in commitBackupOperation() when neither the source file, the staging file, nor the backup target can be found on the source filesystem, so the enumerator cannot tell whether the backup already completed or was lost. It returns FAILED_RETRYABLE and waits for a later checkpoint. It usually means the underlying file vanished mid-operation (external deletion, FS inconsistency) or the operation state points at paths that no longer exist.","triggerScenarios":"During checkpoint commit, getFileStatusIfPresent() returns null for op.getSourcePath(), the staging path, AND op.getBackupTargetPath() — i.e. all three paths are absent from the source filesystem.","commonSituations":"An external process (cleanup job, TTL policy, another pipeline) deleted the source file while the backup was in flight; object-store eventual-consistency hiding recently renamed files; operator state from a failed attempt whose staging and target were already removed.","solutions":["Check whether an external cleanup/TTL job deletes files from the source or backup directories and exclude the in-flight paths","Verify filesystem consistency (especially S3/OSS eventual consistency) and that the paths in the log actually exist","Compare the checkpoint state (sourcePath/backupTargetPath) against the live filesystem to confirm the paths are correct","If the file was legitimately deleted externally, clear the stale operation state or restart the job so it re-discovers files"],"exampleFix":"null","handlingStrategy":"retry","validationCode":"// Pre-check that at least one of source/staging/target exists before committing\nboolean anyPresent = fs.exists(sourcePath) || fs.exists(stagingPath) || fs.exists(targetPath);\nif (!anyPresent) {\n    // surface loudly instead of silently retrying forever\n    throw new IllegalStateException(\"All backup paths absent for split \" + splitId);\n}","typeGuard":null,"tryCatchPattern":"try {\n    OpCommitResult r = enumeratorCommit(op);\n} catch (java.io.IOException e) {\n    // FAILED_RETRYABLE semantics: bound the retries, alert if exceeded\n    if (++attempts > MAX_ATTEMPTS) alertOperator(op, e);\n}","preventionTips":["Exclude source/staging/backup directories from external TTL/cleanup jobs","Run only one job instance per source path; use unique staging paths per job","Verify object-store consistency behavior before enabling post-sync backup on S3/OSS"],"tags":["hadoop","file-source","backup","file-not-found","retryable"],"backgroundTag":"file-not-found","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}