{"record":{"id":"c399da921de03b22","repo":"apache/seatunnel","slug":"post-sync-backup-rename-to-staging-failed-will-r","errorCode":null,"errorMessage":"Post-sync backup: rename-to-staging failed, will retry: source={}, staging={}","messagePattern":"Post-sync backup: rename-to-staging failed, will retry: source=(.+?), staging=(.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/ContinuousMultipleTableFileSourceSplitEnumerator.java","lineNumber":901,"sourceCode":"                    checkpointId);\n            return OpCommitResult.STALE_SKIPPED;\n        }\n\n        if (sourceStatus != null\n                && stagingStatus == null\n                && !isSinkTargetCommitted(ctx, op, checkpointId, op.getSourcePath())) {\n            // Keep the source visible until the sink target reaches its final committed location.\n            // This avoids unnecessary source-side rename/restore churn on file systems such as FTP\n            // where a staged move can temporarily hide the discovery root before the sink commit is\n            // actually durable.\n            return OpCommitResult.FAILED_RETRYABLE;\n        }\n\n        if (stagingStatus == null) {\n            try {\n                ctx.sourceFs.renameFile(op.getSourcePath(), stagingPath, false);\n            } catch (Exception e) {\n                log.warn(\n                        \"Post-sync backup: rename-to-staging failed, will retry: source={}, staging={}\",\n                        maskUriUserInfo(op.getSourcePath()),\n                        maskUriUserInfo(stagingPath),\n                        e);\n                return OpCommitResult.FAILED_RETRYABLE;\n            }\n            stagingStatus = getFileStatusIfPresent(ctx.sourceFs, stagingPath);\n        }\n\n        if (stagingStatus == null) {\n            log.warn(\n                    \"Post-sync backup staging disappeared before verification; operation will be retried: \"\n                            + \"splitId={}, source={}, staging={}, checkpointId={}\",\n                    op.getSplitId(),\n                    maskUriUserInfo(op.getSourcePath()),\n                    maskUriUserInfo(stagingPath),\n                    checkpointId);\n            return OpCommitResult.FAILED_RETRYABLE;","sourceCodeStart":883,"sourceCodeEnd":919,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/ContinuousMultipleTableFileSourceSplitEnumerator.java#L883-L919","documentation":"This WARN is emitted when renaming the source file to the backup staging path fails with an exception during commitBackupOperation(). The rename is the first mutation of the backup flow; when it throws (IO error, permission issue, cross-device move, transient FS failure) the enumerator logs the exception and returns FAILED_RETRYABLE so the next checkpoint attempt redoes it.","triggerScenarios":"ctx.sourceFs.renameFile(op.getSourcePath(), stagingPath, false) throws — e.g. permission denied on source or staging directory, staging directory missing, transient HDFS/S3/network error, or source file locked/being written concurrently.","commonSituations":"Insufficient HDFS/FTP/S3 permissions on the staging directory; staging parent directory not created; object-store throttling (S3 503 slow-down); network partition between the JobManager and the filesystem; another process holding/renaming the same source file.","solutions":["Read the stacked exception in the log — it names the real cause (permission, NoSuchFile, throttling, timeout)","Verify the connector identity has write permission on both the source directory and the staging/backup parent directories","Ensure the staging parent directory exists and is on the same filesystem/device as the source (rename across filesystems is not atomic)","Check object-store rate limits and retry/backoff settings if using S3/OSS; retry the checkpoint commit"],"exampleFix":"// before (FTP user without write access on staging dir)\n# FTP: 550 Permission denied\n// after\n# grant write on the staging directory to the job user\nsite chmod 775 /backup/staging","handlingStrategy":"retry","validationCode":"// Pre-flight: confirm write access on source and staging directories before starting the job\nPath stagingDir = new Path(stagingPath).getParent();\nif (!fs.exists(stagingDir)) { fs.mkdirs(stagingDir); }\nFSDataOutputStream probe = fs.create(new Path(stagingDir, \".write-probe\"));\nprobe.close(); fs.delete(new Path(stagingDir, \".write-probe\"), false);","typeGuard":null,"tryCatchPattern":"try {\n    fs.renameFile(source, staging, false);\n} catch (IOException e) {\n    // inspect e: permission, FileNotFoundException, throttling; backoff and retry\n    Thread.sleep(backoffMs); retry();\n}","preventionTips":["Grant the job user write permission on source, staging, and backup directories","Keep staging on the same filesystem/device as the source for atomic renames","Watch object-store rate limits (S3 503) and configure retry/backoff"],"tags":["hadoop","file-source","rename-failed","io","retryable"],"backgroundTag":"file-write-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}