Post-sync {} cannot restore staged source while waiting for

Error message

Post-sync {} cannot restore staged source while waiting for sink target; operation will be retried with staged file intact: source={}, staging={}, checkpointId={}

What it means

In handleRetryableStagedOperation(), when the sink target is not yet committed, the enumerator restores the staged file back to the source path to avoid hiding data while waiting. This WARN fires when that restore fails; the operation still returns FAILED_RETRYABLE and will be retried with the staged file left intact in staging. It is a degraded but safe state — data exists only under the staging path until a retry succeeds.

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/ContinuousMultipleTableFileSourceSplitEnumerator.java:1111

                op.getSplitId(),
                maskUriUserInfo(op.getSourcePath()),
                maskUriUserInfo(stagedPath),
                checkpointId);
        return OpCommitResult.STALE_SKIPPED;
    }

    private OpCommitResult handleRetryableStagedOperation(
            TableScanContext ctx,
            FileSourceOperationState op,
            long checkpointId,
            String stagedPath,
            String actionLabel,
            FileStatus stagedStatus)
            throws IOException {
        RestoreStagedFileResult restoreResult =
                restoreStagedSource(ctx, op, stagedPath, stagedStatus, actionLabel);
        if (restoreResult == RestoreStagedFileResult.FAILED) {
            log.warn(
                    "Post-sync {} cannot restore staged source while waiting for sink target; "
                            + "operation will be retried with staged file intact: source={}, staging={}, "
                            + "checkpointId={}",
                    actionLabel,
                    maskUriUserInfo(op.getSourcePath()),
                    maskUriUserInfo(stagedPath),
                    checkpointId);
        }
        return OpCommitResult.FAILED_RETRYABLE;
    }

    private RestoreStagedFileResult restoreStagedSource(
            TableScanContext ctx,
            FileSourceOperationState op,
            String stagedPath,
            FileStatus stagedStatus,
            String actionLabel)
            throws IOException {

View on GitHub (pinned to cf67b549a7)

Solutions

Fix the root cause of the restore failure (permissions, pre-existing file at source path) using nearby log details
Check why the sink target is not committing (sink committer logs); the longer the wait, the more restore attempts occur
Verify no other process recreated the source path while the file is staged; remove the duplicate if the staged file is authoritative
Allow the built-in retry to proceed — the staged file is intentionally kept intact; do not delete the staging path manually

Example fix

null

Defensive patterns

Strategy: retry

Validate before calling

// Check the source path is clear before attempting a restore:
if (fs.exists(sourcePath)) {
    // restore would collide; resolve manually instead of failing repeatedly
}

Try / catch

try {
    restoreStagedToSource(stagedPath, sourcePath);
} catch (IOException e) {
    // staged file remains intact; schedule bounded retry, keep monitoring sink commit progress
}

Prevention

Investigate slow sink committers — prolonged waits repeatedly exercise the restore path
Never manually delete staging files while a job is retrying
Ensure only one pipeline writes to a given source path to avoid restore collisions

When it happens

Trigger: isSinkTargetCommitted() returns false (sink target absent or content mismatched) and restoreStagedSource() during the wait returns RestoreStagedFileResult.FAILED for the staged file.

Common situations: Slow sink committer keeps the sink target uncommitted for many checkpoints while source-directory permissions block the restore; a writer recreated the source path so the restore rename refuses; transient FS errors during restore; FTP filesystems where the discovery root was moved into staging.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/98e12b9a848c1362. Report an issue: GitHub.