apache/seatunnel · warning

Transient XA recovery-scan failure on attempt

Error message

Transient XA recovery-scan failure on attempt {}/{}

What it means

recoverCheckpointTransactions calls xaFacade.recover() to list in-doubt XA transactions, retrying up to maxCommitAttempts on TransientXaException with backoff. This WARN records each failed attempt; if all attempts fail the underlying transient exception propagates, failing restoreCommit.

Solutions

1. Ensure maxCommitAttempts is high enough for flaky networks
2. Check DB availability and xa_recover performance/indexes
3. Increase XA/connection timeouts on the datasource
4. Retry the job after the transient DB issue clears

Example fix

// before
max_commit_attempts = 1
// after
max_commit_attempts = 10

// preflight: attempt a cheap recovery scan before restore
try { xaFacade.recover(); } catch (TransientXaException e) { alertDbConnectivity(); }

for (int attempt = 1; attempt <= maxAttempts; attempt++) {
    try { return xaFacade.recover(); }
    catch (TransientXaException e) {
        if (attempt == maxAttempts) throw e;
        backoff(attempt);
    }
}

When it happens

Trigger: xa_recover fails transiently while restoring: temporary connection loss to the database, lock/timeout on the XA recovery scan, DB restarting, or network blips during failover.

Common situations: Recovering a large number of in-doubt transactions that makes xa_recover slow/timeout; DB maintenance window during job restart; transient network partitions.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

• [ ] request http failed
• Edge socket receiver loop exception, retrying
• failed to commit transactions out of (keep them to retry…
• Graph operation failed on attempt
• recovery after rollback prepared transaction failure also…

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/c8bcb79bccf4e991. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/sink/JdbcSinkAggregatedCommitter.java:255

                    alreadyResolved);
        }
    }

    /**
     * Retries the recovery scan with the same bounded budget as XA commit so transient RM outages
     * do not fail restore immediately.
     *
     * @return canonical values returned by the recovery scan
     */
    private Set<XidKey> recoverCheckpointTransactions() {
        int maxCommitAttempts = jdbcSinkConfig.getJdbcConnectionConfig().getMaxCommitAttempts();
        XaFacade.TransientXaException lastTransientFailure = null;
        for (int attempt = 1; attempt <= Math.max(1, maxCommitAttempts); attempt++) {
            try {
                return normalizeXids(xaFacade.recover());
            } catch (XaFacade.TransientXaException e) {
                lastTransientFailure = e;
                log.warn(
                        "Transient XA recovery-scan failure on attempt {}/{}",
                        attempt,
                        Math.max(1, maxCommitAttempts),
                        e);
                if (attempt < Math.max(1, maxCommitAttempts)) {
                    backoffBeforeRetry(
                            "recovery scan", attempt + 1, Math.max(1, maxCommitAttempts));
                }
            }
        }
        throw new JdbcConnectorException(
                CommonErrorCodeDeprecated.WRITER_OPERATION_FAILED,
                String.format(
                        "unable to complete the XA recovery scan within %d attempts",
                        Math.max(1, maxCommitAttempts)),
                lastTransientFailure);
    }

View on GitHub (pinned to cf67b549a7)