apache/seatunnel · error · IOException

sendUntilReceived exhausted reconnect cycles for batchId=

Error message

sendUntilReceived exhausted reconnect cycles for batchId=${batchId}

What it means

EdgeTransportClient.sendUntilReceived retries a batch across full reconnect cycles: each cycle opens a session, authenticates, and tries sendBatchUntilReceived. If all reconnect cycles fail, the last recorded InterruptedException/IOException is rethrown if present; only when no underlying exception was captured does it throw this fallback IOException, meaning the batch could not be confirmed after every reconnect attempt.

Solutions

1. Check earlier log lines for the per-cycle root cause (auth rejection, EOF, unexpected response) and fix that root cause first
2. Increase the reconnect-cycle limit / backoff in EdgeTransportConfig for longer outages
3. Persist the unacknowledged batchId (WAL) and replay it after the collector recovers
4. Verify the collector address and credentials, since non-IO failures (rejection, illegal state) are the usual silent causes

Example fix

// before
client.send(batchId, payload); // throws after all cycles
// after
try {
    client.send(batchId, payload);
} catch (IOException e) {
    wal.persist(batchId, payload); // replay later with same batchId
    throw e;
}

try { client.send(batchId, payload); } catch (IOException e) { if (e.getMessage().contains("exhausted reconnect cycles")) { wal.replayLater(batchId, payload); } throw e; }

When it happens

Trigger: Every reconnect cycle fails in a way that stores neither lastIo nor lastInterrupted (e.g. open aborted via IllegalStateException/EdgeSocketCollectorRejectedException repeatedly), exhausting the cycle budget for the given batchId.

Common situations: Collector persistently rejecting auth so each cycle fails before an IOException is recorded; sustained collector outage combined with a low reconnect-cycle limit; misconfigured host/port making every cycle abort non-IO.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

• Edge socket receiver loop exception, retrying
• Edge transport IO failure, will reconnect. batchId=
• Exceeded maxBatchSendAttempts=
• Airtable API rate limit reached, retry
• Airtable API rate limit reached, retry

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/82b4fb8d4cb1df73. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-edge-agent/seatunnel-edge-agent-transport/src/main/java/org/apache/seatunnel/edge/agent/transport/socket/EdgeTransportClient.java:112

                    invalidateSession();
                    throw ex;
                } catch (InterruptedException ex) {
                    Thread.currentThread().interrupt();
                    lastInterrupted = ex;
                    break;
                } catch (IOException ex) {
                    lastIo = ex;
                    LOG.warn("Edge transport IO failure, will reconnect. batchId={}", batchId, ex);
                    invalidateSession();
                }
            }
            if (lastInterrupted != null) {
                throw lastInterrupted;
            }
            if (lastIo != null) {
                throw lastIo;
            }
            throw new IOException(
                    "sendUntilReceived exhausted reconnect cycles for batchId=" + batchId);
        }
    }

    @Override
    public boolean probeReachable() throws IOException {
        synchronized (connectionLock) {
            try (Socket socket = socketFactory.connect(endpoint, config.getConnectTimeoutMs())) {
                return socket.isConnected();
            } catch (IOException ex) {
                LOG.debug("Probe failed for {}", endpoint, ex);
                return false;
            }
        }
    }

    @Override
    public void close() {

View on GitHub (pinned to cf67b549a7)