apache/seatunnel · error · IOException

sendUntilReceived exhausted reconnect cycles for batchId=${b

Error message

sendUntilReceived exhausted reconnect cycles for batchId=${batchId}

What it means

EdgeTransportClient.sendUntilReceived retries a batch across full reconnect cycles: each cycle opens a session, authenticates, and tries sendBatchUntilReceived. If all reconnect cycles fail, the last recorded InterruptedException/IOException is rethrown if present; only when no underlying exception was captured does it throw this fallback IOException, meaning the batch could not be confirmed after every reconnect attempt.

Source

Thrown at seatunnel-edge-agent/seatunnel-edge-agent-transport/src/main/java/org/apache/seatunnel/edge/agent/transport/socket/EdgeTransportClient.java:112

                    invalidateSession();
                    throw ex;
                } catch (InterruptedException ex) {
                    Thread.currentThread().interrupt();
                    lastInterrupted = ex;
                    break;
                } catch (IOException ex) {
                    lastIo = ex;
                    LOG.warn("Edge transport IO failure, will reconnect. batchId={}", batchId, ex);
                    invalidateSession();
                }
            }
            if (lastInterrupted != null) {
                throw lastInterrupted;
            }
            if (lastIo != null) {
                throw lastIo;
            }
            throw new IOException(
                    "sendUntilReceived exhausted reconnect cycles for batchId=" + batchId);
        }
    }

    @Override
    public boolean probeReachable() throws IOException {
        synchronized (connectionLock) {
            try (Socket socket = socketFactory.connect(endpoint, config.getConnectTimeoutMs())) {
                return socket.isConnected();
            } catch (IOException ex) {
                LOG.debug("Probe failed for {}", endpoint, ex);
                return false;
            }
        }
    }

    @Override
    public void close() {

View on GitHub (pinned to cf67b549a7)

Solutions

Check earlier log lines for the per-cycle root cause (auth rejection, EOF, unexpected response) and fix that root cause first
Increase the reconnect-cycle limit / backoff in EdgeTransportConfig for longer outages
Persist the unacknowledged batchId (WAL) and replay it after the collector recovers
Verify the collector address and credentials, since non-IO failures (rejection, illegal state) are the usual silent causes

Example fix

// before
client.send(batchId, payload); // throws after all cycles
// after
try {
    client.send(batchId, payload);
} catch (IOException e) {
    wal.persist(batchId, payload); // replay later with same batchId
    throw e;
}

Defensive patterns

Strategy: retry

Try / catch

try { client.send(batchId, payload); } catch (IOException e) { if (e.getMessage().contains("exhausted reconnect cycles")) { wal.replayLater(batchId, payload); } throw e; }

Prevention

Read the per-cycle root cause in logs; fix auth/config issues that fail every cycle non-IO
Set reconnect-cycle limits and backoffs to cover realistic outage lengths
Keep a WAL to replay batches that never received RECEIVED
Alert on this error — it means a batch is unacknowledged and needs replay

When it happens

Trigger: Every reconnect cycle fails in a way that stores neither lastIo nor lastInterrupted (e.g. open aborted via IllegalStateException/EdgeSocketCollectorRejectedException repeatedly), exhausting the cycle budget for the given batchId.

Common situations: Collector persistently rejecting auth so each cycle fails before an IOException is recorded; sustained collector outage combined with a low reconnect-cycle limit; misconfigured host/port making every cycle abort non-IO.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/82b4fb8d4cb1df73. Report an issue: GitHub.