{"record":{"id":"69355916e0090fcb","repo":"apache/seatunnel","slug":"edge-transport-io-failure-will-reconnect-batchid","errorCode":null,"errorMessage":"Edge transport IO failure, will reconnect. batchId={}","messagePattern":"Edge transport IO failure, will reconnect\\. batchId=(.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-edge-agent/seatunnel-edge-agent-transport/src/main/java/org/apache/seatunnel/edge/agent/transport/socket/EdgeTransportClient.java","lineNumber":102,"sourceCode":"        synchronized (connectionLock) {\n            IOException lastIo = null;\n            InterruptedException lastInterrupted = null;\n            for (int cycle = 0; cycle < config.getMaxReconnectCycles(); cycle++) {\n                try {\n                    ensureAuthenticatedSession();\n                    lineTransport.sendBatchUntilReceived(\n                            activeSocket.reader, activeSocket.writer, batchId, payload);\n                    return;\n                } catch (EdgeSocketCollectorRejectedException ex) {\n                    invalidateSession();\n                    throw ex;\n                } catch (InterruptedException ex) {\n                    Thread.currentThread().interrupt();\n                    lastInterrupted = ex;\n                    break;\n                } catch (IOException ex) {\n                    lastIo = ex;\n                    LOG.warn(\"Edge transport IO failure, will reconnect. batchId={}\", batchId, ex);\n                    invalidateSession();\n                }\n            }\n            if (lastInterrupted != null) {\n                throw lastInterrupted;\n            }\n            if (lastIo != null) {\n                throw lastIo;\n            }\n            throw new IOException(\n                    \"sendUntilReceived exhausted reconnect cycles for batchId=\" + batchId);\n        }\n    }\n\n    @Override\n    public boolean probeReachable() throws IOException {\n        synchronized (connectionLock) {\n            try (Socket socket = socketFactory.connect(endpoint, config.getConnectTimeoutMs())) {","sourceCodeStart":84,"sourceCodeEnd":120,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-edge-agent/seatunnel-edge-agent-transport/src/main/java/org/apache/seatunnel/edge/agent/transport/socket/EdgeTransportClient.java#L84-L120","documentation":"EdgeTransportClient.sendUntilReceived retries a batch send in a loop. On IOException during a send it logs 'Edge transport IO failure, will reconnect', invalidates the current session, and loops to rebuild the connection and retry. Interrupted attempts break out and are rethrown as InterruptedException.","triggerScenarios":"send() or sendReconnectsWhenFirstSessionFails -> sendUntilReceived encounters IOException writing/reading on the socket (connection reset, broken pipe, read timeout) for the given batchId.","commonSituations":"Server restart or failover between sends, idle-connection timeouts on firewalls/load balancers killing the socket, or transient network blips in datacenter links.","solutions":["This is self-healing (session invalidated and retried); confirm subsequent logs show successful reconnect.","Check receiver logs for resets around the same batchId.","Enable TCP keepalive or reduce idle timeouts if load balancers drop quiet connections.","Increase transport timeouts if large batches routinely exceed limits."],"exampleFix":"// before\ntransport {\n  idle-timeout-ms = 10000\n}\n// after: keep connections alive across LB idle windows\ntransport {\n  idle-timeout-ms = 120000\n}","handlingStrategy":"retry","validationCode":"// Probe endpoint before sending large batches\nif (!client.canReachEndpoint()) { client.ensureAuthenticatedSession(); }","typeGuard":null,"tryCatchPattern":"try {\n    client.send(batchId, payload);\n} catch (IOException e) {\n    // sendUntilReceived already invalidates the session and retries internally;\n    // only handle exhaustion/interrupt at this layer\n}","preventionTips":["Enable TCP keepalive to survive firewall/LB idle timeouts.","Monitor reconnect frequency — occasional reconnects are normal, frequent ones indicate network issues.","Keep receiver instances behind stable endpoints (VIP/DNS) during failovers.","Size batches so a single send fits within the transport timeout."],"tags":["network","socket","reconnect","io"],"backgroundTag":"connection-refused","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}