{"record":{"id":"ff0b0f4dbf9f7c6d","repo":"apache/dolphinscheduler","slug":"remoteexception-serverhost-tostring-cause","errorCode":null,"errorMessage":"RemoteException(serverHost.toString(), cause)","messagePattern":"RemoteException\\(serverHost\\.toString\\(\\), cause\\)","errorType":"exception","errorClass":"RemoteException","httpStatus":null,"severity":"error","filePath":"dolphinscheduler-extract/dolphinscheduler-extract-base/src/main/java/org/apache/dolphinscheduler/extract/base/client/NettyRemotingClient.java","lineNumber":187,"sourceCode":"                return;\n            } else {\n                responseFuture.setSendOk(false);\n            }\n            responseFuture.setCause(future.cause());\n            responseFuture.putResponse(null);\n            log.error(\"Send Sync request {} to host {} failed\", transporter, serverHost, responseFuture.getCause());\n        });\n        /*\n         * sync wait for result\n         */\n        final IRpcResponse iRpcResponse = responseFuture.waitResponse();\n        if (iRpcResponse != null) {\n            return iRpcResponse;\n        }\n        if (responseFuture.isSendOK()) {\n            throw new RemoteTimeoutException(serverHost.toString(), timeoutMills, responseFuture.getCause());\n        } else {\n            throw new RemoteException(serverHost.toString(), responseFuture.getCause());\n        }\n    }\n\n    Channel getOrCreateChannel(Host host) {\n        Channel channel = channels.get(host);\n        if (channel != null && channel.isActive()) {\n            return channel;\n        }\n        try {\n            channelsLock.lock();\n            channel = channels.get(host);\n            if (channel != null && channel.isActive()) {\n                return channel;\n            }\n            channel = createChannel(host);\n            channels.put(host, channel);\n        } finally {\n            channelsLock.unlock();","sourceCodeStart":169,"sourceCodeEnd":205,"githubUrl":"https://github.com/apache/dolphinscheduler/blob/02eac45a1b6676e639fcbfb4be2243de5771b05d/dolphinscheduler-extract/dolphinscheduler-extract-base/src/main/java/org/apache/dolphinscheduler/extract/base/client/NettyRemotingClient.java#L169-L205","documentation":"RemoteException(host, cause) thrown by doSendSync when no response was received AND the request was never successfully written to the channel (isSendOK() is false). responseFuture.getCause() holds the underlying failure — typically the write failed because the channel closed mid-flight or the async write listener reported an error. Distinguish this from the timeout case (572): here the send itself failed.","triggerScenarios":"sendSync -> doSendSync: channel.writeAndFlush future fails (connection reset/closed between getOrCreateChannel and the write, TCP RST, half-closed socket), so setSendOk(true) is never called; waitResponse() then returns null with a cause set and isSendOK false.","commonSituations":"Worker restarted mid-request (connection dropped); idle connection reaped by an intermediary (NAT/LB timeout) but still cached in the client's channels map; TLS/protocol mismatch closing the socket; remote port briefly unavailable during rolling restarts.","solutions":["Inspect responseFuture.getCause() (via the exception cause) for the write failure root cause","Clear/rebuild the cached channel: connections can go stale — retry once with a fresh connection","Check whether the remote host was restarting or killing connections (rolling upgrade window)","Ensure Netty client-side idle detection/reconnect is configured so dead channels are evicted before writes","Add retry with backoff around sendSync for transient connection resets"],"exampleFix":"// before\nIRpcResponse resp = client.sendSync(host, request, timeout); // fails on stale channel\n// after\nIRpcResponse resp;\ntry {\n    resp = client.sendSync(host, request, timeout);\n} catch (RemoteException e) {\n    client.closeChannel(host); // drop stale connection\n    resp = client.sendSync(host, request, timeout); // retry on fresh channel\n}","handlingStrategy":"retry","validationCode":"// Evict stale channels before retrying:\nChannel ch = client.getOrCreateChannel(host);\nif (ch == null || !ch.isActive()) {\n    client.closeChannel(host); // force fresh connection on next call\n}","typeGuard":"null","tryCatchPattern":"try {\n    return client.sendSync(host, request, timeout);\n} catch (RemoteException e) {\n    if (e.getCause() != null && !isTimeout(e)) { // send failed, not timed out\n        client.closeChannel(host);              // drop stale connection\n        return client.sendSync(host, request, timeout); // one retry on fresh channel\n    }\n    throw e;\n}","preventionTips":["Enable Netty idle-state handling so dead cached channels are evicted before writes fail","Avoid long-lived idle connections through NAT/LBs — set TCP keepalive or reconnect periodically","Coordinate retries with rolling-restart windows so connection resets aren't mistaken for real failures","Always inspect getCause(): write failures vs timeouts require different retry policies"],"tags":["rpc","network","connection-reset"],"backgroundTag":"broken-pipe","analyzedSha":"02eac45a1b6676e639fcbfb4be2243de5771b05d","analyzedAt":"2026-09-06T17:43:00.555Z","contentChangedAt":"2026-09-06T17:43:00.555Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}