{"record":{"id":"672ecfd47c2e72f5","repo":"apache/dolphinscheduler","slug":"connect-to-s-fail","errorCode":null,"errorMessage":"connect to : %s fail","messagePattern":"connect to : (.+?) fail","errorType":"exception","errorClass":"RemoteException","httpStatus":null,"severity":"error","filePath":"dolphinscheduler-extract/dolphinscheduler-extract-base/src/main/java/org/apache/dolphinscheduler/extract/base/client/NettyRemotingClient.java","lineNumber":163,"sourceCode":"                    throw (RemoteException) ex;\n                } else {\n                    throw new RemoteException(\"Call method to \" + host + \" failed\", ex);\n                }\n            } finally {\n                ClientSyncDurationMetrics clientSyncDurationMetrics = ClientSyncDurationMetrics\n                        .of(syncRequestDto)\n                        .withMilliseconds(System.currentTimeMillis() - start);\n                RpcMetrics.recordClientSyncRequestDuration(clientSyncDurationMetrics);\n            }\n        }\n    }\n\n    private IRpcResponse doSendSync(final Transporter transporter,\n                                    final Host serverHost,\n                                    long timeoutMills) throws RemoteException, InterruptedException {\n        final Channel channel = getOrCreateChannel(serverHost);\n        if (channel == null) {\n            throw new RemoteException(String.format(\"connect to : %s fail\", serverHost));\n        }\n        final ResponseFuture responseFuture = new ResponseFuture(transporter.getHeader().getOpaque(), timeoutMills);\n        channel.writeAndFlush(transporter).addListener(future -> {\n            if (future.isSuccess()) {\n                responseFuture.setSendOk(true);\n                return;\n            } else {\n                responseFuture.setSendOk(false);\n            }\n            responseFuture.setCause(future.cause());\n            responseFuture.putResponse(null);\n            log.error(\"Send Sync request {} to host {} failed\", transporter, serverHost, responseFuture.getCause());\n        });\n        /*\n         * sync wait for result\n         */\n        final IRpcResponse iRpcResponse = responseFuture.waitResponse();\n        if (iRpcResponse != null) {","sourceCodeStart":145,"sourceCodeEnd":181,"githubUrl":"https://github.com/apache/dolphinscheduler/blob/02eac45a1b6676e639fcbfb4be2243de5771b05d/dolphinscheduler-extract/dolphinscheduler-extract-base/src/main/java/org/apache/dolphinscheduler/extract/base/client/NettyRemotingClient.java#L145-L181","documentation":"doSendSync throws this RemoteException when getOrCreateChannel cannot obtain an active Netty channel to the target host — i.e. the TCP connection to the remote server could not be established. The RPC request was never written to the wire. The formatted message includes the Host (ip:port) that failed.","triggerScenarios":"sendSync -> doSendSync calls getOrCreateChannel(serverHost); the underlying bootstrap.connect fails (connection refused, timeout, DNS failure) and createChannel throws, or returns a channel that never becomes active, so null is returned and this RemoteException is thrown.","commonSituations":"Worker process is down or still booting; wrong port in worker registration; container hostname not resolvable from master pod (K8s headless service issues); host firewalled; connect timeout too short for slow network.","solutions":["Verify the remote process is running and listening on the configured port (ss -tlnp | grep <port> or telnet)","Confirm the Host ip/port from the registry (ZooKeeper) is correct and current; stale registry entries are common after an unclean shutdown","Check firewall/security-group rules allow TCP between master and worker","If in containers/K8s, verify DNS resolution of the host name from the calling pod","Increase netty client connect timeout if the network is slow, and retry the request"],"exampleFix":"// before\n// connect fails silently downstream, only generic exception seen\n// after\nChannel ch = client.getOrCreateChannel(host);\nif (ch == null || !ch.isActive()) {\n    log.error(\"Cannot connect to {}, check process/port/firewall\", host);\n}","handlingStrategy":"retry","validationCode":"boolean canConnect(Host host) {\n    try (Socket s = new Socket()) {\n        s.connect(new InetSocketAddress(host.getIp(), host.getPort()), 2000);\n        return true;\n    } catch (IOException e) {\n        log.warn(\"Host {} not connectable: {}\", host, e.getMessage());\n        return false;\n    }\n}","typeGuard":"null","tryCatchPattern":"try {\n    IRpcResponse resp = client.sendSync(host, request, timeout);\n} catch (RemoteException e) {\n    if (String.valueOf(e.getMessage()).startsWith(\"connect to\")) {\n        // pick another host from registry / schedule retry with backoff\n    }\n    throw e;\n}","preventionTips":["Ensure workers are started before masters begin dispatching","Monitor registry (ZooKeeper) liveness so dead workers are evicted promptly","Verify port and firewall rules between all master/worker pairs after infra changes","In K8s, confirm headless service and DNS names resolve correctly"],"tags":["rpc","network","connection-refused","netty"],"backgroundTag":"connection-refused","analyzedSha":"02eac45a1b6676e639fcbfb4be2243de5771b05d","analyzedAt":"2026-09-06T17:43:00.555Z","contentChangedAt":"2026-09-06T17:43:00.555Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}