What causes "Call method to " + host + " failed" in apache/dolphinscheduler?

Check that the target host is reachable: ping/telnet the host IP and port configured in the registry (e.g. nc -zv )

apache/dolphinscheduler · error · RemoteException

Call method to " + host + " failed

Error message

Call method to " + host + " failed

What it means

NettyRemotingClient.sendSync wraps any non-RemoteException throwable raised while sending an RPC request synchronously to a remote worker/master host into a generic RemoteException. It means the synchronous call to the remote host failed for a reason other than a remote-side business error, e.g. connection setup failure, write failure, or timeout surfaced as a local exception. The original exception is attached as the cause for diagnosis.

Source

Thrown at dolphinscheduler-extract/dolphinscheduler-extract-base/src/main/java/org/apache/dolphinscheduler/extract/base/client/NettyRemotingClient.java:147

                return doSendSync(transporter, host, timeoutMillis);
            } catch (Exception ex) {
                ClientSyncExceptionMetrics clientSyncExceptionMetrics =
                        ClientSyncExceptionMetrics.of(syncRequestDto, ex);
                RpcMetrics.recordClientSyncRequestException(clientSyncExceptionMetrics);

                if (currentExecuteTimes < maxRetryTimes
                        && Arrays.stream(retryStrategy.retryFor()).anyMatch(e -> e.isInstance(ex))) {
                    currentExecuteTimes++;
                    if (retryStrategy.retryInterval() > 0) {
                        ThreadUtils.sleep(retryStrategy.retryInterval());
                    }
                    continue;
                }

                if (ex instanceof RemoteException) {
                    throw (RemoteException) ex;
                } else {
                    throw new RemoteException("Call method to " + host + " failed", ex);
                }
            } finally {
                ClientSyncDurationMetrics clientSyncDurationMetrics = ClientSyncDurationMetrics
                        .of(syncRequestDto)
                        .withMilliseconds(System.currentTimeMillis() - start);
                RpcMetrics.recordClientSyncRequestDuration(clientSyncDurationMetrics);
            }
        }
    }

    private IRpcResponse doSendSync(final Transporter transporter,
                                    final Host serverHost,
                                    long timeoutMills) throws RemoteException, InterruptedException {
        final Channel channel = getOrCreateChannel(serverHost);
        if (channel == null) {
            throw new RemoteException(String.format("connect to : %s fail", serverHost));
        }
        final ResponseFuture responseFuture = new ResponseFuture(transporter.getHeader().getOpaque(), timeoutMills);

View on GitHub (pinned to 02eac45a1b)

Solutions

Check that the target host is reachable: ping/telnet the host IP and port configured in the registry (e.g. nc -zv <ip> <port>)
Inspect the chained cause in the exception (ex.getCause()) to find the root failure (timeout, refused, interrupted)
Verify the host registered in ZooKeeper matches the actual worker/master address and port; restart the remote process if stale
Check network/firewall/security-group rules between master and worker on the DolphinScheduler RPC port
Retry the operation; if it happens persistently, check remote JVM GC pauses or thread pool exhaustion on the remote side

Example fix

// before
IRpcResponse resp = nettyRemotingClient.sendSync(host, request, 3000);
// after
IRpcResponse resp;
try {
    resp = nettyRemotingClient.sendSync(host, request, 30000);
} catch (RemoteException e) {
    log.error("RPC to {} failed: {}", host, e.getCause(), e);
    throw e;
}

Defensive patterns

Strategy: try-catch

Validate before calling

boolean isHostReachable(Host host) {
    try (Socket s = new Socket()) {
        s.connect(new InetSocketAddress(host.getIp(), host.getPort()), 3000);
        return true;
    } catch (IOException e) {
        return false;
    }
}

Type guard

null

Try / catch

try {
    IRpcResponse resp = client.sendSync(host, request, timeoutMills);
} catch (RemoteTimeoutException e) {
    // retry with larger timeout
} catch (RemoteException e) {
    log.error("RPC to {} failed, cause: {}", host, e.getCause(), e);
    // failover to another host or rethrow
}

Prevention

Always log e.getCause() of RemoteException — the root network error is there
Pre-flight check host reachability before dispatching tasks to a newly registered worker
Keep worker registration (ZooKeeper) consistent with actual listening ports
Retry idempotent RPCs with exponential backoff

When it happens

Trigger: Calling sendSync (via a generated client proxy, e.g. ISchedulerClient or IWorkflowClient) when the target host is down, unreachable, the channel write fails, doSendSync throws a non-RemoteException (IllegalArgumentException from createChannel, RuntimeException on InterruptedException), or the await is interrupted/fails unexpectedly.

Common situations: Worker or master process crashed mid-deployment; DNS or /etc/hosts entries pointing to a stale IP; firewall or security group blocking the RPC port; cluster scaled down but scheduler config still references removed hosts; network partition between master and worker nodes.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/34768bca2bcdabf3. Report an issue: GitHub.