apache/dolphinscheduler · error · ServiceException

no master server available

Error message

no master server available

What it means

Thrown when the API cannot find a master server in the registry while attempting to recover a suspended workflow instance. The recovery request must be sent to a master; an empty master list aborts the operation.

Source

Thrown at dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/executor/workflow/RecoverSuspendedWorkflowInstanceExecutorDelegate.java:52

@Component
public class RecoverSuspendedWorkflowInstanceExecutorDelegate
        implements
            IExecutorDelegate<RecoverSuspendedWorkflowInstanceExecutorDelegate.RecoverSuspendedWorkflowInstanceOperation, Void> {

    @Autowired
    private RegistryClient registryClient;

    @Override
    public Void execute(RecoverSuspendedWorkflowInstanceOperation workflowInstanceControlRequest) {
        final WorkflowInstance workflowInstance = workflowInstanceControlRequest.workflowInstance;
        if (!workflowInstance.getState().isPaused() && !workflowInstance.getState().isStopped()) {
            throw new ServiceException(
                    String.format("The workflow instance: %s state is %s, cannot recovery", workflowInstance.getName(),
                            workflowInstance.getState()));
        }
        final Server masterServer = registryClient.getRandomServer(RegistryNodeType.MASTER).orElse(null);
        if (masterServer == null) {
            throw new ServiceException("no master server available");
        }
        final WorkflowInstanceRecoverSuspendTasksRequest recoverSuspendTaskRequest =
                WorkflowInstanceRecoverSuspendTasksRequest.builder()
                        .workflowInstanceId(workflowInstance.getId())
                        .userId(workflowInstanceControlRequest.executeUser.getId())
                        .build();

        final WorkflowInstanceRecoverSuspendTasksResponse recoverSuspendTaskResponse = Clients
                .withService(IWorkflowControlClient.class)
                .withHost(masterServer.getHost() + ":" + masterServer.getPort())
                .triggerFromSuspendTasks(recoverSuspendTaskRequest);
        if (!recoverSuspendTaskResponse.isSuccess()) {
            throw new ServiceException("Recover workflow instance failed: " + recoverSuspendTaskResponse.getMessage());
        }
        return null;
    }

    public static class RecoverSuspendedWorkflowInstanceOperation {

View on GitHub (pinned to 02eac45a1b)

Solutions

  1. Ensure at least one MasterServer is running and registered in ZooKeeper
  2. Align registry configuration between masters, workers, and the API server
  3. Check ZooKeeper availability and network paths; restart failed masters
  4. Retry the recover operation once the master list is populated

Example fix

// before
api.recoverSuspendedWorkflowInstance(instanceId, userId);
// after
if (registryClient.getRandomServer(RegistryNodeType.MASTER).isPresent()) {
    api.recoverSuspendedWorkflowInstance(instanceId, userId);
} else {
    throw new IllegalStateException("Cluster has no active master server");
}
Defensive patterns

Strategy: validation

Validate before calling

boolean masterAvailable = registryClient.getRandomServer(RegistryNodeType.MASTER).isPresent();
if (!masterAvailable) throw new IllegalStateException("No master registered; cannot recover suspended instance");

Type guard

boolean hasMaster(RegistryClient c) { return c.getRandomServer(RegistryNodeType.MASTER).isPresent(); }

Try / catch

try { delegate.recoverSuspended(op); } catch (ServiceException e) { if (e.getMessage().contains("no master server available")) { waitForMasterAndRetry(op); } else { throw e; } }

Prevention

When it happens

Trigger: RecoverSuspendedWorkflowInstanceExecutorDelegate.execute calls registryClient.getRandomServer(RegistryNodeType.MASTER) after the state check and gets an empty result — no masters registered.

Common situations: All masters stopped for maintenance while users attempt recoveries; ZooKeeper connectivity problems hiding healthy masters; master registration failures after a config change; partial cluster startup.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/21261101f24c33ad. Report an issue: GitHub.