apache/dolphinscheduler · error · ServiceException

no master server available

Error message

no master server available

What it means

Thrown by the API layer's backfill delegate when no master server can be found in the registry. Backfilling a workflow requires forwarding the trigger request to a live master node; if the registry reports no master, the API cannot proceed and throws ServiceException before any request is sent.

Source

Thrown at dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/executor/workflow/BackfillWorkflowExecutorDelegate.java:168

        int start = 0;
        for (int i = 0; i < numParts; i++) {
            int currentSize = baseSize;
            if (i == numParts - 1) {
                currentSize += remainder;
            }
            List<ZonedDateTime> part = dateTimeList.subList(start, start + currentSize);
            result.add(part);
            start += currentSize;
        }

        return result;
    }

    private Integer doBackfillWorkflow(final BackfillWorkflowDTO backfillWorkflowDTO,
                                       final List<ZonedDateTime> backfillDateTimes) {
        final Server masterServer = registryClient.getRandomServer(RegistryNodeType.MASTER).orElse(null);
        if (masterServer == null) {
            throw new ServiceException("no master server available");
        }

        final List<String> backfillTimeList =
                backfillDateTimes.stream().map(DateUtils::dateToString).collect(Collectors.toList());

        final WorkflowDefinition workflowDefinition = backfillWorkflowDTO.getWorkflowDefinition();
        final WorkflowBackfillTriggerRequest backfillTriggerRequest = WorkflowBackfillTriggerRequest.builder()
                .userId(backfillWorkflowDTO.getLoginUser().getId())
                .backfillTimeList(backfillTimeList)
                .workflowCode(workflowDefinition.getCode())
                .workflowVersion(workflowDefinition.getVersion())
                .startNodes(backfillWorkflowDTO.getStartNodes())
                .failureStrategy(backfillWorkflowDTO.getFailureStrategy())
                .taskDependType(backfillWorkflowDTO.getTaskDependType())
                .warningType(backfillWorkflowDTO.getWarningType())
                .warningGroupId(backfillWorkflowDTO.getWarningGroupId())
                .workflowInstancePriority(backfillWorkflowDTO.getWorkflowInstancePriority())
                .workerGroup(backfillWorkflowDTO.getWorkerGroup())

View on GitHub (pinned to 02eac45a1b)

Solutions

1. Start at least one MasterServer and verify it registers in ZooKeeper under the masters node
2. Check masters' application.yaml registry (ZooKeeper) connect settings match the API server's registry settings
3. Verify ZooKeeper connectivity from the API host (zkCli / telnet to the quorum)
4. Check master logs for registration/heartbeat failures and restart failed masters
5. Re-run the backfill once the cluster shows the master as healthy

Example fix

// before: blindly calling backfill API with cluster down -> 500 'no master server available'
apiService.backfillWorkflow(...)
// after: pre-check cluster health before invoking
if (registryClient.getRandomServer(RegistryNodeType.MASTER).isEmpty()) {
    throw new IllegalStateException("Start a master server before backfilling workflows");
}
apiService.backfillWorkflow(...)

boolean masterAvailable = !registryClient.getRandomServer(RegistryNodeType.MASTER).orElse(new Server()).getHost().isEmpty();
if (!masterAvailable) throw new IllegalStateException("No master server registered; cannot backfill");

boolean hasMaster(RegistryClient c) { return c.getRandomServer(RegistryNodeType.MASTER).isPresent(); }

try { delegate.backfill(dto, dates); } catch (ServiceException e) { if (e.getMessage().contains("no master server available")) { alertOpsClusterDown(); } else { throw e; } }

When it happens

Trigger: Calling the workflow backfill API (BackfillWorkflowExecutorDelegate.doBackfillWorkflow) while registryClient.getRandomServer(RegistryNodeType.MASTER) returns empty — i.e. zero registered master nodes in ZooKeeper.

Common situations: All master services are down or crashed; masters failed to register with ZooKeeper (wrong registry address in master config); ZooKeeper itself is unreachable so the server list is empty; network partition between API server and masters; masters still starting up during cluster rollout.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

• no master server available
• no master server available
• Backfill workflow failed: %s
• The workflow instance: %s status is %s, can not pause
• WorkflowInstance: %s pause failed: %s

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/3ff7e2b68c705a6c. Report an issue: GitHub.