apache/dolphinscheduler · error · ServiceException

Backfill workflow failed: %s

Error message

Backfill workflow failed: %s

What it means

Thrown after the API forwards a workflow-backfill trigger request to a master and the master responds with a non-success WorkflowBackfillTriggerResponse. The master rejected or failed the backfill; the master's own error message is wrapped into this ServiceException.

Source

Thrown at dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/executor/workflow/BackfillWorkflowExecutorDelegate.java:196

                .workflowCode(workflowDefinition.getCode())
                .workflowVersion(workflowDefinition.getVersion())
                .startNodes(backfillWorkflowDTO.getStartNodes())
                .failureStrategy(backfillWorkflowDTO.getFailureStrategy())
                .taskDependType(backfillWorkflowDTO.getTaskDependType())
                .warningType(backfillWorkflowDTO.getWarningType())
                .warningGroupId(backfillWorkflowDTO.getWarningGroupId())
                .workflowInstancePriority(backfillWorkflowDTO.getWorkflowInstancePriority())
                .workerGroup(backfillWorkflowDTO.getWorkerGroup())
                .tenantCode(backfillWorkflowDTO.getTenantCode())
                .environmentCode(backfillWorkflowDTO.getEnvironmentCode())
                .startParamList(backfillWorkflowDTO.getStartParamList())
                .dryRun(backfillWorkflowDTO.getDryRun())
                .build();

        final WorkflowBackfillTriggerResponse backfillTriggerResponse =
                triggerBackfillWorkflow(backfillTriggerRequest, masterServer);
        if (!backfillTriggerResponse.isSuccess()) {
            throw new ServiceException("Backfill workflow failed: " + backfillTriggerResponse.getMessage());
        }
        return backfillTriggerResponse.getWorkflowInstanceId();
    }

    protected WorkflowBackfillTriggerResponse triggerBackfillWorkflow(final WorkflowBackfillTriggerRequest request,
                                                                      final Server masterServer) {
        return Clients
                .withService(IWorkflowControlClient.class)
                .withHost(masterServer.getHost() + ":" + masterServer.getPort())
                .backfillTriggerWorkflow(request);
    }

    /**
     * Builds {@link BackfillWorkflowDTO} list for resolved downstream workflows.
     * {@link RunMode} in each downstream {@link BackfillWorkflowDTO.BackfillParamsDTO} matches the root (see
     * {@link #executeWithDependentExpansion(BackfillWorkflowDTO)}).
     */
    private List<BackfillWorkflowDTO> buildResolvedDownstreamBackfillDtos(final BackfillWorkflowDTO backfillWorkflowDTO,

View on GitHub (pinned to 02eac45a1b)

Solutions

1. Read backfillTriggerResponse.getMessage() in the exception text — it contains the master's root cause; fix that specific issue
2. Verify the workflow definition is valid and not deleted/modified before backfilling
3. Check master server logs at the time of the trigger for the underlying stack trace
4. Confirm API and master versions are compatible (same release line)
5. Retry the backfill after correcting parameters or cluster issues

Example fix

// before
BackfillWorkflowRequest req = BackfillWorkflowRequest.builder()
    .workflowDefinition(wf).backfillDateTimes(dates).dryRun(false).build();
// after: dry-run first to surface master-side rejection before real backfill
BackfillWorkflowRequest req = BackfillWorkflowRequest.builder()
    .workflowDefinition(wf).backfillDateTimes(dates)
    .dryRun(true) // validate first
    .build();

// pre-validate backfill inputs: definition exists and dates non-empty
if (wfDefinition == null || backfillDates.isEmpty()) throw new IllegalArgumentException("invalid backfill request");

try { delegate.backfill(dto, dates); } catch (ServiceException e) { log.error("Master rejected backfill: {}", e.getMessage()); throw new BackfillException(e.getMessage(), e); }

When it happens

Trigger: triggerBackfillWorkflow(...) returns a response whose isSuccess() is false — e.g. master-side validation of the backfill request failed, schedule/definition problems, or an internal master error during trigger processing.

Common situations: Backfilling with an invalid time range or command parameters the master rejects; workflow definition modified/deleted concurrently; master internal errors (DB failures, command insert failures); version mismatch between API and master producing malformed requests.

Related errors

• Recover workflow instance failed: %s
• WorkflowInstance: %s pause failed: %s
• no master server available
• The workflow instance: %s status is %s, can not pause
• WorkflowInstance: %s pause failed

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/9717ae171d2999e2. Report an issue: GitHub.