apache/dolphinscheduler · error · EmrTaskException

ds emr addJobFlowStepsTask only support one step

Error message

ds emr addJobFlowStepsTask only support one step

What it means

A semantic validation guard after parsing stepsDefineJson: DolphinScheduler only orchestrates a single EMR step per task, so an AddJobFlowStepsRequest whose Steps list contains more than one step is rejected outright. The offending input is the stepsDefineJson containing multiple step entries; it is a usage error, not an AWS failure.

Source

Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-emr/src/main/java/org/apache/dolphinscheduler/plugin/task/emr/EmrAddStepsTask.java:146

        final AddJobFlowStepsRequest addJobFlowStepsRequest;
        String jobStepDefineJson = null;
        try {
            jobStepDefineJson = ParameterUtils.convertParameterPlaceholders(
                    emrParameters.getStepsDefineJson(),
                    ParameterUtils.convert(taskExecutionContext.getPrepareParamsMap()));
            addJobFlowStepsRequest =
                    objectMapper.readValue(jobStepDefineJson, AddJobFlowStepsRequest.class);
        } catch (JsonProcessingException e) {
            throw new EmrTaskException("can not parse AddJobFlowStepsRequest from json: " + jobStepDefineJson, e);
        }

        // When a single task definition is associated with multiple steps, the state tracking will have high
        // complexity.
        // Therefore, A task definition only supports the association of a single step, which can better ensure the
        // reliability of the task state.
        if (addJobFlowStepsRequest.getSteps().size() > 1) {
            throw new EmrTaskException("ds emr addJobFlowStepsTask only support one step");
        }

        return addJobFlowStepsRequest;
    }

    /**
     * calculate task exitStatusCode
     *
     * @param stepStatus aws emr execution status details of the cluster step.
     * @return exitStatusCode
     */
    private int calculateExitStatusCode(StepStatus stepStatus) {
        if (stepStatus == null) {
            return TaskConstants.EXIT_CODE_FAILURE;
        } else {
            String state = stepStatus.getState();
            StepState stepState = StepState.valueOf(state);
            switch (stepState) {

View on GitHub (pinned to 02eac45a1b)

Solutions

1. Reduce stepsDefineJson to exactly one entry in the Steps array
2. Chain multiple EMR AddSteps task nodes in the workflow instead of embedding several steps in one task

When it happens

Trigger: Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-emr/src/main/java/org/apache/dolphinscheduler/plugin/task/emr/EmrAddStepsTask.java:146 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/5b07e2c3f256fab2. Report an issue: GitHub.