apache/dolphinscheduler · error · EmrTaskException

can not parse AddJobFlowStepsRequest from json: ${jobStepDef

Error message

can not parse AddJobFlowStepsRequest from json: ${jobStepDefineJson}

What it means

createAddJobFlowStepsRequest caught JsonProcessingException: after placeholder substitution, the stepsDefineJson string could not be deserialized into AWS SDK's AddJobFlowStepsRequest — the JSON is syntactically invalid or does not match the request schema (wrong Steps array shape, bad instance config). The offending input is the stepsDefineJson parameter; nothing has been submitted to EMR yet.

Source

Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-emr/src/main/java/org/apache/dolphinscheduler/plugin/task/emr/EmrAddStepsTask.java:138

    }

    /**
     * parse json string to AddJobFlowStepsRequest
     *
     * @return AddJobFlowStepsRequest
     */
    protected AddJobFlowStepsRequest createAddJobFlowStepsRequest() {

        final AddJobFlowStepsRequest addJobFlowStepsRequest;
        String jobStepDefineJson = null;
        try {
            jobStepDefineJson = ParameterUtils.convertParameterPlaceholders(
                    emrParameters.getStepsDefineJson(),
                    ParameterUtils.convert(taskExecutionContext.getPrepareParamsMap()));
            addJobFlowStepsRequest =
                    objectMapper.readValue(jobStepDefineJson, AddJobFlowStepsRequest.class);
        } catch (JsonProcessingException e) {
            throw new EmrTaskException("can not parse AddJobFlowStepsRequest from json: " + jobStepDefineJson, e);
        }

        // When a single task definition is associated with multiple steps, the state tracking will have high
        // complexity.
        // Therefore, A task definition only supports the association of a single step, which can better ensure the
        // reliability of the task state.
        if (addJobFlowStepsRequest.getSteps().size() > 1) {
            throw new EmrTaskException("ds emr addJobFlowStepsTask only support one step");
        }

        return addJobFlowStepsRequest;
    }

    /**
     * calculate task exitStatusCode
     *
     * @param stepStatus aws emr execution status details of the cluster step.
     * @return exitStatusCode

View on GitHub (pinned to 02eac45a1b)

Solutions

  1. Validate stepsDefineJson against the AddJobFlowStepsRequest schema (Steps[], Name, ActionOnFailure, HadoopJarStep)
  2. Check that placeholder substitution did not leave unexpanded ${var} tokens or break quoting
  3. Test the JSON with a schema validator or a local objectMapper round-trip before saving the task
Defensive patterns

Strategy: try-catch

When it happens

Trigger: Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-emr/src/main/java/org/apache/dolphinscheduler/plugin/task/emr/EmrAddStepsTask.java:138 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/3ef632c9649212db. Report an issue: GitHub.