apache/dolphinscheduler · error · EmrTaskException

can not parse RunJobFlowRequest from json: ${jobFlowDefineJs

Error message

can not parse RunJobFlowRequest from json: ${jobFlowDefineJson}

What it means

createRunJobFlowRequest caught JsonProcessingException: after placeholder substitution, jobFlowDefineJson could not be deserialized into AWS SDK's RunJobFlowRequest — invalid JSON syntax or a structure not matching the request schema (wrong Name/Instances/Steps fields). The offending input is the jobFlowDefineJson task parameter; no AWS call has been made yet.

Source

Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-emr/src/main/java/org/apache/dolphinscheduler/plugin/task/emr/EmrJobFlowTask.java:131

        }
    }

    /**
     * parse json string to RunJobFlowRequest
     *
     * @return RunJobFlowRequest
     */
    protected RunJobFlowRequest createRunJobFlowRequest() {

        final RunJobFlowRequest runJobFlowRequest;
        String jobFlowDefineJson = null;
        try {
            jobFlowDefineJson = ParameterUtils.convertParameterPlaceholders(
                    emrParameters.getJobFlowDefineJson(),
                    ParameterUtils.convert(taskExecutionContext.getPrepareParamsMap()));
            runJobFlowRequest = objectMapper.readValue(jobFlowDefineJson, RunJobFlowRequest.class);
        } catch (JsonProcessingException e) {
            throw new EmrTaskException("can not parse RunJobFlowRequest from json: " + jobFlowDefineJson, e);
        }

        return runJobFlowRequest;
    }

    /**
     * calculate task exitStatusCode
     *
     * @param clusterStatus aws emr cluster status
     * @return exitStatusCode
     */
    private int calculateExitStatusCode(ClusterStatus clusterStatus) {
        if (clusterStatus == null) {
            return TaskConstants.EXIT_CODE_FAILURE;
        } else {
            String state = clusterStatus.getState();
            ClusterStateChangeReason stateChangeReason = clusterStatus.getStateChangeReason();
            ClusterState clusterState = ClusterState.valueOf(state);

View on GitHub (pinned to 02eac45a1b)

Solutions

  1. Validate jobFlowDefineJson against the RunJobFlowRequest schema (Applications, Instances, JobFlowRole, ServiceRole, Steps)
  2. Check placeholder substitution output — unexpanded ${var} tokens or broken quoting cause parse failure
  3. Round-trip test the JSON in the task definition before execution
Defensive patterns

Strategy: try-catch

When it happens

Trigger: Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-emr/src/main/java/org/apache/dolphinscheduler/plugin/task/emr/EmrJobFlowTask.java:131 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/d116932a2b74e56f. Report an issue: GitHub.