apache/dolphinscheduler · error · TaskException

Execute emr task failed

Error message

Execute emr task failed

What it means

A catch-all in EmrJobFlowTask.trackApplicationStatus for non-SDK exceptions raised while polling the cluster status loop (getClusterStatus, sleep-based waiting) — e.g. an IllegalStateException from an unexpected cluster state mapping or a runtime error in status interpretation. SDK errors are logged separately; this branch rethrows as a generic task-execution failure with the cause attached.

Source

Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-emr/src/main/java/org/apache/dolphinscheduler/plugin/task/emr/EmrJobFlowTask.java:108

            log.info("emr task finished with cluster status : {}", clusterStatus);
        }
    }

    @Override
    public void trackApplicationStatus() throws TaskException {
        ClusterStatus clusterStatus = null;
        try {
            clusterStatus = getClusterStatus();

            while (waitingStateSet.contains(clusterStatus.getState())) {
                TimeUnit.SECONDS.sleep(10);
                clusterStatus = getClusterStatus();
            }
        } catch (EmrTaskException | SdkBaseException e) {
            log.error("emr task failed with error", e);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            throw new TaskException("Execute emr task failed", e);
        } finally {
            final int exitStatusCode = calculateExitStatusCode(clusterStatus);
            setExitStatusCode(exitStatusCode);
            log.info("emr task finished with cluster status : {}", clusterStatus);
        }
    }

    /**
     * parse json string to RunJobFlowRequest
     *
     * @return RunJobFlowRequest
     */
    protected RunJobFlowRequest createRunJobFlowRequest() {

        final RunJobFlowRequest runJobFlowRequest;
        String jobFlowDefineJson = null;
        try {
            jobFlowDefineJson = ParameterUtils.convertParameterPlaceholders(

View on GitHub (pinned to 02eac45a1b)

Solutions

1. Inspect the caused-by exception to identify which status/state mapping failed
2. Check the EMR cluster's actual state in the AWS console — a state outside the waiting/terminal sets triggers the failure path
3. Handle the cluster state in the exit-code switch or report the state via the AWS API

When it happens

Trigger: Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-emr/src/main/java/org/apache/dolphinscheduler/plugin/task/emr/EmrJobFlowTask.java:108 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/b2e7f46bf371004d. Report an issue: GitHub.