What causes "Execute emr task failed" in apache/dolphinscheduler?

Expected on kill — verify the exit status handling in finally recorded the correct state and cancel orphaned EMR steps if needed

apache/dolphinscheduler · warning · TaskException

Execute emr task failed

Error message

Execute emr task failed

What it means

EmrAddStepsTask.trackApplicationStatus polls ListSteps/DescribeStep for step completion. SdkBaseException and EmrTaskException during polling are only logged (task continues with last-known stepStatus), but an InterruptedException is rethrown as TaskException 'Execute emr task failed' after restoring the interrupt flag. This signals the polling thread was interrupted — normally a task kill or worker shutdown.

Source

Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-emr/src/main/java/org/apache/dolphinscheduler/plugin/task/emr/EmrAddStepsTask.java:114

            setExitStatusCode(exitStatusCode);
            log.info("emr task finished with step status : {}", stepStatus);
        }
    }

    @Override
    public void trackApplicationStatus() throws TaskException {
        StepStatus stepStatus = getStepStatus();

        try {
            while (waitingStateSet.contains(stepStatus.getState())) {
                TimeUnit.SECONDS.sleep(10);
                stepStatus = getStepStatus();
            }
        } catch (EmrTaskException | SdkBaseException e) {
            log.error("emr task failed with error", e);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            throw new TaskException("Execute emr task failed", e);
        } finally {
            final int exitStatusCode = calculateExitStatusCode(stepStatus);
            setExitStatusCode(exitStatusCode);
            log.info("emr task finished with step status : {}", stepStatus);
        }
    }

    /**
     * parse json string to AddJobFlowStepsRequest
     *
     * @return AddJobFlowStepsRequest
     */
    protected AddJobFlowStepsRequest createAddJobFlowStepsRequest() {

        final AddJobFlowStepsRequest addJobFlowStepsRequest;
        String jobStepDefineJson = null;
        try {
            jobStepDefineJson = ParameterUtils.convertParameterPlaceholders(

View on GitHub (pinned to 02eac45a1b)

Solutions

Expected on kill — verify the exit status handling in finally recorded the correct state and cancel orphaned EMR steps if needed
If unexpected, check worker logs for shutdown or kill events at that timestamp
Adjust task timeout so long steps are not interrupted by scheduler timeouts
For step cleanup on kill, use the EMR console/AWS CLI to cancel pending steps of the cluster

Defensive patterns

Strategy: try-catch

Try / catch

try {
    task.trackApplicationStatus();
} catch (TaskException e) {
    if (e.getCause() instanceof InterruptedException) {
        log.warn("EMR tracking interrupted (kill/shutdown); step may still run in AWS");
        // follow up: cancel pending steps via emrClient.cancelSteps if needed
    } else {
        throw e;
    }
}

Prevention

Set workflow/task timeouts above the longest expected EMR step duration
On kill, cancel outstanding EMR steps (emr cancel-steps) to avoid orphaned work
Use ActionOnFailure=TERMINATE_CLUSTER to bound step runtimes on failure
Schedule worker deployments outside heavy EMR job windows to reduce drain interruptions

When it happens

Trigger: Thread.interrupt() while the polling loop sleeps between step-status checks — workflow stop/kill, worker shutdown, or timeout-based cancellation by the scheduler.

Common situations: User kills a long-running EMR step workflow; worker drain during deployment; retry/timeout policy interrupts the tracker. The EMR step itself keeps running in AWS unless the step's ActionOnFailure/cancel handling stops it.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/c07cb87fab1234e0. Report an issue: GitHub.