apache/dolphinscheduler · error · AliyunServerlessSparkTaskException

Failed to get job run!

Error message

Failed to get job run!

What it means

The Aliyun Serverless Spark task wraps any exception from the getJobRun API call into AliyunServerlessSparkTaskException with this message, then retries it per the retry policy. It means the polling call to query the job run's state failed (network, auth, or API error) and the actual job state could not be determined.

Source

Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-aliyunserverlessspark/src/main/java/org/apache/dolphinscheduler/plugin/task/aliyunserverlessspark/AliyunServerlessSparkTask.java:174

            } catch (Exception e) {
                throw new AliyunServerlessSparkTaskException("Failed to start job run! ", e);
            }
        }, retryPolicy);

        jobRunId = startJobRunResponse.getBody().getJobRunId();
        setAppIds(jobRunId);
        log.info("Successfully submitted serverless spark job, jobRunId - {}", jobRunId);

        while (!RunState.isFinal(currentState)) {
            GetJobRunRequest getJobRunRequest = buildGetJobRunRequest();

            GetJobRunResponse getJobRunResponse = RetryUtils.retryFunction(() -> {
                try {
                    return aliyunServerlessSparkClient
                            .getJobRun(aliyunServerlessSparkParameters.getWorkspaceId(), jobRunId,
                                    getJobRunRequest);
                } catch (Exception e) {
                    throw new AliyunServerlessSparkTaskException("Failed to get job run!", e);
                }
            }, retryPolicy);

            currentState = RunState.valueOf(getJobRunResponse.getBody().getJobRun().getState());
            log.info("job - {} state - {}", jobRunId, currentState);

            try {
                Thread.sleep(10 * 1000L);
            } catch (InterruptedException e) {
                break;
            }
        }

        setExitStatusCode(mapFinalStateToExitCode(currentState));
    }

    @Override
    public void submitApplication() throws TaskException {

View on GitHub (pinned to 02eac45a1b)

Solutions

  1. Verify workspaceId and that the jobRunId exists in the Aliyun Serverless Spark console
  2. Check the wrapped cause in the stack trace for the real SDK error (auth, throttle, 404)
  3. Confirm the access key/secret are valid and have GetJobRun permissions
  4. Increase/reconfigure the retry policy to tolerate transient throttling
  5. Check worker network access to the Aliyun endpoint
Defensive patterns

Strategy: try-catch

Validate before calling

// before running
if (params.getWorkspaceId() == null || params.getWorkspaceId().isEmpty()) {
    throw new IllegalArgumentException("workspaceId is required");
}
// verify credentials via a lightweight API call or config check

Try / catch

try {
    taskClient.run();
} catch (TaskException e) {
    if (e.getCause() instanceof AliyunServerlessSparkTaskException) {
        log.error("Aliyun getJobRun failed: {}", e.getCause().getMessage(), e);
        // decide retry vs fail based on the underlying SDK error
    }
}

Prevention

When it happens

Trigger: aliyunServerlessSparkClient.getJobRun(workspaceId, jobRunId, getJobRunRequest) throws — invalid workspaceId, nonexistent/deleted jobRunId, expired or missing credentials, network partition, or Aliyun service throttling/outage.

Common situations: Wrong workspace ID in task config; the job run was cancelled/purged server-side while the task still polls it; AK/SK credentials rotated or lacking AliyunServerlessSpark permissions; intermittent connectivity from the worker to the Aliyun endpoint.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/0fb8e41d157f43ad. Report an issue: GitHub.