What causes "Failed to build Aliyun-Serverless-Spark client!" in apache/dolphinscheduler?

Verify the stored accessKeyId/accessSecret (user token) and that they are valid for the account

apache/dolphinscheduler · error · AliyunServerlessSparkTaskException

Failed to build Aliyun-Serverless-Spark client!

Error message

Failed to build Aliyun-Serverless-Spark client!

What it means

During init, AliyunServerlessSparkTask builds the Aliyun (EMR Serverless Spark) SDK client from accessKeyId/accessSecret/regionId/endpoint. Any exception while constructing that client is wrapped and rethrown as AliyunServerlessSparkTaskException "Failed to build Aliyun-Serverless-Spark client!".

Source

Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-aliyunserverlessspark/src/main/java/org/apache/dolphinscheduler/plugin/task/aliyunserverlessspark/AliyunServerlessSparkTask.java:121

        ResourceParametersHelper resourceParametersHelper = taskExecutionContext.getResourceParametersHelper();
        DataSourceParameters dataSourceParameters = (DataSourceParameters) resourceParametersHelper
                .getResourceParameters(ResourceType.DATASOURCE, aliyunServerlessSparkParameters.getDatasource());
        aliyunServerlessSparkConnectionParam = (AliyunServerlessSparkConnectionParam) DataSourceUtils
                .buildConnectionParams(
                        DbType.valueOf(aliyunServerlessSparkParameters.getType()),
                        dataSourceParameters.getConnectionParams());

        accessKeyId = aliyunServerlessSparkConnectionParam.getAccessKeyId();
        accessKeySecret = aliyunServerlessSparkConnectionParam.getAccessKeySecret();
        regionId = aliyunServerlessSparkConnectionParam.getRegionId();
        endpoint = aliyunServerlessSparkConnectionParam.getEndpoint();

        try {
            aliyunServerlessSparkClient =
                    buildAliyunServerlessSparkClient(accessKeyId, accessKeySecret, regionId, endpoint);
        } catch (Exception e) {
            log.error("Failed to build Aliyun-Serverless-Spark client!", e);
            throw new AliyunServerlessSparkTaskException("Failed to build Aliyun-Serverless-Spark client!");
        }

        currentState = RunState.Submitted;
    }

    @Override
    public void handle(TaskCallBack taskCallBack) throws TaskException {
        GetTemplateResponse getTemplateResponse = RetryUtils.retryFunction(() -> {
            try {
                return aliyunServerlessSparkClient.getTemplate(
                        aliyunServerlessSparkParameters.getWorkspaceId(),
                        buildGetTemplateRequest());
            } catch (Exception e) {
                throw new TaskException("Failed to get template info", e);
            }
        }, retryPolicy);

        if (getTemplateResponse != null) {

View on GitHub (pinned to 02eac45a1b)

Solutions

Verify the stored accessKeyId/accessSecret (user token) and that they are valid for the account
Check regionId and endpoint values match each other and the target workspace region
Ensure network/DNS/proxy reachability of the endpoint from the worker host
Confirm the Aliyun SDK dependency is present and versions are compatible

Example fix

// before
endpoint="https://emr-serverless.spark.cn-beijing.invalid.aliyuncs.com"
// after
endpoint="https://emr-serverless-spark.cn-hangzhou.aliyuncs.com"  // correct region endpoint

Defensive patterns

Strategy: try-catch

Validate before calling

if (accessKeyId == null || accessKeyId.isEmpty() || accessKeySecret == null || accessKeySecret.isEmpty()) {
    throw new IllegalArgumentException("Aliyun credentials must be configured before init");
}
// also sanity-check endpoint/region formats

Try / catch

try {
    task.init();
} catch (AliyunServerlessSparkTaskException e) {
    if (e.getMessage().contains("Failed to build")) {
        log.error("Client construction failed; check credentials/endpoint/SDK deps", e);
    }
}

Prevention

Store valid Aliyun credentials in the token/user management, not ad-hoc
Verify endpoint matches the workspace region
Pin compatible Aliyun SDK versions on worker classpath
Test endpoint connectivity from worker hosts (curl/telnet) before rollout

When it happens

Trigger: buildAliyunServerlessSparkClient throwing due to invalid/missing accessKeyId or accessKeySecret, bad regionId or endpoint format, missing SDK classes, or network failure resolving the endpoint — inside init().

Common situations: Wrong or absent Aliyun credentials (tenant-level or stored token); endpoint/region mismatch (e.g. endpoint for the wrong region); corporate proxy/firewall blocking the endpoint; Aliyun Java SDK dependency conflicts.

Related errors

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/463011a8d33f8f99. Report an issue: GitHub.