apache/dolphinscheduler · error · AliyunServerlessSparkTaskException

Aliyun-Serverless-Spark task parameters are not valid!

Error message

Aliyun-Serverless-Spark task parameters are not valid!

What it means

AliyunServerlessSparkTask.init parses taskParams into AliyunServerlessSparkParameters and validates them via checkParameters(); if parsing yields null or validation fails, it throws AliyunServerlessSparkTaskException indicating the task parameters are invalid. The task cannot run without a valid workspaceId/datasource/params.

Source

Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-aliyunserverlessspark/src/main/java/org/apache/dolphinscheduler/plugin/task/aliyunserverlessspark/AliyunServerlessSparkTask.java:100

    private String regionId;

    private String endpoint;

    private RetryUtils.RetryPolicy retryPolicy = new RetryUtils.RetryPolicy(10, 1000L);

    protected AliyunServerlessSparkTask(TaskExecutionContext taskExecutionContext) {
        super(taskExecutionContext);
        this.taskExecutionContext = taskExecutionContext;
    }

    @Override
    public void init() {
        final String taskParams = taskExecutionContext.getTaskParams();
        aliyunServerlessSparkParameters = JSONUtils.parseObject(taskParams, AliyunServerlessSparkParameters.class);
        log.info("aliyunServerlessSparkParameters - {}", aliyunServerlessSparkParameters);
        if (this.aliyunServerlessSparkParameters == null || !this.aliyunServerlessSparkParameters.checkParameters()) {
            throw new AliyunServerlessSparkTaskException("Aliyun-Serverless-Spark task parameters are not valid!");
        }

        ResourceParametersHelper resourceParametersHelper = taskExecutionContext.getResourceParametersHelper();
        DataSourceParameters dataSourceParameters = (DataSourceParameters) resourceParametersHelper
                .getResourceParameters(ResourceType.DATASOURCE, aliyunServerlessSparkParameters.getDatasource());
        aliyunServerlessSparkConnectionParam = (AliyunServerlessSparkConnectionParam) DataSourceUtils
                .buildConnectionParams(
                        DbType.valueOf(aliyunServerlessSparkParameters.getType()),
                        dataSourceParameters.getConnectionParams());

        accessKeyId = aliyunServerlessSparkConnectionParam.getAccessKeyId();
        accessKeySecret = aliyunServerlessSparkConnectionParam.getAccessKeySecret();
        regionId = aliyunServerlessSparkConnectionParam.getRegionId();
        endpoint = aliyunServerlessSparkConnectionParam.getEndpoint();

        try {
            aliyunServerlessSparkClient =
                    buildAliyunServerlessSparkClient(accessKeyId, accessKeySecret, regionId, endpoint);

View on GitHub (pinned to 02eac45a1b)

Solutions

  1. Open the task definition in the UI and fill all required Aliyun Serverless Spark parameters
  2. Validate taskParams JSON parses into AliyunServerlessSparkParameters with workspaceId and other required fields set
  3. Re-save/re-create the task node to regenerate a valid parameter payload

Example fix

// before
{"localParams":[],"resourceList":[]}  // missing workspaceId
// after
{"workspaceId":"w-xxxx","datasource":1,"jobName":"etl","localParams":[],"resourceList":[]}
Defensive patterns

Strategy: validation

Validate before calling

AliyunServerlessSparkParameters p = JSONUtils.parseObject(taskParams, AliyunServerlessSparkParameters.class);
if (p == null || !p.checkParameters()) {
    throw new IllegalArgumentException("Invalid Aliyun Serverless Spark task params: workspaceId and required fields must be set");
}

Type guard

boolean isValidParams(AliyunServerlessSparkParameters p) {
    return p != null && p.checkParameters();
}

Try / catch

try {
    task.init();
} catch (AliyunServerlessSparkTaskException e) {
    log.error("Parameter validation failed: {}", e.getMessage());
    task.getTaskExecutionContext().setNeedFaultTolerance(false); // fail fast, no retry
}

Prevention

When it happens

Trigger: Task definition JSON whose taskParams are empty/malformed (JSONUtils.parseObject returns null) or missing required fields checked by checkParameters() (e.g. workspaceId, clusterId, jobTemplateId, datasource).

Common situations: User saved the Aliyun Serverless Spark task node without filling required fields; manual JSON edits breaking the params; upgrading DolphinScheduler where parameter schema changed so old params fail checkParameters.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/4959357ae90d21e2. Report an issue: GitHub.