apache/dolphinscheduler · error · IllegalArgumentException

spark datasource param is not valid

Error message

spark datasource param is not valid

What it means

AliyunServerlessSparkDataSourceProcessor.checkDatasourceParam validates datasource form parameters. It requires both regionId and accessKeyId to be non-empty; if either is missing it throws IllegalArgumentException with the generic message 'spark datasource param is not valid'.

Source

Thrown at dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-aliyunserverlessspark/src/main/java/org/apache/dolphinscheduler/plugin/datasource/aliyunserverlessspark/param/AliyunServerlessSparkDataSourceProcessor.java:53

import com.google.auto.service.AutoService;

@AutoService(DataSourceProcessor.class)
@Slf4j
public class AliyunServerlessSparkDataSourceProcessor extends AbstractDataSourceProcessor {

    @Override
    public BaseDataSourceParamDTO castDatasourceParamDTO(String paramJson) {
        return JSONUtils.parseObject(paramJson, AliyunServerlessSparkDataSourceParamDTO.class);
    }

    @Override
    public void checkDatasourceParam(BaseDataSourceParamDTO datasourceParamDTO) {
        AliyunServerlessSparkDataSourceParamDTO aliyunServerlessSparkDataSourceParamDTO =
                (AliyunServerlessSparkDataSourceParamDTO) datasourceParamDTO;
        if (StringUtils.isEmpty(aliyunServerlessSparkDataSourceParamDTO.getRegionId()) ||
                StringUtils.isEmpty(aliyunServerlessSparkDataSourceParamDTO.getAccessKeyId())) {
            throw new IllegalArgumentException("spark datasource param is not valid");
        }
    }

    @Override
    public String getDatasourceUniqueId(ConnectionParam connectionParam, DbType dbType) {
        AliyunServerlessSparkConnectionParam baseConnectionParam =
                (AliyunServerlessSparkConnectionParam) connectionParam;
        return MessageFormat.format(
                "{0}@{1}@{2}@{3}",
                dbType.getName(),
                baseConnectionParam.getRegionId(),
                PasswordUtils.encodePassword(baseConnectionParam.getAccessKeyId()),
                PasswordUtils.encodePassword(baseConnectionParam.getAccessKeySecret()));
    }

    @Override
    public BaseDataSourceParamDTO createDatasourceParamDTO(String connectionJson) {
        AliyunServerlessSparkConnectionParam connectionParams =

View on GitHub (pinned to 02eac45a1b)

Solutions

1. Set a valid Aliyun regionId (e.g. cn-hangzhou) in the datasource parameters.
2. Provide a non-empty accessKeyId (and secret key) for the Aliyun account.
3. Re-save the datasource via the UI/API and confirm the JSON param contains both fields before testing the connection.

Example fix

// before
{
  "regionId": "",
  "accessKeyId": "",
  "accessKeySecret": "***"
}
// after
{
  "regionId": "cn-hangzhou",
  "accessKeyId": "LTAI5t...",
  "accessKeySecret": "***"
}

AliyunServerlessSparkDataSourceParamDTO p = (AliyunServerlessSparkDataSourceParamDTO) dto;
if (StringUtils.isEmpty(p.getRegionId()) || StringUtils.isEmpty(p.getAccessKeyId())) {
    throw new IllegalArgumentException("regionId and accessKeyId are required for Aliyun Serverless Spark");
}

boolean isValidSparkParam(AliyunServerlessSparkDataSourceParamDTO p) {
    return p != null && StringUtils.isNotEmpty(p.getRegionId()) && StringUtils.isNotEmpty(p.getAccessKeyId());
}

try {
    dataSourceService.createDataSource(dto);
} catch (IllegalArgumentException e) {
    log.error("invalid spark datasource params: {}", e.getMessage());
}

When it happens

Trigger: Creating or updating an Aliyun Serverless Spark datasource connection with an empty regionId or accessKeyId (or with the JSON key absent from the datasource param).

Common situations: Leaving the region field blank in the datasource form; using an access key that was not copied/pasted correctly; migration/import of datasource configs where these fields were dropped.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• 1400004
• datasource host illegal
• database name illegal
• datasource other params: + entry.getKey() + illegal
• address is null.

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/9bda053916b52772. Report an issue: GitHub.