apache/dolphinscheduler · error · EmrServerlessTaskException

EMR Serverless task params are not valid

Error message

EMR Serverless task params are not valid

What it means

EmrServerlessTask.init parses taskExecutionContext.getTaskParams() into EmrServerlessParameters and validates them. If the JSON is unparseable (null result) or EmrServerlessParameters.checkParameters() fails (required fields missing, e.g. jobRunId vs applicationId for the chosen operation), it throws EmrServerlessTaskException 'EMR Serverless task params are not valid'.

Source

Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-emr-serverless/src/main/java/org/apache/dolphinscheduler/plugin/task/emrserverless/EmrServerlessTask.java:121

    /**
     * jobRunId returned by StartJobRun or recovered from appIds
     */
    private String jobRunId;

    protected EmrServerlessTask(TaskExecutionContext taskExecutionContext) {
        super(taskExecutionContext);
        this.taskExecutionContext = taskExecutionContext;
    }

    @Override
    public void init() {
        final String taskParams = taskExecutionContext.getTaskParams();
        emrServerlessParameters = JSONUtils.parseObject(taskParams, EmrServerlessParameters.class);
        log.info("Initialize EMR Serverless task params: {}", JSONUtils.toPrettyJsonString(taskParams));

        if (emrServerlessParameters == null || !emrServerlessParameters.checkParameters()) {
            throw new EmrServerlessTaskException("EMR Serverless task params are not valid");
        }

        emrServerlessClient = createEmrServerlessClient();
    }

    @Override
    public void submitApplication() throws TaskException {
        try {
            StartJobRunRequest request = buildStartJobRunRequest();

            log.info("Submitting EMR Serverless job run to application: {}",
                    emrServerlessParameters.getApplicationId());
            StartJobRunResult result = emrServerlessClient.startJobRun(request);

            jobRunId = result.getJobRunId();
            // Store jobRunId for failover recovery; applicationId is always available from parameters
            setAppIds(jobRunId);
            log.info("Successfully submitted EMR Serverless job run, jobRunId: {}", jobRunId);

View on GitHub (pinned to 02eac45a1b)

Solutions

1. Open the EMR Serverless task node and complete all required fields per the selected operation type (applicationId, and jobRun fields for job runs).
2. Read EmrServerlessParameters.checkParameters() to see the exact required-field rules and satisfy them.
3. Validate the stored taskParams JSON parses and matches current EmrServerlessParameters schema (camelCase field names).
4. If upgrading DolphinScheduler, re-save old EMR Serverless nodes so parameter JSON matches the new schema.

Example fix

// before: job run missing jobRunId
{"emrServerlessType":"JOB_RUN","applicationId":"00f...","localParams":[]}
// after
{"emrServerlessType":"JOB_RUN","applicationId":"00f...","jobRunId":"jr-...","releaseLabel":"emr-6.9.0","jobDriver":{...},"executionRoleArn":"arn:aws:iam::...:role/EMRServerlessRole","localParams":[]}

EmrServerlessParameters p = JSONUtils.parseObject(taskParams, EmrServerlessParameters.class);
if (p == null || !p.checkParameters()) {
    throw new IllegalArgumentException("EMR Serverless params invalid: required fields missing for operation " + p.getEmrServerlessType());
}

boolean emrServerlessParamsValid(String json) {
    EmrServerlessParameters p = JSONUtils.parseObject(json, EmrServerlessParameters.class);
    return p != null && p.checkParameters();
}

try {
    emrServerlessTask.init();
} catch (EmrServerlessTaskException e) {
    // prompt user to complete the EMR Serverless node form fields
}

When it happens

Trigger: init() runs for every EMR Serverless task start; throws when taskParams JSON is empty/malformed or checkParameters() finds missing required fields such as emrServerlessType, applicationId, or (for job runs) jobRunId/releaseLabel/executionRoleArn/jobDriver.

Common situations: EMR Serverless node saved with incomplete form; switching operation type (start application vs run job) without filling the newly required fields; older workflow JSON incompatible after plugin parameter changes; typo in JSON field casing.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

• dvc task params is not valid
• emr task params is not valid
• shell task params is not valid
• EMR Serverless task submit failed
• jobRunId is empty, cannot track application status

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/38cbe729958ec458. Report an issue: GitHub.