{"record":{"id":"9d1427d4710bdc92","repo":"apache/dolphinscheduler","slug":"emr-serverless-task-submit-failed","errorCode":null,"errorMessage":"EMR Serverless task submit failed","messagePattern":"EMR Serverless task submit failed","errorType":"exception","errorClass":"TaskException","httpStatus":null,"severity":"error","filePath":"dolphinscheduler-task-plugin/dolphinscheduler-task-emr-serverless/src/main/java/org/apache/dolphinscheduler/plugin/task/emrserverless/EmrServerlessTask.java","lineNumber":143,"sourceCode":"    }\n\n    @Override\n    public void submitApplication() throws TaskException {\n        try {\n            StartJobRunRequest request = buildStartJobRunRequest();\n\n            log.info(\"Submitting EMR Serverless job run to application: {}\",\n                    emrServerlessParameters.getApplicationId());\n            StartJobRunResult result = emrServerlessClient.startJobRun(request);\n\n            jobRunId = result.getJobRunId();\n            // Store jobRunId for failover recovery; applicationId is always available from parameters\n            setAppIds(jobRunId);\n            log.info(\"Successfully submitted EMR Serverless job run, jobRunId: {}\", jobRunId);\n\n        } catch (EmrServerlessTaskException | SdkBaseException e) {\n            log.error(\"EMR Serverless task submit failed\", e);\n            throw new TaskException(\"EMR Serverless task submit failed\", e);\n        }\n    }\n\n    @Override\n    public void trackApplicationStatus() throws TaskException {\n        try {\n            // Recover jobRunId from appIds if needed (failover case)\n            if (StringUtils.isEmpty(jobRunId) && StringUtils.isNotEmpty(getAppIds())) {\n                jobRunId = getAppIds();\n                log.info(\"Recovered EMR Serverless jobRunId from appIds: {}\", jobRunId);\n            }\n\n            if (StringUtils.isEmpty(jobRunId)) {\n                throw new EmrServerlessTaskException(\"jobRunId is empty, cannot track application status\");\n            }\n\n            String currentState = getJobRunState();\n            while (WAITING_STATES.contains(currentState)) {","sourceCodeStart":125,"sourceCodeEnd":161,"githubUrl":"https://github.com/apache/dolphinscheduler/blob/02eac45a1b6676e639fcbfb4be2243de5771b05d/dolphinscheduler-task-plugin/dolphinscheduler-task-emr-serverless/src/main/java/org/apache/dolphinscheduler/plugin/task/emrserverless/EmrServerlessTask.java#L125-L161","documentation":"Dolphinscheduler's EMR Serverless plugin wraps any failure during the StartJobRun call (request-building errors from buildStartJobRunRequest or AWS SDK SdkBaseException) into a TaskException with this message. It means the job run was not accepted by the EMR Serverless service, so no jobRunId exists to track. The original AWS exception is attached as the cause and logged.","triggerScenarios":"emrServerlessClient.startJobRun(request) throws an AWS SDK exception: invalid applicationId (application not found / not started), executionRoleArn missing permissions, malformed startJobRunRequestJson, invalid job driver config (Spark submit args), network/credential failures, or a nested EmrServerlessTaskException from placeholder resolution or JSON parsing in buildStartJobRunRequest.","commonSituations":"Application ID typo or application in wrong region; IAM role lacking emr-serverless:StartJobRun or S3 access to scripts; bad JSON payload with wrong case keys (UpperCamelCaseStrategy is required); expired AWS credentials in the worker environment; VPC/endpoint misconfiguration blocking the AWS API.","solutions":["Read the 'caused by' exception in the worker log to find the exact AWS error code","Verify the applicationId exists and is in STARTED state in the configured region","Check aws.emr.* credentials config or the DefaultAWSCredentialsProviderChain on the worker (env vars, ~/.aws, instance profile)","Validate executionRoleArn IAM policy (emr-serverless:StartJobRun plus S3 read/write on script and log locations)","Validate the startJobRunRequestJson against the AWS StartJobRun API shape (UpperCamelCase keys, e.g. jobDriver, releaseLabel)"],"exampleFix":"// before\nstartJobRunRequestJson = \"{ \\\"jobDriver\\\": { \\\"sparkSubmit\\\": { \\\"entryPoint\\\": \\\"s3://bucket/job.py\\\" } } }\" // missing releaseLabel -> ValidationException\n// after\nstartJobRunRequestJson = \"{ \\\"releaseLabel\\\": \\\"emr-6.9.0\\\", \\\"jobDriver\\\": { \\\"sparkSubmitJobDriver\\\": { \\\"sparkSubmitParameters\\\": \\\"--conf spark.executor.instances=2\\\", \\\"entryPoint\\\": \\\"s3://bucket/job.py\\\" } } }\"","handlingStrategy":"try-catch","validationCode":"// pre-flight checks before task run\nif (StringUtils.isBlank(emrServerlessParameters.getApplicationId())) throw new IllegalArgumentException(\"applicationId required\");\nif (StringUtils.isBlank(emrServerlessParameters.getStartJobRunRequestJson())) throw new IllegalArgumentException(\"startJobRunRequestJson required\");\nobjectMapper.readTree(emrServerlessParameters.getStartJobRunRequestJson()); // fail fast on invalid JSON\n// ensure AWS creds resolvable:\nnew DefaultAWSCredentialsProviderChain().getCredentials();","typeGuard":"boolean isSdkBaseException(Throwable t) {\n    return t instanceof SdkBaseException || t.getCause() instanceof SdkBaseException;\n}","tryCatchPattern":"try {\n    emrServerlessClient.startJobRun(request);\n} catch (SdkBaseException e) {\n    log.error(\"AWS error code={} requestId={}\", e instanceof AmazonServiceException\n        ? ((AmazonServiceException) e).getErrorCode() : \"client\",\n        e instanceof AmazonServiceException ? ((AmazonServiceException) e).getRequestId() : \"n/a\", e);\n    throw new TaskException(\"EMR Serverless task submit failed\", e);\n}","preventionTips":["Validate the JSON request shape locally with the same Jackson ObjectMapper before deploying","Verify the EMR Serverless application is STARTED and in the client's region before submitting","Attach an IAM policy granting emr-serverless:StartJobRun plus S3 access to scripts/logs","Keep aws.emr.* credential config consistent across API/master and worker nodes"],"tags":["aws","emr-serverless","sdk","task-submit"],"backgroundTag":"api-request-failed","analyzedSha":"02eac45a1b6676e639fcbfb4be2243de5771b05d","analyzedAt":"2026-09-06T17:43:00.555Z","contentChangedAt":"2026-09-06T17:43:00.555Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}