{"record":{"id":"deac5ea8ac08be83","repo":"apache/seatunnel","slug":"run-seatunnel-on-spark-failed","errorCode":null,"errorMessage":"Run SeaTunnel on spark failed","messagePattern":"Run SeaTunnel on spark failed","errorType":"exception","errorClass":"CommandExecuteException","httpStatus":null,"severity":"error","filePath":"seatunnel-core/seatunnel-spark-starter/seatunnel-spark-starter-common/src/main/java/org/apache/seatunnel/core/starter/spark/command/SparkTaskExecuteCommand.java","lineNumber":71,"sourceCode":"\n    @Override\n    public void execute() throws CommandExecuteException {\n        Path configFile = resolveConfigPath(sparkCommandArgs, SparkFiles::get);\n        checkConfigExist(configFile);\n        Config config =\n                MetalakeConfigUtils.getMetalakeConfig(\n                        ConfigBuilder.of(configFile, sparkCommandArgs.getVariables()));\n        if (!sparkCommandArgs.getJobName().equals(Constants.LOGO)) {\n            config =\n                    config.withValue(\n                            ConfigUtil.joinPath(\"env\", \"job.name\"),\n                            ConfigValueFactory.fromAnyRef(sparkCommandArgs.getJobName()));\n        }\n        try {\n            SparkExecution seaTunnelTaskExecution = new SparkExecution(config);\n            seaTunnelTaskExecution.execute();\n        } catch (Exception e) {\n            throw new CommandExecuteException(\"Run SeaTunnel on spark failed\", e);\n        }\n    }\n\n    static Path resolveConfigPath(\n            SparkCommandArgs sparkCommandArgs, Function<String, String> sparkFilesResolver) {\n        Path configFile = FileUtils.getConfigPath(sparkCommandArgs);\n        if (!DeployMode.CLUSTER.equals(sparkCommandArgs.getDeployMode())) {\n            log.info(\"Resolved SeaTunnel config file: {}\", configFile.toAbsolutePath());\n            return configFile;\n        }\n\n        Path fileName = configFile.getFileName();\n        if (fileName == null) {\n            log.info(\"Resolved SeaTunnel config file: {}\", configFile.toAbsolutePath());\n            return configFile;\n        }\n\n        String sparkFilePath;","sourceCodeStart":53,"sourceCodeEnd":89,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-core/seatunnel-spark-starter/seatunnel-spark-starter-common/src/main/java/org/apache/seatunnel/core/starter/spark/command/SparkTaskExecuteCommand.java#L53-L89","documentation":"SparkTaskExecuteCommand.execute wraps the whole Spark job execution (building SparkExecution and calling execute()) in a try/catch and rethrows any failure as CommandExecuteException with the generic message 'Run SeaTunnel on spark failed'. The original exception is attached as the cause, so the real error (bad config, connector failure, Spark startup problem) is in the stack trace below this message.","triggerScenarios":"Any Exception escaping new SparkExecution(config).execute(): invalid config file, missing connector jar, Spark session creation failure, job-level runtime errors in source/transform/sink. It is a wrapper, not a root cause.","commonSituations":"Config file references a connector not installed in $SEATUNNEL_HOME/connectors; Spark cannot initialize (wrong master/URL); a source table schema mismatch; checkpoint/savepoint restore failures.","solutions":["Read the 'Caused by' chain of the CommandExecuteException to find the root cause","Verify the config file syntax and that all referenced plugins are installed (sh bin/install-plugin.sh)","Run the job with a minimal batch config template to isolate environment vs config problems","Check Spark master/cluster logs for submit-side failures"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Verify config and plugins before submitting:\n// sh bin/seatunnel.sh --config job.conf -e local --dry-run static","typeGuard":null,"tryCatchPattern":"try {\n    submitJob();\n} catch (CommandExecuteException e) {\n    log.error(\"Spark job failed\", e.getCause()); // root cause is in getCause()\n}","preventionTips":["Always inspect the 'Caused by' chain, the message itself is generic","Install all referenced connector plugins before running (install-plugin.sh)","Validate config with --dry-run static before a real submission"],"tags":["spark","command-execution","job-failure"],"backgroundTag":"job-execution-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}