apache/dolphinscheduler · error · TaskException

run python task error

Error message

run python task error

What it means

PythonTask.handle wraps any exception raised while executing the submitted Python script via ShellCommandExecutor into a generic TaskException with message "run python task error". The original exception is attached as the cause; the task's exit status is also set to failure before throwing. It signals that the Python task execution failed for any reason (command failure, parameter handling, IO).

Solutions

1. Inspect the logged 'python task failure' stack trace and the task instance log for the real root cause (python exit code, missing binary, script error).
2. Verify python is installed and on PATH on the worker host, and that PYTHON_HOME/env is configured if required.
3. Run the Python script manually on the worker with the same user to reproduce syntax/import/runtime errors.
4. Check taskParams rawScript and output parameter (varPool) definitions for invalid references or malformed JSON.

Example fix

// before: opaque failure
throw new TaskException("run python task error", e);
// after: surface root cause in message
throw new TaskException("run python task error: " + e.getMessage(), e);

// before submitting
ProcessBuilder check = new ProcessBuilder("python", "--version");
check.start().waitFor(); // throws IOException if python is missing
assert rawScript != null && !rawScript.isBlank();

boolean hasScript(PythonParameters p) { return p != null && p.getRawScript() != null && !p.getRawScript().isBlank(); }

try {
    pythonTask.handle();
} catch (TaskException e) {
    logger.error("Python task failed: {}", e.getCause() != null ? e.getCause().getMessage() : e.getMessage(), e);
    // inspect worker logs / python exit code before retrying
}

When it happens

Trigger: Calling PythonTask.handle() when: the python executable is missing or fails (non-zero exit), the shell command executor throws an IOException/InterruptedException, or dealOutParam/setVarPool throws while processing task output parameters.

Common situations: python not installed or not on PATH on the worker; script syntax/runtime errors; bad varPool/output-parameter references in the task definition; worker interrupted during execution.

Related errors

• cancel application error
• cancel application error
• cancel application error
• cancel linkis task error
• Cannot create TaskInstance WorkingDirectory:

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/27b8c8d6218bbba9. Report an issue: GitHub.

Appendix: source

Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-python/src/main/java/org/apache/dolphinscheduler/plugin/task/python/PythonTask.java:97

            // generate the file path of this python script
            String pythonScriptFile = buildPythonCommandFilePath();

            // create this file
            createPythonCommandFileIfNotExists(pythonScriptContent, pythonScriptFile);

            IShellInterceptorBuilder<?, ?> shellActuatorBuilder = ShellInterceptorBuilderFactory.newBuilder()
                    .appendScript(buildPythonExecuteCommand(pythonScriptFile));

            TaskResponse taskResponse = shellCommandExecutor.run(shellActuatorBuilder, taskCallBack);
            setExitStatusCode(taskResponse.getExitStatusCode());
            setProcessId(taskResponse.getProcessId());
            setTaskOutputParams(shellCommandExecutor.getTaskOutputParams());
            pythonParameters.dealOutParam(shellCommandExecutor.getTaskOutputParams());
            taskRequest.setVarPool(pythonParameters.getVarPool());
        } catch (Exception e) {
            log.error("python task failure", e);
            setExitStatusCode(TaskConstants.EXIT_CODE_FAILURE);
            throw new TaskException("run python task error", e);
        }
    }

    @Override
    public void cancel() throws TaskException {
        // cancel process
        try {
            shellCommandExecutor.cancelApplication();
        } catch (Exception e) {
            throw new TaskException("cancel application error", e);
        }
    }

    @Override
    public AbstractParameters getParameters() {
        return pythonParameters;
    }

View on GitHub (pinned to 02eac45a1b)