{"record":{"id":"ae80bee3b3c44580","repo":"apache/dolphinscheduler","slug":"execute-jupyter-task-failed","errorCode":null,"errorMessage":"Execute jupyter task failed","messagePattern":"Execute jupyter task failed","errorType":"exception","errorClass":"TaskException","httpStatus":null,"severity":"error","filePath":"dolphinscheduler-task-plugin/dolphinscheduler-task-jupyter/src/main/java/org/apache/dolphinscheduler/plugin/task/jupyter/JupyterTask.java","lineNumber":101,"sourceCode":"    public void handle(TaskCallBack taskCallBack) throws TaskException {\n        try {\n            IShellInterceptorBuilder<?, ?> shellActuatorBuilder = ShellInterceptorBuilderFactory.newBuilder()\n                    .properties(ParameterUtils.convert(taskRequest.getPrepareParamsMap()))\n                    .appendScript(buildCommand());\n\n            TaskResponse response = shellCommandExecutor.run(shellActuatorBuilder, taskCallBack);\n            setExitStatusCode(response.getExitStatusCode());\n            setAppIds(String.join(TaskConstants.COMMA, getApplicationIds()));\n            setProcessId(response.getProcessId());\n        } catch (InterruptedException e) {\n            Thread.currentThread().interrupt();\n            log.error(\"The current Jupyter task has been interrupted\", e);\n            setExitStatusCode(TaskConstants.EXIT_CODE_FAILURE);\n            throw new TaskException(\"The current Jupyter task has been interrupted\", e);\n        } catch (Exception e) {\n            log.error(\"jupyter task execution failure\", e);\n            exitStatusCode = -1;\n            throw new TaskException(\"Execute jupyter task failed\", e);\n        }\n    }\n\n    @Override\n    public void submitApplication() throws TaskException {\n\n    }\n\n    @Override\n    public void trackApplicationStatus() throws TaskException {\n\n    }\n\n    /**\n     * command will be like: papermill [OPTIONS] NOTEBOOK_PATH [OUTPUT_PATH]\n     */\n    protected String buildCommand() throws IOException {\n","sourceCodeStart":83,"sourceCodeEnd":119,"githubUrl":"https://github.com/apache/dolphinscheduler/blob/02eac45a1b6676e639fcbfb4be2243de5771b05d/dolphinscheduler-task-plugin/dolphinscheduler-task-jupyter/src/main/java/org/apache/dolphinscheduler/plugin/task/jupyter/JupyterTask.java#L83-L119","documentation":"JupyterTask.handle() wraps any non-interruption failure of the jupyter command execution into TaskException 'Execute jupyter task failed'. It is the generic catch-all when the jupyter/conda/pip shell process fails to run or exits with an error; exitStatusCode is set to -1.","triggerScenarios":"shellCommandExecutor.run(jupyterTaskCommand) throws any Exception other than InterruptedException — command construction failure, jupyter/conda binary not found, process spawn error, or the TaskException thrown by handle's own init/prepare path propagating here.","commonSituations":"jupyter or conda not installed/on PATH on the worker, invalid notebook path at runtime, conda env activation failure, network issue downloading pip requirements, or malformed generated command line.","solutions":["Read the wrapped stack trace and the shell command output in worker logs for the root cause.","Verify jupyter/conda/pip are installed and on PATH for the worker user.","Run the logged jupyter command manually on the worker to reproduce the failure.","Check the notebook path and conda env/requirements referenced in task params exist.","If using packed/remote envs, verify the env directory and permissions."],"exampleFix":"// before\n} catch (Exception e) {\n    throw new TaskException(\"Execute jupyter task failed\", e);\n}\n// after\n} catch (Exception e) {\n    log.error(\"jupyter task failed, command: {}\", String.join(\" \", getJupyterTaskCommand()), e);\n    throw new TaskException(\"Execute jupyter task failed: \" + e.getMessage(), e);\n}","handlingStrategy":"try-catch","validationCode":"// preflight: confirm binaries exist where the task will run\nProcess p = Runtime.getRuntime().exec(new String[]{\"sh\",\"-c\",\"command -v jupyter && command -v conda\"});\nif (p.waitFor() != 0) throw new IllegalStateException(\"jupyter/conda not found on worker PATH\");","typeGuard":"static boolean isJupyterExecutionFailure(TaskException e) {\n    return e.getMessage() != null && e.getMessage().startsWith(\"Execute jupyter task failed\");\n}","tryCatchPattern":"try {\n    jupyterTask.handle(callBack);\n} catch (TaskException e) {\n    log.error(\"jupyter execution failed; root cause:\", e.getCause());\n    // retry only on transient env issues, fail fast on config issues\n    if (isTransient(e.getCause())) retry();\n}","preventionTips":["Install jupyter/conda on worker images and verify on PATH for the task user.","Validate notebook paths and env names before submitting the workflow.","Run the generated jupyter command manually to reproduce env issues.","Keep conda environments pre-built to avoid network-dependent pip installs at task time."],"tags":["jupyter","task-execution","shell","dolphinscheduler"],"backgroundTag":"task-execution-failed","analyzedSha":"02eac45a1b6676e639fcbfb4be2243de5771b05d","analyzedAt":"2026-09-06T17:43:00.555Z","contentChangedAt":"2026-09-06T17:43:00.555Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}