What causes "Execute dvc task failed" in apache/dolphinscheduler?

Inspect the wrapped cause in the worker log — it identifies the concrete failure (missing binary, git auth, IO).

apache/dolphinscheduler · error · TaskException

Execute dvc task failed

Error message

Execute dvc task failed

What it means

Catch-all in DvcTask.handle: any non-interrupt exception while building/running the shell command (script generation, shell interceptor setup, IO problems, non-managed subprocess failures) is logged, sets exit code failure, and is rethrown as TaskException 'Execute dvc task failed' with the original cause.

Source

Thrown at dolphinscheduler-task-plugin/dolphinscheduler-task-dvc/src/main/java/org/apache/dolphinscheduler/plugin/task/dvc/DvcTask.java:79

    @Override
    public void handle(TaskCallBack taskCallBack) throws TaskException {
        try {
            // construct process
            IShellInterceptorBuilder<?, ?> shellActuatorBuilder = ShellInterceptorBuilderFactory.newBuilder()
                    .appendScript(buildCommand());
            TaskResponse commandExecuteResult = shellCommandExecutor.run(shellActuatorBuilder, taskCallBack);
            setExitStatusCode(commandExecuteResult.getExitStatusCode());
            setProcessId(commandExecuteResult.getProcessId());
            parameters.dealOutParam(shellCommandExecutor.getTaskOutputParams());
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            log.error("The current DvcTask has been interrupted", e);
            setExitStatusCode(EXIT_CODE_FAILURE);
            throw new TaskException("The current DvcTask has been interrupted", e);
        } catch (Exception e) {
            log.error("dvc task error", e);
            setExitStatusCode(EXIT_CODE_FAILURE);
            throw new TaskException("Execute dvc task failed", e);
        }
    }

    @Override
    public void cancel() throws TaskException {
        // cancel process
        try {
            shellCommandExecutor.cancelApplication();
        } catch (Exception e) {
            throw new TaskException("cancel application error", e);
        }
    }

    public String buildCommand() {
        String command = "";
        String taskType = parameters.getDvcTaskType();
        if (taskType.equals(DvcConstants.DVC_TASK_TYPE.UPLOAD)) {
            command = buildUploadCommond();

View on GitHub (pinned to 02eac45a1b)

Solutions

Inspect the wrapped cause in the worker log — it identifies the concrete failure (missing binary, git auth, IO).
Verify dvc and git are installed and on PATH on the worker, and git credentials are configured for the private repository.
Verify remote storage credentials (e.g. AWS creds for S3 store URL) on the worker environment.
Re-run with dvc CLI manually on the worker host to reproduce outside the scheduler.

Example fix

// before: worker missing dvc -> command fails
// worker host
// after
pip install dvc[s3] && git --version && dvc version

Defensive patterns

Strategy: try-catch

Validate before calling

// precheck on the worker host before running DVC tasks
boolean ok = isOnPath("dvc") && isOnPath("git")
          && hasGitCredentials(params.getDvcRepository())
          && hasStoreCredentials(params.getDvcStoreUrl());

Try / catch

try {
    dvcTask.handle(null);
} catch (TaskException e) {
    log.error("DVC task failed; cause=", e.getCause()); // root cause identifies missing binary/cred/IO
}

Prevention

Install dvc and git on every worker image.
Pre-configure git credentials and remote store credentials on workers.
Ensure adequate disk space in the worker working directory.
Validate parameters so buildCommand never assembles an empty/partial command.

When it happens

Trigger: handle() called and any Exception other than InterruptedException escapes: ShellInterceptorBuilderFactory failure, IO errors writing the temp script, command not found / non-zero dvc exit surfaced by the executor, or parameters.dealOutParam NPEs.

Common situations: dvc/git not installed on the worker host; worker lacks git credentials for the DVC repository; remote store (S3 etc.) credentials missing; disk full when writing output; malformed command built from empty parameters.

Related errors

AI-assisted analysis of apache/dolphinscheduler@02eac45a1b (2026-09-06). Data as JSON: /api/errors/a7f0d3b358de258f. Report an issue: GitHub.