{"record":{"id":"785bd943f4a2da5b","repo":"wavetermdev/waveterm","slug":"failed-to-start-remote-job-w","errorCode":null,"errorMessage":"failed to start remote job: %w","messagePattern":"failed to start remote job: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/jobcontroller/jobcontroller.go","lineNumber":737,"sourceCode":"\tif err != nil {\n\t\tlog.Printf(\"[job:%s] RemoteStartJobCommand failed: %v\", jobId, err)\n\t\terrMsg := fmt.Sprintf(\"failed to start job: %v\", err)\n\t\tvar updatedJob *waveobj.Job\n\t\twstore.DBUpdateFn(ctx, jobId, func(job *waveobj.Job) {\n\t\t\tjob.JobManagerStatus = JobManagerStatus_Done\n\t\t\tjob.JobManagerDoneReason = JobDoneReason_StartupError\n\t\t\tjob.JobManagerStartupError = errMsg\n\t\t\tupdatedJob = job\n\t\t})\n\t\tsendBlockJobStatusEventByJob(ctx, updatedJob)\n\t\ttelemetry.GoRecordTEventWrap(&telemetrydata.TEvent{\n\t\t\tEvent: \"job:done\",\n\t\t\tProps: telemetrydata.TEventProps{\n\t\t\t\tJobDoneReason: JobDoneReason_StartupError,\n\t\t\t\tJobKind:       params.JobKind,\n\t\t\t},\n\t\t})\n\t\treturn \"\", fmt.Errorf(\"failed to start remote job: %w\", err)\n\t}\n\n\tlog.Printf(\"[job:%s] RemoteStartJobCommand succeeded, cmdpid=%d cmdstartts=%d jobmanagerpid=%d jobmanagerstartts=%d\", jobId, rtnData.CmdPid, rtnData.CmdStartTs, rtnData.JobManagerPid, rtnData.JobManagerStartTs)\n\tvar updatedJob *waveobj.Job\n\terr = wstore.DBUpdateFn(ctx, jobId, func(job *waveobj.Job) {\n\t\tjob.CmdPid = rtnData.CmdPid\n\t\tjob.CmdStartTs = rtnData.CmdStartTs\n\t\tjob.JobManagerPid = rtnData.JobManagerPid\n\t\tjob.JobManagerStartTs = rtnData.JobManagerStartTs\n\t\tjob.JobManagerStatus = JobManagerStatus_Running\n\t\tupdatedJob = job\n\t})\n\tif err != nil {\n\t\tlog.Printf(\"[job:%s] warning: failed to update job status to running: %v\", jobId, err)\n\t} else {\n\t\tlog.Printf(\"[job:%s] job status updated to running\", jobId)\n\t\tsendBlockJobStatusEventByJob(ctx, updatedJob)\n\t}","sourceCodeStart":719,"sourceCodeEnd":755,"githubUrl":"https://github.com/wavetermdev/waveterm/blob/a4447c1563b2df285ab89e76c82f91e1a1a49c1e/pkg/jobcontroller/jobcontroller.go#L719-L755","documentation":"StartJob finishes by issuing RemoteStartJobCommand over the RPC connection to actually launch the job/manager process on the remote host; any failure there is wrapped with this message (after telemetry records JobDoneReason_StartupError). Unlike earlier checks, this fires when the remote side rejects or fails the start — bad command, missing shell, remote error, timeout, or connection drop mid-call.","triggerScenarios":"RemoteStartJobCommand returns an error: the command binary does not exist on the remote host, the remote shell/user lacks permission, the RPC connection drops during the call, or the remote job manager fails to spawn.","commonSituations":"Cmd points to a program not installed on the remote machine, remote account shell restrictions, SSH connection flaky under VPN, remote host out of resources, command exits immediately with an error before the job manager registers.","solutions":["Inspect the wrapped cause (%w) — it carries the remote-side error detail.","Verify the command exists on the remote host (which <cmd>) and the remote user has execute permission.","Confirm the connection is still alive and retry; re-establish SSH if it dropped.","Test the command manually over SSH on the target host to reproduce the failure."],"exampleFix":"// before\np.Cmd = \"zsh\" // not installed on remote\n// after\np.Cmd = \"bash\" // verified present on the remote host","handlingStrategy":"try-catch","validationCode":"if connected, _ := conncontroller.IsConnected(params.ConnName); !connected {\n    return fmt.Errorf(\"cannot start remote job: %s not connected\", params.ConnName)\n}","typeGuard":null,"tryCatchPattern":"jobId, err := jobcontroller.StartJob(ctx, params)\nif err != nil && strings.Contains(err.Error(), \"failed to start remote job\") {\n    // remote side rejected the start: log wrapped cause, verify command exists on host, retry after reconnect\n    log.Printf(\"remote job start failed: %v\", err)\n}","preventionTips":["Verify the command exists and is executable on the target remote host before launching jobs.","Keep SSH connections healthy (keepalives) so RPC calls don't die mid-start.","Test new job commands interactively over SSH before automating them.","Watch remote host resources (memory, process limits) that can prevent job manager spawn."],"tags":["remote-execution","rpc","jobcontroller"],"backgroundTag":"remote-command-start-failed","analyzedSha":"a4447c1563b2df285ab89e76c82f91e1a1a49c1e","analyzedAt":"2026-09-01T15:26:23.972Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}