{"record":{"id":"a45ed67ef955dc59","repo":"hashicorp/nomad","slug":"executor-error-waiting-on-process-v","errorCode":null,"errorMessage":"executor: error waiting on process: %v","messagePattern":"executor: error waiting on process: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"drivers/exec/driver.go","lineNumber":595,"sourceCode":"func (d *Driver) WaitTask(ctx context.Context, taskID string) (<-chan *drivers.ExitResult, error) {\n\thandle, ok := d.tasks.Get(taskID)\n\tif !ok {\n\t\treturn nil, drivers.ErrTaskNotFound\n\t}\n\n\tch := make(chan *drivers.ExitResult)\n\tgo d.handleWait(ctx, handle, ch)\n\n\treturn ch, nil\n}\n\nfunc (d *Driver) handleWait(ctx context.Context, handle *taskHandle, ch chan *drivers.ExitResult) {\n\tdefer close(ch)\n\tvar result *drivers.ExitResult\n\tps, err := handle.exec.Wait(ctx)\n\tif err != nil {\n\t\tresult = &drivers.ExitResult{\n\t\t\tErr: fmt.Errorf(\"executor: error waiting on process: %v\", err),\n\t\t}\n\t\t// if process state is nil, we've probably been killed, so return a reasonable\n\t\t// exit state to the handlers\n\t\tif ps == nil {\n\t\t\tresult.ExitCode = -1\n\t\t\tresult.OOMKilled = false\n\t\t}\n\t} else {\n\t\tresult = &drivers.ExitResult{\n\t\t\tExitCode:  ps.ExitCode,\n\t\t\tSignal:    ps.Signal,\n\t\t\tOOMKilled: ps.OOMKilled,\n\t\t}\n\t}\n\n\tselect {\n\tcase <-ctx.Done():\n\t\treturn","sourceCodeStart":577,"sourceCodeEnd":613,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/drivers/exec/driver.go#L577-L613","documentation":"This error is produced by the exec driver's handleWait goroutine when the executor's call to wait on the task process fails. Instead of returning a normal exit code, the driver wraps the wait error in an ExitResult so the task's exit is reported as failed. It typically means the executor could not observe the process termination (e.g. the process was killed abruptly or the executor plugin died).","triggerScenarios":"handle.Wait(ctx) on the executor returns a non-nil error; ps (process state) is nil, in which case ExitCode is set to -1 and OOMKilled to false to give handlers a reasonable exit state.","commonSituations":"The task process was SIGKILLed from outside Nomad; the executor subprocess crashed or its plugin client exited; OOM killer or cgroup teardown interrupted the wait; Nomad client restarted mid-task.","solutions":["Inspect the agent/executor logs around the failure to find the underlying wait error (%v) for the real cause.","Check for external SIGKILLs (OOM killer via dmesg/journalctl -k, manual kills, deployment scripts) targeting the task process.","Verify the executor plugin process is healthy and the client machine is not killing processes (cgroup, systemd-oomd, container runtime interference).","Re-run the task; if reproducible, reduce memory limits or fix the application crash that prevents a clean exit."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"result, err := taskHandle.Wait(ctx)\nif err != nil || result.Err != nil || result.ExitCode == -1 {\n    // treat as abnormal termination: alert, inspect executor logs, reschedule\n    log.Printf(\"abnormal task exit: %v\", firstNonNil(err, result.Err))\n}","preventionTips":["Monitor dmesg/systemd-oomd for OOM kills of task processes.","Avoid sending SIGKILL to Nomad-managed processes from deployment tooling.","Alert on tasks exiting with code -1 and OOMKilled=false."],"tags":["exec-driver","process-wait","nomad"],"backgroundTag":"process-wait-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}