{"record":{"id":"70ff7532be768a79","repo":"hashicorp/nomad","slug":"executor-shutdown-error-v","errorCode":null,"errorMessage":"executor shutdown error: %v","messagePattern":"executor shutdown error: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"drivers/shared/executor/executor_unix.go","lineNumber":53,"sourceCode":"\tif e.childCmd.SysProcAttr != nil && e.childCmd.SysProcAttr.Setpgid {\n\t\te.logger.Trace(\"sending sigkill to process group\", \"pid\", pid, \"negative\", negative, \"signal\", signal)\n\t\tif err := syscall.Kill(negative, signal); err != nil && err.Error() != noSuchProcessErr {\n\t\t\treturn err\n\t\t}\n\t\treturn nil\n\t}\n\treturn process.Kill()\n}\n\n// Only send the process a shutdown signal (default INT), doesn't\n// necessarily kill it.\nfunc (e *UniversalExecutor) shutdownProcess(sig os.Signal, proc *os.Process) error {\n\tif sig == nil {\n\t\tsig = os.Interrupt\n\t}\n\n\tif err := proc.Signal(sig); err != nil && err.Error() != finishedErr {\n\t\treturn fmt.Errorf(\"executor shutdown error: %v\", err)\n\t}\n\n\treturn nil\n}\n\n// setCmdUser takes a user id as a string and looks up the user, and sets the command\n// to execute as that user.\nfunc setCmdUser(cmd *exec.Cmd, userid string) error {\n\tu, err := users.Lookup(userid)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to identify user %v: %v\", userid, err)\n\t}\n\n\t// Get the groups the user is a part of\n\tgidStrings, err := u.GroupIds()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"unable to lookup user's group membership: %v\", err)\n\t}","sourceCodeStart":35,"sourceCodeEnd":71,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/drivers/shared/executor/executor_unix.go#L35-L71","documentation":"shutdownProcess sends a signal (default SIGINT) to the task's OS process during Shutdown. If Signal returns any error other than the expected 'process already finished' (finishedErr), it is wrapped in this message. Commonly it means the process already exited in a different way or is not signalable (e.g. zombie/permission).","triggerScenarios":"UniversalExecutor.Shutdown -> shutdownProcess calls proc.Signal(sig); signal fails with 'os: process already finished' variants not matching finishedErr, 'no such process', or EPERM.","commonSituations":"Task process crashed just before shutdown (already finished with slightly different error text); process reaped by another watcher; client stopping a task whose PID changed (exec'd wrapper exited); permission issues signaling a setuid'd user process.","solutions":["Treat as benign if the task already exited — check task logs and let Nomad reconcile state","Use ForceStop/kill escalation if a stale process remains (kill the process group)","Check whether a wrapper/exec chain exited early leaving a stale PID, and fix the driver's process handling","Upgrade Nomad if a known mismatch in finishedErr comparison was fixed in newer versions"],"exampleFix":"// before\nif err := proc.Signal(sig); err != nil && err.Error() != finishedErr {\n    return fmt.Errorf(\"executor shutdown error: %v\", err)\n}\n// after (caller-side tolerance)\nif err := exec.Shutdown(); err != nil {\n    if strings.Contains(err.Error(), \"process already finished\") {\n        return nil // benign\n    }\n    return err\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err := executor.Shutdown(); err != nil {\n    if strings.Contains(err.Error(), \"executor shutdown error\") {\n        // likely process already exited; check state and continue teardown\n        logger.Warn(\"shutdown signal failed\", \"err\", err)\n        return nil\n    }\n    return err\n}","preventionTips":["Treat signal failures on already-exited processes as benign","Stop tasks via Nomad APIs so PIDs are managed in one place","Check task exit state before/after shutdown to confirm the process ended"],"tags":["process-management","signals","shutdown"],"backgroundTag":"process-already-finished","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}