{"record":{"id":"58c8bf2093dcca85","repo":"hashicorp/nomad","slug":"docker-container-exited-with-non-zero-exit-code","errorCode":null,"errorMessage":"Docker container exited with non-zero exit code: %d","messagePattern":"Docker container exited with non-zero exit code: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"drivers/docker/handle.go","lineNumber":299,"sourceCode":"func (h *taskHandle) run() {\n\tdefer h.shutdownLogger()\n\n\th.startCpusetFixer()\n\n\tvar werr error\n\tvar exitCode containerapi.WaitResponse\n\t// this needs to use the background context because the container can\n\t// outlive Nomad itself\n\twaitResult := h.infinityClient.ContainerWait(\n\t\tcontext.Background(),\n\t\th.containerID,\n\t\tmclient.ContainerWaitOptions{Condition: containerapi.WaitConditionNotRunning},\n\t)\n\n\tselect {\n\tcase exitCode = <-waitResult.Result:\n\t\tif exitCode.StatusCode != 0 {\n\t\t\twerr = fmt.Errorf(\"Docker container exited with non-zero exit code: %d\", exitCode.StatusCode)\n\t\t}\n\tcase werr = <-waitResult.Error:\n\t\th.logger.Error(\"failed to wait for container; already terminated\")\n\t}\n\n\tctx, inspectCancel := context.WithTimeout(context.Background(), 10*time.Second)\n\tdefer inspectCancel()\n\n\tcontainer, ierr := h.dockerClient.ContainerInspect(ctx, h.containerID, mclient.ContainerInspectOptions{})\n\toom := false\n\tif ierr != nil {\n\t\th.logger.Error(\"failed to inspect container\", \"error\", ierr)\n\t} else if container.Container.State.OOMKilled {\n\t\th.logger.Error(\"OOM Killed\",\n\t\t\t\"container_id\", h.containerID,\n\t\t\t\"container_image\", h.containerImage,\n\t\t\t\"nomad_job_name\", h.task.JobName,\n\t\t\t\"nomad_task_name\", h.task.Name,","sourceCodeStart":281,"sourceCodeEnd":317,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/drivers/docker/handle.go#L281-L317","documentation":"After a task's container exits, the driver waits on it (WaitConditionNotRunning) and, when the exit status code is non-zero, wraps it as 'Docker container exited with non-zero exit code: <code>'. This is the standard way the docker driver surfaces application failure — the container ran but the process returned a failing exit code.","triggerScenarios":"The containerized process terminated by itself with a non-zero status (the select on waitResult.Result yields StatusCode != 0). Not a Docker API error — the wait succeeded, the app failed.","commonSituations":"Application crash, unhandled exception, failed health/startup checks, bad entrypoint arguments, OOM-kill (exit 137), config errors in the app; exit 125/126 for Docker-level command errors.","solutions":["Read the task's stdout/stderr via 'nomad alloc logs <alloc>' to find the application-level failure.","Map the exit code: 137/143 = OOM or SIGKILL/SIGTERM (check memory resources), 1 = app error, 125/126 = container/command setup error.","Fix the app bug, command, or configuration in the job spec and redeploy.","Increase task memory/CPU if the exit was resource-related."],"exampleFix":"// before: app crashes on missing env\nenv { MISSING_VAR = \"\" }\n// after\nenv { MISSING_VAR = \"required-value\" }","handlingStrategy":"fallback","validationCode":"// validate entrypoint/cmd args and required env before starting the task\nfor _, v := range requiredEnv {\n    if os.Getenv(v) == \"\" {\n        return fmt.Errorf(\"missing required env %s\", v)\n    }\n}","typeGuard":null,"tryCatchPattern":"ev, err := client.Allocations().Logs(ctx, alloc, task, \"stdout\", false, nil)\nif err == nil {\n    log.Printf(\"task output: %s\", ev)\n}\n// inspect exit code to branch: 137/143 resource/kill vs app error","preventionTips":["Always fetch alloc logs first — this is an application failure, not a driver bug.","Set restart policies and appropriate memory/CPU to absorb transient failures.","Add health checks/startup validation inside the app to fail fast with clear logs.","Know common codes: 1 app error, 125/126 docker/cmd error, 137 OOM/SIGKILL."],"tags":["docker-driver","exit-code","container-crash"],"backgroundTag":"container-exited-nonzero","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}