{"record":{"id":"0203fb0b374bb81c","repo":"hashicorp/nomad","slug":"failed-to-stop-container-s-s","errorCode":null,"errorMessage":"Failed to stop container %s: %s","messagePattern":"Failed to stop container (.+?): (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"drivers/docker/handle.go","lineNumber":223,"sourceCode":"\n\t\t// Stop the container forcefully.\n\t\t_, err = h.dockerClient.ContainerStop(context.Background(), h.containerID, mclient.ContainerStopOptions{Timeout: new(0)})\n\t}\n\n\tif err != nil {\n\t\t// Container has already been removed.\n\t\tif errdefs.IsNotFound(err) {\n\t\t\th.logger.Debug(\"attempted to stop nonexistent container\")\n\t\t\treturn nil\n\t\t}\n\t\t// Container has already been stopped.\n\t\tif errdefs.IsNotModified(err) {\n\t\t\th.logger.Debug(\"attempted to stop an not-running container\")\n\t\t\treturn nil\n\t\t}\n\n\t\th.logger.Error(\"failed to stop container\", \"error\", err)\n\t\treturn fmt.Errorf(\"Failed to stop container %s: %s\", h.containerID, err)\n\t}\n\n\th.logger.Info(\"stopped container\")\n\treturn nil\n}\n\nfunc (h *taskHandle) shutdownLogger() {\n\tif h.dlogger == nil {\n\t\treturn\n\t}\n\n\tif err := h.dlogger.Stop(); err != nil {\n\t\th.logger.Error(\"failed to stop docker logger process during StopTask\",\n\t\t\t\"error\", err, \"logger_pid\", h.dloggerPluginClient.ReattachConfig().Pid)\n\t}\n\th.dloggerPluginClient.Kill()\n}\n","sourceCodeStart":205,"sourceCodeEnd":241,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/drivers/docker/handle.go#L205-L241","documentation":"When killing a task, the driver eventually issues a ContainerStop to shut the container down. If stop fails with any error other than 'not modified' (already stopped), the driver logs it and returns 'Failed to stop container <id>: <err>'. The task was not confirmed stopped.","triggerScenarios":"h.dockerClient.ContainerStop(ctx, h.containerID, ...) returns a daemon error: container already removed, daemon down, timeout reached while the container ignored SIGTERM and stop force-failed, or storage/driver-level errors during shutdown.","commonSituations":"App ignoring SIGTERM so stop times out; Docker daemon crash during shutdown; overlay2/storage errors; container was deleted concurrently by another tool; host under heavy load slowing container teardown.","solutions":["Handle SIGTERM properly in the application or add a trap so the container exits promptly on stop.","Check the Docker daemon logs and restart Docker if it is wedged, then verify the container is gone ('docker rm -f <id>').","Force-remove the orphaned container manually, then re-run 'nomad alloc stop'.","Increase the task's kill_timeout so stop has more time before failing."],"exampleFix":"// before: app ignores SIGTERM\nkill_timeout = \"5s\"\n// after\nkill_timeout = \"30s\"  // plus a SIGTERM handler in the app","handlingStrategy":"try-catch","validationCode":"// give apps a workable SIGTERM window in the job spec\n// kill_timeout = \"30s\"","typeGuard":null,"tryCatchPattern":"err := client.Nodes.StopAlloc(ctx, allocID, nil)\nif err != nil && strings.Contains(err.Error(), \"Failed to stop container\") {\n    // container may still exist; force cleanup path\n    log.Printf(\"stop failed, checking container state: %v\", err)\n    // verify alloc state / force-remove orphan via host tooling\n}","preventionTips":["Ensure apps handle SIGTERM and exit promptly.","Set a realistic kill_timeout for slow-shutdown services.","Keep Docker daemon and storage driver healthy on clients."],"tags":["docker-driver","kill","container-stop"],"backgroundTag":"container-stop-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}