{"record":{"id":"2a295edb3976eac1","repo":"containerd/containerd","slug":"failed-to-delete-task-w-2a295e","errorCode":null,"errorMessage":"failed to delete task: %w","messagePattern":"failed to delete task: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/cri/server/podsandbox/recover.go","lineNumber":138,"sourceCode":"\t\t\t// Task does not exist, set sandbox state as NOTREADY.\n\t\t\tstatus.State = sandboxstore.StateNotReady\n\t\t} else {\n\t\t\tif taskStatus.Status == containerd.Running {\n\t\t\t\texitCh, err := t.Wait(ctrdutil.NamespacedContext())\n\t\t\t\tif err != nil {\n\t\t\t\t\tif !errdefs.IsNotFound(err) {\n\t\t\t\t\t\treturn status, channel, fmt.Errorf(\"failed to wait for sandbox container task: %w\", err)\n\t\t\t\t\t}\n\t\t\t\t\tstatus.State = sandboxstore.StateNotReady\n\t\t\t\t} else {\n\t\t\t\t\tstatus.State = sandboxstore.StateReady\n\t\t\t\t\tstatus.Pid = t.Pid()\n\t\t\t\t\tchannel = exitCh\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\t// Task is not running. Delete the task and set sandbox state as NOTREADY.\n\t\t\t\tif _, err := t.Delete(ctx, containerd.WithProcessKill); err != nil && !errdefs.IsNotFound(err) {\n\t\t\t\t\treturn status, channel, fmt.Errorf(\"failed to delete task: %w\", err)\n\t\t\t\t}\n\t\t\t\tstatus.State = sandboxstore.StateNotReady\n\t\t\t}\n\t\t}\n\t\treturn status, channel, nil\n\t}()\n\tif err != nil {\n\t\tlog.G(ctx).WithError(err).Errorf(\"Failed to load sandbox status for %q\", cntr.ID())\n\t}\n\n\t// save it to cache in the podsandbox controller\n\tpodSandbox := types.NewPodSandbox(cntr.ID(), s)\n\tpodSandbox.Container = cntr\n\tif meta != nil {\n\t\tpodSandbox.Metadata = *meta\n\t}\n\tpodSandbox.Runtime = sandbox2.RuntimeOpts{\n\t\tName:    info.Runtime.Name,","sourceCodeStart":120,"sourceCodeEnd":156,"githubUrl":"https://github.com/containerd/containerd/blob/4246446a2bf7d03837b0244118d858799393bd80/internal/cri/server/podsandbox/recover.go#L120-L156","documentation":"If the sandbox task exists but is not running, recovery deletes it (t.Delete with WithProcessKill) to clean up leftover state. If Delete fails with a non-NotFound error, it is wrapped with this message. This leaves a dead task behind, preventing clean sandbox recovery and potentially blocking later container/sandbox teardown.","triggerScenarios":"RecoverContainer startup: task status is not Running and t.Delete(ctx, WithProcessKill) returns a genuine error - shim failed to kill/remove the task, RPC error, or the underlying init process cannot be signaled.","commonSituations":"Zombie/unkillable process in the sandbox (e.g. stuck in uninterruptible D state); orphaned shim after reboot; shim binary/runtime mismatch preventing task cleanup after upgrade.","solutions":["Read the wrapped error to see if the kill or the post-kill cleanup failed","Check for processes stuck in D state (ps axo pid,stat,comm) and resolve underlying IO/storage issues","Restart containerd; if the shim is orphaned, manually kill the shim process so the task can be removed","Remove the stale sandbox container record so recovery and kubelet can recreate the pod"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"if _, err := t.Delete(ctx, containerd.WithProcessKill); err != nil && !errdefs.IsNotFound(err) {\n  log.Warnf(\"task delete failed for %s: %v\", id, err)\n  // check for D-state processes, kill orphaned shim, then retry once\n}","preventionTips":["Watch for processes stuck in uninterruptible D state on the node","Fix underlying IO/storage hangs that keep tasks unkillable","Remove orphaned shims after reboots before CRI recovery runs"],"tags":["containerd","task","delete","cleanup","recovery"],"backgroundTag":"task-delete-failed","analyzedSha":"4246446a2bf7d03837b0244118d858799393bd80","analyzedAt":"2026-09-02T00:14:43.053Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}