What causes "failed to handle sandbox TaskExit event: %w" in containerd/containerd?

Check the wrapped 'failed to update sandbox state' error; if the sandbox was removed concurrently it is benign

failed to handle sandbox TaskExit event: %w

Error message

failed to handle sandbox TaskExit event: %w

What it means

When a TaskExit event belongs to a sandbox rather than a pod container (container store lookup was NotFound), HandleEvent delegates to handleSandboxExit. This error wraps failures of that handler — e.g. sb.Status.Update failing — for the TaskExit event path. The sandbox exit handler transitions the sandbox to NotReady and propagates the stop to waiters.

Source

Thrown at internal/cri/server/events.go:343

	defer cancel()

	switch e := any.(type) {
	case *eventtypes.TaskExit:
		log.L.Infof("TaskExit event %+v", e)
		// Use ID instead of ContainerID to rule out TaskExit event for exec.
		cntr, err := ce.c.containerStore.Get(e.ID)
		if err == nil {
			if err := ce.c.handleContainerExit(ctx, e, cntr, cntr.SandboxID); err != nil {
				return fmt.Errorf("failed to handle container TaskExit event: %w", err)
			}
			return nil
		} else if !errdefs.IsNotFound(err) {
			return fmt.Errorf("can't find container for TaskExit event: %w", err)
		}
		sb, err := ce.c.sandboxStore.Get(e.ID)
		if err == nil {
			if err := ce.c.handleSandboxExit(ctx, sb, e.ExitStatus, e.ExitedAt.AsTime()); err != nil {
				return fmt.Errorf("failed to handle sandbox TaskExit event: %w", err)
			}
			return nil
		} else if !errdefs.IsNotFound(err) {
			return fmt.Errorf("can't find sandbox for TaskExit event: %w", err)
		}
		return nil
	case *eventtypes.SandboxExit:
		log.L.Infof("SandboxExit event %+v", e)
		sb, err := ce.c.sandboxStore.Get(e.GetSandboxID())
		if err == nil {
			if err := ce.c.handleSandboxExit(ctx, sb, e.ExitStatus, e.ExitedAt.AsTime()); err != nil {
				return fmt.Errorf("failed to handle sandbox TaskExit event: %w", err)
			}
			return nil
		} else if !errdefs.IsNotFound(err) {
			return fmt.Errorf("can't find sandbox for TaskExit event: %w", err)
		}
		return nil

View on GitHub (pinned to 4246446a2b)

Solutions

Check the wrapped 'failed to update sandbox state' error; if the sandbox was removed concurrently it is benign
Allow event backoff to retry; stale events for removed sandboxes are skipped on retry
Check metadata store health (disk IO, bolt errors) if persistent
Verify sandbox lifecycle: crictl pods -a to see stale sandboxes

Defensive patterns

Strategy: try-catch

Validate before calling

// Go: confirm sandbox still exists before deeper handling:
sb, err := sandboxStore.Get(e.ID)
if errdefs.IsNotFound(err) {
    return nil // sandbox already gone
}

Try / catch

if err := handleSandboxExit(ctx, sb, exitStatus, exitedAt); err != nil {
    if strings.Contains(err.Error(), "failed to update sandbox state") {
        // likely concurrent removal; retry via backoff, skip when NotFound on retry
        monitor.Backoff(id, e)
    }
    return fmt.Errorf("failed to handle sandbox TaskExit event: %w", err)
}

Prevention

Avoid deleting sandboxes while their exit events are pending (kubelet/cri timing)
Watch metadata DB write errors in containerd logs
Let backoff retries resolve races with concurrent sandbox removal
Verify with crictl pods -a that sandbox state converges after retries

When it happens

Trigger: TaskExit event whose ID resolves in the sandbox store, and sb.Status.Update fails (concurrent sandbox removal or metadata write error), raising 'failed to update sandbox state: %w' which this wrapper re-raises.

Common situations: Race between sandbox removal by kubelet and the exit event processing; metadata DB write failure under load; sandbox record already cleaned but event requeued by backoff.

Related errors

AI-assisted analysis of containerd/containerd@4246446a2b (2026-09-02). Data as JSON: /api/errors/eaa92635fb76e863. Report an issue: GitHub.