containerd/containerd · warning

an error occurred when trying to find sandbox %s: %w

Error message

an error occurred when trying to find sandbox %s: %w

What it means

PodSandboxStats first resolves the requested sandbox ID in the in-memory sandbox store; this error wraps the store's not-found (or other store) error. It means the CRI server has no record of the given pod sandbox ID.

Source

Thrown at internal/cri/server/sandbox_stats.go:35

package server

import (
	"context"
	"fmt"

	cg1 "github.com/containerd/cgroups/v3/cgroup1/stats"
	cg2 "github.com/containerd/cgroups/v3/cgroup2/stats"
	runtime "k8s.io/cri-api/pkg/apis/runtime/v1"
)

func (c *criService) PodSandboxStats(
	ctx context.Context,
	r *runtime.PodSandboxStatsRequest,
) (*runtime.PodSandboxStatsResponse, error) {

	sandbox, err := c.sandboxStore.Get(r.GetPodSandboxId())
	if err != nil {
		return nil, fmt.Errorf("an error occurred when trying to find sandbox %s: %w", r.GetPodSandboxId(), err)
	}

	podSandboxStats, err := c.podSandboxStats(ctx, sandbox)
	if err != nil {
		return nil, fmt.Errorf("failed to decode pod sandbox metrics %s: %w", r.GetPodSandboxId(), err)
	}

	return &runtime.PodSandboxStatsResponse{Stats: podSandboxStats}, nil
}

type cgroupMetrics struct {
	v1 *cg1.Metrics
	v2 *cg2.Metrics
}

View on GitHub (pinned to 4246446a2b)

Solutions

1. List sandboxes (PodSandbox/ListSandboxes) and confirm the ID exists before requesting stats
2. Handle the wrapped errdefs.ErrNotFound in the caller and drop/invalidate the stale cache entry
3. Verify the sandbox ID is exact (64-hex containerd ID), not a pod name or partial prefix
4. If it occurs after restarts, reconcile kubelet/runtime state (restart kubelet or let it resync)

Example fix

// before
stats, err := runtimeSvc.PodSandboxStats(ctx, &runtime.PodSandboxStatsRequest{PodSandboxId: id})
// after: guard with existence check
if _, err := store.Get(id); err != nil { return nil // sandbox gone; skip stats }
stats, err := runtimeSvc.PodSandboxStats(ctx, &runtime.PodSandboxStatsRequest{PodSandboxId: id})

// confirm existence before requesting stats
resp, err := client.ListPodSandbox(ctx, &runtime.ListPodSandboxRequest{})
if err != nil { return err }
found := false
for _, s := range resp.GetItems() {
    if s.GetId() == sandboxID { found = true; break }
}
if !found { return errdefs.ErrNotFound } // skip stats call

func sandboxExists(items []*runtime.PodSandbox, id string) bool {
    for _, s := range items {
        if s.GetId() == id { return true }
    }
    return false
}

stats, err := client.PodSandboxStats(ctx, req)
if err != nil {
    if errors.Is(err, errdefs.ErrNotFound) || strings.Contains(err.Error(), "trying to find sandbox") {
        cache.Invalidate(sandboxID) // stale entry; skip without alerting
        return nil
    }
    return err
}

When it happens

Trigger: Kubelet calls PodSandboxStats with r.GetPodSandboxId() that is not in the sandboxStore — the sandbox was removed, the runtime restarted and lost in-memory state, or the ID string is wrong/empty.

Common situations: Pod deleted concurrently while kubelet polls stats; containerd restart wiping the sandbox store while the kubelet's cache still holds the ID; metrics pipeline scraping stale sandbox IDs.

Related errors

• windows stats is empty
• failed to decode container metrics: %w
• failed to fetch containers and stats: %w
• failed to build metrics request: %w
• failed to convert to cri containerd stats format: %w

AI-assisted analysis of containerd/containerd@4246446a2b (2026-09-02). Data as JSON: /api/errors/4f106a019dd24774. Report an issue: GitHub.