containerd/containerd · error

an error occurred when try to find container %q: %w

Error message

an error occurred when try to find container %q: %w

What it means

This error wraps any non-NotFound failure returned by the containerd metadata store (containerStore.Get) while looking up a container during RemoveContainer. It means the CRI plugin could not even consult its local metadata cache for the container ID — something failed below the 'not found' level, such as a broken boltDB metadata store or an internal decode error. It deliberately does NOT fire for unknown IDs; those return success silently.

Source

Thrown at internal/cri/server/container_remove.go:41

	"time"

	containerd "github.com/containerd/containerd/v2/client"
	containerstore "github.com/containerd/containerd/v2/internal/cri/store/container"
	"github.com/containerd/containerd/v2/pkg/tracing"
	"github.com/containerd/errdefs"
	"github.com/containerd/log"
	runtime "k8s.io/cri-api/pkg/apis/runtime/v1"
)

// RemoveContainer removes the container.
func (c *criService) RemoveContainer(ctx context.Context, r *runtime.RemoveContainerRequest) (_ *runtime.RemoveContainerResponse, retErr error) {
	span := tracing.SpanFromContext(ctx)
	start := time.Now()
	ctrID := r.GetContainerId()
	container, err := c.containerStore.Get(ctrID)
	if err != nil {
		if !errdefs.IsNotFound(err) {
			return nil, fmt.Errorf("an error occurred when try to find container %q: %w", ctrID, err)
		}
		// Do not return error if container metadata doesn't exist.
		log.G(ctx).Tracef("RemoveContainer called for container %q that does not exist", ctrID)
		return &runtime.RemoveContainerResponse{}, nil
	}

	defer c.nri.BlockPluginSync().Unblock()

	id := container.ID
	span.SetAttributes(tracing.Attribute("container.id", id))
	i, err := container.Container.Info(ctx)
	if err != nil {
		if !errdefs.IsNotFound(err) {
			return nil, fmt.Errorf("get container info: %w", err)
		}
		// Since containerd doesn't see the container and criservice's content store does,
		// we should try to recover from this state by removing entry for this container
		// from the container store as well and return successfully.

View on GitHub (pinned to 4246446a2b)

Solutions

1. Inspect containerd logs for the wrapped root error to identify whether it is a boltDB/I/O failure
2. Verify filesystem health of /var/lib/containerd (mount rw, not full): check df, dmesg, and disk errors
3. Restart containerd (systemctl restart containerd / crictl info) to clear transient store issues
4. If metadata.db is corrupted, stop containerd, back up and remove/repair metadata.db, then restart (containers may need recreation)
5. Upgrade containerd to a recent release if deserialization errors follow a version change

// Go: check container exists and log the ID before removing
id := r.GetContainerId()
if id == "" { return status.Error(codes.InvalidArgument, "container id required") }
if _, err := crictlInspectContainer(id); err != nil && !isNotFound(err) {
    log.Printf("container %s store lookup degraded: %v", id, err)
}

func isRealStoreFailure(err error) bool {
    return err != nil && !errdefs.IsNotFound(err)
}

resp, err := client.RemoveContainer(ctx, &runtime.RemoveContainerRequest{ContainerId: id})
if err != nil {
    if st, ok := status.FromError(err); ok && st.Code() == codes.NotFound {
        return nil // already gone, treat as success
    }
    return fmt.Errorf("remove container %s: metadata store read failed: %w", id, err)
}

When it happens

Trigger: containerStore.Get returns an error that is not errdefs.IsNotFound — e.g. the containerd metadata (bolt) DB is corrupted, the container record fails to deserialize, or the underlying store returns an internal error while reading the container by ID.

Common situations: Disk corruption or truncation of containerd's metadata.db after a crash/power loss; containerd version downgrade leaving unreadable records; filesystem I/O errors (ENOSPC, EIO) on the state directory; concurrent restart of containerd while the CRI call is in flight.

Related errors

• get container info: %w
• an error occurred when try to find container %q: %w
• failed to get container spec: %w
• failed to update container spec: %w
• `mirrors` cannot be set when `config_path` is provided

AI-assisted analysis of containerd/containerd@4246446a2b (2026-09-02). Data as JSON: /api/errors/6cc27f9b6a1910df. Report an issue: GitHub.