containerd/containerd · error

failed to update sandbox %s in core store: %w

Error message

failed to update sandbox %s in core store: %w

What it means

After adding the extension, the code persists it with c.client.SandboxStore().Update(ctx, sandboxInfo, "extensions"); failure is wrapped as `failed to update sandbox %s in core store: %w`. The CRI status was updated but the durable record could not be written, so the update is incomplete and will be lost on restart.

Source

Thrown at internal/cri/server/sandbox_update_resources.go:76

	)
	if err != nil {
		return nil, fmt.Errorf("failed to update sandbox store: %w", err)
	}

	sandboxInfo, err := c.client.SandboxStore().Get(ctx, sandbox.ID)
	if err != nil {
		return nil, fmt.Errorf("failed to get sandbox %s from sandbox store: %w", sandbox.ID, err)
	}

	updatedRes := podsandbox.UpdatedResources{
		Overhead:  overhead,
		Resources: resources,
	}
	if err := sandboxInfo.AddExtension(podsandbox.UpdatedResourcesKey, &updatedRes); err != nil {
		return nil, fmt.Errorf("failed to add updated sandbox resources extension: %w", err)
	}
	if _, err := c.client.SandboxStore().Update(ctx, sandboxInfo, "extensions"); err != nil {
		return nil, fmt.Errorf("failed to update sandbox %s in core store: %w", sandbox.ID, err)
	}

	if err := c.sandboxService.UpdateSandbox(ctx, sandboxInfo.Sandboxer, sandboxInfo.ID, sandboxInfo, "extensions"); err != nil {
		// Tolerate these errors for older sandbox controllers that may not support this yet.
		if !errdefs.IsNotImplemented(err) {
			return nil, fmt.Errorf("failed to update sandbox controller: %w", err)
		}
		log.G(ctx).Tracef("sandbox controller %q does not implement Update endpoint", sandboxInfo.Sandboxer)
	}

	err = c.nri.PostUpdatePodSandboxResources(ctx, &sandbox)
	if err != nil {
		log.G(ctx).WithError(err).Errorf("NRI post-update notification failed")
	}

	return &runtime.UpdatePodSandboxResourcesResponse{}, nil
}

View on GitHub (pinned to 4246446a2b)

Solutions

1. Retry UpdatePodSandboxResources — transient concurrent-update conflicts usually resolve.
2. Verify disk space and metadata DB health (`df -h /var/lib/containerd`, containerd logs for boltDB errors).
3. Confirm the sandbox still exists (`crictl pods`) — if deleted concurrently, recreate instead of retrying.
4. Restart containerd if the metadata store is stuck in a locked/inconsistent state.

Example fix

// before
sudo crictl updatep ...  # fails while disk full
// after
df -h /var/lib/containerd && sudo journalctl -u containerd --since -5m | grep -i bolt
sudo crictl updatep --memory 536870912 "$id"

// pre-checks: disk free and sandbox still present
// df -h /var/lib/containerd && crictl pods | grep <id>

try {
  await client.updatePodSandboxResources(req);
} catch (err) {
  if (String(err).includes('in core store')) {
    // likely concurrent writer/delete — short backoff, verify existence, retry once
    await sleep(500);
    if (await podExists(id)) return client.updatePodSandboxResources(req);
  }
  throw err;
}

When it happens

Trigger: Core SandboxStore.Update with fieldpath "extensions" fails — concurrent writers causing optimistic-concurrency/ETag conflicts, the sandbox record vanished between Get and Update, or the metadata DB write errored (disk full/locked).

Common situations: Kubelet and another controller updating the same pod concurrently; containerd metadata boltDB lock contention; disk-full on /var/lib/containerd; pod deleted between the Get and Update calls.

Related errors

• failed to get sandbox container info: %w
• failed to get sandbox %s from sandbox store: %w
• untrusted workload with explicit runtime handler is not allo
• untrusted workload with host access is not allowed
• failed to query sandbox platform: %w

AI-assisted analysis of containerd/containerd@4246446a2b (2026-09-02). Data as JSON: /api/errors/f01f7cf72d84bd30. Report an issue: GitHub.