containerd/containerd · error

unexpected metrics response: %+v

Error message

unexpected metrics response: %+v

What it means

ContainerStats filters the task Metrics response by container ID and requires exactly one matching metric. If the response contains zero or multiple metrics, the response shape violates the contract and this error names the unexpected response payload. This guards against containerd returning inconsistent results for an ID-filtered query.

Source

Thrown at internal/cri/server/container_stats.go:40

	"github.com/containerd/containerd/api/services/tasks/v1"
	runtime "k8s.io/cri-api/pkg/apis/runtime/v1"
)

// ContainerStats returns stats of the container. If the container does not
// exist, the call returns an error.
func (c *criService) ContainerStats(ctx context.Context, in *runtime.ContainerStatsRequest) (*runtime.ContainerStatsResponse, error) {
	cntr, err := c.containerStore.Get(in.GetContainerId())
	if err != nil {
		return nil, fmt.Errorf("failed to find container: %w", err)
	}
	request := &tasks.MetricsRequest{Filters: []string{"id==" + cntr.ID}}
	resp, err := c.client.TaskService().Metrics(ctx, request)
	if err != nil {
		return nil, fmt.Errorf("failed to fetch metrics for task: %w", err)
	}
	if len(resp.Metrics) != 1 {
		return nil, fmt.Errorf("unexpected metrics response: %+v", resp.Metrics)
	}

	handler, err := c.getMetricsHandler(ctx, cntr.SandboxID)
	if err != nil {
		return nil, err
	}

	cs, err := handler(cntr.Metadata, resp.Metrics[0])
	if err != nil {
		return nil, fmt.Errorf("failed to decode container metrics: %w", err)
	}
	return &runtime.ContainerStatsResponse{Stats: cs.stats}, nil
}

View on GitHub (pinned to 4246446a2b)

Solutions

1. Retry once — the common cause is a race where the task exited mid-query
2. Re-check container state and treat zero metrics as container-exited
3. Upgrade containerd if duplicate/incorrect metric responses persist
4. Check that the container ID used for the filter has no unusual characters and matches exactly one task

// ensure exactly one task exists for the ID before querying
list, err := runtimeService.ListContainers(&runtime.ContainerFilter{Id: id})
if err != nil || len(list) != 1 { return fmt.Errorf("expected exactly one container for %s", id) }

stats, err := runtimeService.ContainerStats(id)
if err != nil && strings.Contains(err.Error(), "unexpected metrics response") {
    // likely the task exited mid-query; refresh state and retry once
    time.Sleep(200 * time.Millisecond)
    return collectStats(id) // bounded retry
}

When it happens

Trigger: Calling ContainerStats when the task vanished between the store lookup and the Metrics call (0 results), or when a containerd bug/filter misconfiguration returns more than one metric for the ID filter.

Common situations: Task exiting concurrently with the stats request; containerd version bugs in Metrics filtering; custom shim returning duplicate metric entries; ID filter string built incorrectly by an older CRI shim.

Related errors

• failed to fetch metrics for task: %w
• failed to decode container metrics: %w
• failed to fetch metrics for tasks: %w
• failed to decode container metrics for %q: %w
• failed to get task for container %s: %w

AI-assisted analysis of containerd/containerd@4246446a2b (2026-09-02). Data as JSON: /api/errors/5ab772725bda00a7. Report an issue: GitHub.