{"record":{"id":"5ab772725bda00a7","repo":"containerd/containerd","slug":"unexpected-metrics-response-v","errorCode":null,"errorMessage":"unexpected metrics response: %+v","messagePattern":"unexpected metrics response: %\\+v","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/cri/server/container_stats.go","lineNumber":40,"sourceCode":"\n\t\"github.com/containerd/containerd/api/services/tasks/v1\"\n\truntime \"k8s.io/cri-api/pkg/apis/runtime/v1\"\n)\n\n// ContainerStats returns stats of the container. If the container does not\n// exist, the call returns an error.\nfunc (c *criService) ContainerStats(ctx context.Context, in *runtime.ContainerStatsRequest) (*runtime.ContainerStatsResponse, error) {\n\tcntr, err := c.containerStore.Get(in.GetContainerId())\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to find container: %w\", err)\n\t}\n\trequest := &tasks.MetricsRequest{Filters: []string{\"id==\" + cntr.ID}}\n\tresp, err := c.client.TaskService().Metrics(ctx, request)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to fetch metrics for task: %w\", err)\n\t}\n\tif len(resp.Metrics) != 1 {\n\t\treturn nil, fmt.Errorf(\"unexpected metrics response: %+v\", resp.Metrics)\n\t}\n\n\thandler, err := c.getMetricsHandler(ctx, cntr.SandboxID)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tcs, err := handler(cntr.Metadata, resp.Metrics[0])\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to decode container metrics: %w\", err)\n\t}\n\treturn &runtime.ContainerStatsResponse{Stats: cs.stats}, nil\n}\n","sourceCodeStart":22,"sourceCodeEnd":54,"githubUrl":"https://github.com/containerd/containerd/blob/4246446a2bf7d03837b0244118d858799393bd80/internal/cri/server/container_stats.go#L22-L54","documentation":"ContainerStats filters the task Metrics response by container ID and requires exactly one matching metric. If the response contains zero or multiple metrics, the response shape violates the contract and this error names the unexpected response payload. This guards against containerd returning inconsistent results for an ID-filtered query.","triggerScenarios":"Calling ContainerStats when the task vanished between the store lookup and the Metrics call (0 results), or when a containerd bug/filter misconfiguration returns more than one metric for the ID filter.","commonSituations":"Task exiting concurrently with the stats request; containerd version bugs in Metrics filtering; custom shim returning duplicate metric entries; ID filter string built incorrectly by an older CRI shim.","solutions":["Retry once — the common cause is a race where the task exited mid-query","Re-check container state and treat zero metrics as container-exited","Upgrade containerd if duplicate/incorrect metric responses persist","Check that the container ID used for the filter has no unusual characters and matches exactly one task"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// ensure exactly one task exists for the ID before querying\nlist, err := runtimeService.ListContainers(&runtime.ContainerFilter{Id: id})\nif err != nil || len(list) != 1 { return fmt.Errorf(\"expected exactly one container for %s\", id) }","typeGuard":null,"tryCatchPattern":"stats, err := runtimeService.ContainerStats(id)\nif err != nil && strings.Contains(err.Error(), \"unexpected metrics response\") {\n    // likely the task exited mid-query; refresh state and retry once\n    time.Sleep(200 * time.Millisecond)\n    return collectStats(id) // bounded retry\n}","preventionTips":["Bound retries to 1-2 attempts — this is usually a task-exit race","Keep containerd updated if duplicate metric responses recur","Alert on persistent occurrences; they indicate containerd/shim bugs, not workload issues"],"tags":["containerd","metrics","invariant-violation","cri"],"backgroundTag":"unexpected-metrics-response","analyzedSha":"4246446a2bf7d03837b0244118d858799393bd80","analyzedAt":"2026-09-02T00:14:43.053Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}