{"record":{"id":"16581912d333bd8b","repo":"containerd/containerd","slug":"failed-to-retrieve-checkpoint-index-blob-from-cont","errorCode":null,"errorMessage":"failed to retrieve checkpoint index blob from content store: %w","messagePattern":"failed to retrieve checkpoint index blob from content store: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/cri/server/container_checkpoint_linux.go","lineNumber":234,"sourceCode":"\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"checkpointing container %q failed: %w\", container.ID, err)\n\t}\n\n\t// the checkpoint image has been provided as an index with manifests representing the tar of criu data, the rw layer, and the config\n\tvar (\n\t\tindex        v1.Index\n\t\trawIndex     []byte\n\t\ttargetDesc   = img.Target()\n\t\tcontentStore = img.ContentStore()\n\t)\n\n\t// Once all content from the checkpoint image has been saved, the\n\t// checkpoint image can be remove from the local image store.\n\tdefer c.client.ImageService().Delete(ctx, img.Metadata().Name)\n\n\trawIndex, err = content.ReadBlob(ctx, contentStore, targetDesc)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to retrieve checkpoint index blob from content store: %w\", err)\n\t}\n\tif err = json.Unmarshal(rawIndex, &index); err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to unmarshall blob into checkpoint data OCI index: %w\", err)\n\t}\n\n\t// This internal containerd file is used by checkpointctl for checkpoint archive\n\t// analysis. It lives in the container state dir, which can hold files from a\n\t// prior checkpoint operation, so it is read without following symlinks.\n\tif err := copyNoFollow(\n\t\tfilepath.Join(c.getContainerRootDir(container.ID), crmetadata.StatusFile),\n\t\tfilepath.Join(cpPath, crmetadata.StatusFile),\n\t\t0o600,\n\t); err != nil {\n\t\treturn nil, err\n\t}\n\n\t// dump.log and stats-dump are written directly into cpPath by CRIU via its\n\t// work directory (see withCheckpointOpts above), so they are already present","sourceCodeStart":216,"sourceCodeEnd":252,"githubUrl":"https://github.com/containerd/containerd/blob/4246446a2bf7d03837b0244118d858799393bd80/internal/cri/server/container_checkpoint_linux.go#L216-L252","documentation":"After a successful Checkpoint, containerd returns an image whose index manifest blob must be read back from the local content store via content.ReadBlob. This error means the blob referenced by the checkpoint image's target descriptor could not be fetched from the content store. It usually indicates content-store corruption or concurrent garbage collection of the just-created blob.","triggerScenarios":"Content store GC ran between task.Checkpoint returning and ReadBlob; content store database corruption after an unclean containerd shutdown; disk I/O errors reading the content-store files.","commonSituations":"Nodes with aggressive GC settings; containerd restarted uncleanly (power loss) leaving dangling references; full or failing disks on /var/lib/containerd.","solutions":["Retry the checkpoint — GC races are transient","Run containerd's content-store consistency checks / ctr content ls and remove orphaned blobs, or restart containerd","Disable/relax content GC or protect checkpoint blobs during the operation window","Check disk health and free space under /var/lib/containerd/io.containerd.content.v1.content"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// pre-check disk and store health on the node:\n// df -h /var/lib/containerd && ctr content ls | head","typeGuard":null,"tryCatchPattern":"_, err := s.CheckpointContainer(ctx, req)\nif err != nil && strings.Contains(err.Error(), \"checkpoint index blob from content store\") {\n    // transient GC/corruption: restart containerd, then retry once\n}","preventionTips":["Avoid aggressive content GC settings","Shut down containerd cleanly to prevent store corruption","Monitor disk health on /var/lib/containerd"],"tags":["content-store","checkpoint","containerd"],"backgroundTag":"content-store-blob-missing","analyzedSha":"4246446a2bf7d03837b0244118d858799393bd80","analyzedAt":"2026-09-02T00:14:43.053Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}