{"record":{"id":"b874b56b008799ab","repo":"cilium/cilium","slug":"retrieving-ciliumnodes-store-w","errorCode":null,"errorMessage":"retrieving CiliumNodes store: %w","messagePattern":"retrieving CiliumNodes store: %w","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"operator/pkg/kvstore/nodesgc/gc.go","lineNumber":145,"sourceCode":"\t\t}),\n\t\tjob.OneShot(\"watch-kvstore\", func(ctx context.Context, health cell.Health) error {\n\t\t\thealth.OK(\"Primed\")\n\t\t\tin.StoreFactory.NewWatchStore(g.cinfo.Name, nodeStore.KeyCreator, &observer{g.queue},\n\t\t\t\tstore.RWSWithOnSyncCallback(func(context.Context) { health.OK(\"Synced\") }),\n\t\t\t).Watch(ctx, g.client, kvstore.JoinKey(nodeStore.NodeStorePrefix, g.cinfo.Name))\n\t\t\treturn nil\n\t\t}),\n\t)\n\n\treturn &g, nil\n}\n\nfunc (g *gc) run(ctx context.Context, health cell.Health) error {\n\thealth.OK(\"Initializing\")\n\n\tciliumNodes, err := g.ciliumNodes.Store(ctx)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"retrieving CiliumNodes store: %w\", err)\n\t}\n\n\tpods, err := g.pods.Store(ctx)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"retrieving Pods store: %w\", err)\n\t}\n\n\thealth.OK(\"Initialized\")\n\tfor g.processNextWorkItem(func(nodeName nodeName) error {\n\t\t// Check if the CiliumNode still exists, or got recreated, as we don't\n\t\t// need to do anything in that case.\n\t\tif _, exists, err := ciliumNodes.GetByKey(resource.Key{Name: string(nodeName)}); err != nil {\n\t\t\treturn fmt.Errorf(\"retrieving CiliumNode %q: %w\", nodeName, err)\n\t\t} else if exists {\n\t\t\treturn nil\n\t\t}\n\n\t\t// Check if a Cilium agent is still running on the given node, and","sourceCodeStart":127,"sourceCodeEnd":163,"githubUrl":"https://github.com/cilium/cilium/blob/ac7b90affa4baf0642e6685319d56907b3a73a6d/operator/pkg/kvstore/nodesgc/gc.go#L127-L163","documentation":"In gc.run (operator/pkg/kvstore/nodesgc/gc.go:145), the GC worker first obtains the shared CiliumNode cache store via g.ciliumNodes.Store(ctx). This blocks until the CiliumNode informer has fully synchronized; if it cannot (context cancelled or the standard 60s resource.StoreTimeout elapses without sync), the job aborts with this wrapped error, so stale kvstore node entries cannot be garbage-collected.","triggerScenarios":"ciliumNodes.Store(ctx) returns ErrTimeout (no Sync event within 60s) or ctx.Err(): API server unreachable, RBAC denies listing ciliumnodes, or shutdown cancels ctx mid-startup.","commonSituations":"Operator starting while K8s API is degraded; missing ciliumnodes resource permissions for the operator service account; CRD CiliumNode not yet registered in a freshly installed cluster (webhook/CRD install race).","solutions":["Ensure the CiliumNode CRD is installed and established (kubectl get crd ciliumnodes.cilium.io).","Verify operator RBAC allows list/watch of ciliumnodes; reapply cilium.io RBAC manifests/Helm chart.","Check API server connectivity and restart the operator; inspect the wrapped error for ErrTimeout vs context cancelled."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"kubectl get crd ciliumnodes.cilium.io -o jsonpath='{.status.conditions[?(@.type==\"Established\")].status}'\nkubectl auth can-i list ciliumnodes.cilium.io --as=system:serviceaccount:cilium:cilium-operator","typeGuard":null,"tryCatchPattern":"ciliumNodes, err := g.ciliumNodes.Store(ctx)\nif errors.Is(err, resource.ErrTimeout) {\n    return fmt.Errorf(\"retrieving CiliumNodes store: %w\", err) // retry via job restart\n}","preventionTips":["Install CRDs before the operator (helm install order or Argo/Flux sync waves).","Keep operator RBAC generated from the upstream chart, never hand-trimmed.","Monitor CRD establishment and API server health during cluster bootstrap."],"tags":["kubernetes","operator","informer-cache","crd","go"],"backgroundTag":"k8s-informer-sync-timeout","analyzedSha":"ac7b90affa4baf0642e6685319d56907b3a73a6d","analyzedAt":"2026-08-31T18:27:15.868Z","schemaVersion":2},"datasetVersion":"2026-08-31T19:17:28.585Z"}