cilium/cilium · error

retrieving CiliumNodes store: %w

Error message

retrieving CiliumNodes store: %w

What it means

In gc.run (operator/pkg/kvstore/nodesgc/gc.go:145), the GC worker first obtains the shared CiliumNode cache store via g.ciliumNodes.Store(ctx). This blocks until the CiliumNode informer has fully synchronized; if it cannot (context cancelled or the standard 60s resource.StoreTimeout elapses without sync), the job aborts with this wrapped error, so stale kvstore node entries cannot be garbage-collected.

Source

Thrown at operator/pkg/kvstore/nodesgc/gc.go:145

		}),
		job.OneShot("watch-kvstore", func(ctx context.Context, health cell.Health) error {
			health.OK("Primed")
			in.StoreFactory.NewWatchStore(g.cinfo.Name, nodeStore.KeyCreator, &observer{g.queue},
				store.RWSWithOnSyncCallback(func(context.Context) { health.OK("Synced") }),
			).Watch(ctx, g.client, kvstore.JoinKey(nodeStore.NodeStorePrefix, g.cinfo.Name))
			return nil
		}),
	)

	return &g, nil
}

func (g *gc) run(ctx context.Context, health cell.Health) error {
	health.OK("Initializing")

	ciliumNodes, err := g.ciliumNodes.Store(ctx)
	if err != nil {
		return fmt.Errorf("retrieving CiliumNodes store: %w", err)
	}

	pods, err := g.pods.Store(ctx)
	if err != nil {
		return fmt.Errorf("retrieving Pods store: %w", err)
	}

	health.OK("Initialized")
	for g.processNextWorkItem(func(nodeName nodeName) error {
		// Check if the CiliumNode still exists, or got recreated, as we don't
		// need to do anything in that case.
		if _, exists, err := ciliumNodes.GetByKey(resource.Key{Name: string(nodeName)}); err != nil {
			return fmt.Errorf("retrieving CiliumNode %q: %w", nodeName, err)
		} else if exists {
			return nil
		}

		// Check if a Cilium agent is still running on the given node, and

View on GitHub (pinned to ac7b90affa)

Solutions

1. Ensure the CiliumNode CRD is installed and established (kubectl get crd ciliumnodes.cilium.io).
2. Verify operator RBAC allows list/watch of ciliumnodes; reapply cilium.io RBAC manifests/Helm chart.
3. Check API server connectivity and restart the operator; inspect the wrapped error for ErrTimeout vs context cancelled.

kubectl get crd ciliumnodes.cilium.io -o jsonpath='{.status.conditions[?(@.type=="Established")].status}'
kubectl auth can-i list ciliumnodes.cilium.io --as=system:serviceaccount:cilium:cilium-operator

ciliumNodes, err := g.ciliumNodes.Store(ctx)
if errors.Is(err, resource.ErrTimeout) {
    return fmt.Errorf("retrieving CiliumNodes store: %w", err) // retry via job restart
}

When it happens

Trigger: ciliumNodes.Store(ctx) returns ErrTimeout (no Sync event within 60s) or ctx.Err(): API server unreachable, RBAC denies listing ciliumnodes, or shutdown cancels ctx mid-startup.

Common situations: Operator starting while K8s API is degraded; missing ciliumnodes resource permissions for the operator service account; CRD CiliumNode not yet registered in a freshly installed cluster (webhook/CRD install race).

Related errors

• unable to retrieve Pod store from Pod resource watcher: %w
• retrieving Pods store: %w
• retrieving pods indexed by node %q: %w
• poolOnUpsert: %w
• poolOnDelete: %w

AI-assisted analysis of cilium/cilium@ac7b90affa (2026-08-31). Data as JSON: /api/errors/b874b56b008799ab. Report an issue: GitHub.