{"record":{"id":"91d7c580cff2fecc","repo":"kubernetes/kops","slug":"error-listing-nodes-v","errorCode":null,"errorMessage":"error listing nodes: %v","messagePattern":"error listing nodes: (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"dns-controller/pkg/watchers/node.go","lineNumber":87,"sourceCode":"\tstopCh := c.StopChannel()\n\tgo c.runWatcher(stopCh)\n\n\t<-stopCh\n\tklog.Infof(\"shutting down node controller\")\n}\n\nfunc (c *NodeController) runWatcher(stopCh <-chan struct{}) {\n\trunOnce := func() (bool, error) {\n\t\tctx := context.TODO()\n\n\t\tvar listOpts metav1.ListOptions\n\t\tklog.V(4).Infof(\"querying without field filter\")\n\n\t\t// Note we need to watch all the nodes, to set up alias targets\n\t\tallKeys := c.scope.AllKeys()\n\t\tnodeList, err := c.client.CoreV1().Nodes().List(ctx, listOpts)\n\t\tif err != nil {\n\t\t\treturn false, fmt.Errorf(\"error listing nodes: %v\", err)\n\t\t}\n\t\tfoundKeys := make(map[string]bool)\n\t\tfor i := range nodeList.Items {\n\t\t\tnode := &nodeList.Items[i]\n\t\t\tklog.V(4).Infof(\"found node: %v\", node.Name)\n\t\t\tkey := c.updateNodeRecords(node)\n\t\t\tfoundKeys[key] = true\n\t\t}\n\t\tfor _, key := range allKeys {\n\t\t\tif !foundKeys[key] {\n\t\t\t\t// The node previously existed, but no longer exists; delete it from the scope\n\t\t\t\tklog.V(2).Infof(\"removing node not found in list: %s\", key)\n\t\t\t\tc.scope.Replace(key, nil)\n\t\t\t}\n\t\t}\n\t\tc.scope.MarkReady()\n\n\t\tlistOpts.Watch = true","sourceCodeStart":69,"sourceCodeEnd":105,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/dns-controller/pkg/watchers/node.go#L69-L105","documentation":"The dns-controller's nodeController.runUpdater wraps any failure from the Kubernetes Nodes().List API call into \"error listing nodes: %v\". The controller must watch all nodes to set up alias DNS targets, so if the initial list fails it cannot sync records and returns this error to the retry loop. The underlying cause (the %v suffix) is the client-go error, typically a network/timeout, RBAC denial, or invalid ResourceVersion issue.","triggerScenarios":"Occurs when client.CoreV1().Nodes().List(ctx, listOpts) inside nodeController.runUpdater returns an error: API server unreachable, 403 on nodes list (missing RBAC permission on nodes resource), context deadline exceeded, or an expired/invalid ResourceVersion in listOpts when resuming.","commonSituations":"Running dns-controller with a ServiceAccount lacking 'get/list nodes' cluster rights; network partition or firewall blocking the apiserver; apiserver restarting during controller startup; misconfigured --kubeconfig or in-cluster credentials.","solutions":["Check RBAC: grant the controller's ServiceAccount cluster-wide list/watch on nodes (the bundled dns-controller RBAC manifest includes this).","Verify apiserver connectivity from the controller pod: kubectl exec into it and curl the apiserver health endpoint.","Inspect the wrapped error after '%v' in the log to identify the exact cause (403 vs timeout vs DNS failure).","If it happens once at startup, note the controller retries automatically; only intervene if it loops continuously."],"exampleFix":"// before: clusterrole without node access\nrules:\n- apiGroups: [\"\"]\n  resources: [\"services\"]\n  verbs: [\"list\",\"watch\"]\n// after: include nodes\nrules:\n- apiGroups: [\"\"]\n  resources: [\"services\",\"nodes\"]\n  verbs: [\"list\",\"watch\"]","handlingStrategy":"try-catch","validationCode":"// Go: preflight the API before starting the controller\n_, err := client.CoreV1().Nodes().List(ctx, metav1.ListOptions{Limit: 1})\nif err != nil {\n    return fmt.Errorf(\"preflight node list failed: %w\", err)\n}","typeGuard":null,"tryCatchPattern":"nodes, err := client.CoreV1().Nodes().List(ctx, listOpts)\nif err != nil {\n    klog.Errorf(\"node list failed (rbac? connectivity?): %v\", err)\n    // backoff and retry the whole runUpdater cycle\n    time.Sleep(backoff)\n    return false, nil\n}","preventionTips":["Install the kops dns-controller RBAC manifest granting list/watch on nodes.","Smoke-test ServiceAccount permissions with 'kubectl auth can-i list nodes --as=system:serviceaccount:<ns>:<sa>'.","Watch logs for the wrapped error text after 'error listing nodes:' to classify cause."],"tags":["kubernetes","dns-controller","rbac","api-server"],"backgroundTag":"kubernetes-api-list-failed","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}