kubernetes/kops · error

error listing nodes: %v

Error message

error listing nodes: %v

What it means

ClusterValidator.Validate lists all Nodes from the live cluster via the Kubernetes API to correlate them with cloud groups and instance groups. If the Nodes().List call fails, this error wraps the client error. Since node listing is foundational to validation, the whole validation fails rather than returning partial results.

Source

Thrown at pkg/validation/validate_cluster.go:184

		if hasPlaceHolderIPAddress != "" {
			message := fmt.Sprintf("Validation Failed\n\n"+
				"The %[1]v Kubernetes deployment has not updated the Kubernetes cluster's API DNS entry to the correct IP address."+
				"  The API DNS IP address is the placeholder address that kops creates: %[2]v."+
				"  Please wait about 5-10 minutes for a control plane node to start, %[1]v to launch, and DNS to propagate."+
				"  The %[1]v deployment logs may contain more diagnostic information."+
				"  Etcd and the API DNS entries must be updated for a kops Kubernetes cluster to start.", dnsProvider, hasPlaceHolderIPAddress)
			validation.addError(&ValidationError{
				Kind:    "dns",
				Name:    "apiserver",
				Message: message,
			})
			return validation, nil
		}
	}

	nodeList, err := v.k8sClient.CoreV1().Nodes().List(ctx, metav1.ListOptions{})
	if err != nil {
		return nil, fmt.Errorf("error listing nodes: %v", err)
	}

	warnUnmatched := false
	cloudGroups, err := v.cloud.GetCloudGroups(v.cluster, v.allInstanceGroups, warnUnmatched, nodeList.Items)
	if err != nil {
		return nil, err
	}

	var toleratedNodes map[string]bool
	if v.maxUnreadyNodes > 0 {
		var notReadyWorkerNodes []string
		for _, cloudGroup := range cloudGroups {
			if cloudGroup.InstanceGroup != nil && cloudGroup.InstanceGroup.Spec.Role.HasNode() {
				var allMembers []*cloudinstances.CloudInstance
				allMembers = append(allMembers, cloudGroup.Ready...)
				allMembers = append(allMembers, cloudGroup.NeedUpdate...)

				for _, member := range allMembers {

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Run kubectl get nodes with the same kubeconfig to reproduce and read the raw error (auth vs network vs RBAC)
  2. Fix credentials: kops export kubecfg <cluster> or refresh OIDC/SSO tokens
  3. If RBAC, grant the user/serviceaccount list permission on nodes at cluster scope
  4. If network, run validation from inside the VPC or fix the API endpoint reachability; retry after transient API server issues

Example fix

// before
$ kops validate cluster
# error listing nodes: Unauthorized
// after
$ kops export kubecfg prod.example.com
$ kubectl get nodes   # works
$ kops validate cluster
Defensive patterns

Strategy: retry

Validate before calling

_, err := k8sClient.CoreV1().Nodes().List(ctx, metav1.ListOptions{})
if err != nil {
    return fmt.Errorf("precondition failed, cannot list nodes: %v", err)
}

Try / catch

err := wait.PollImmediate(5*time.Second, 2*time.Minute, func() (bool, error) {
    _, err := v.k8sClient.CoreV1().Nodes().List(ctx, metav1.ListOptions{})
    if err != nil {
        if apierrors.IsUnauthorized(err) || apierrors.IsForbidden(err) {
            return false, err // not retryable
        }
        return false, nil // transient, retry
    }
    return true, nil
})

Prevention

When it happens

Trigger: v.k8sClient.CoreV1().Nodes().List(ctx, ...) returns an error: expired/insufficient credentials, RBAC denying node list, API server unreachable, or context timeout/cancellation.

Common situations: kubeconfig points to the wrong context or an expired OIDC token; validating an internal-API cluster from outside the VPC; RBAC user lacking cluster-scoped nodes list permission; API server briefly down during an upgrade.

Related errors


AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/8a94ab96cb2ecfde. Report an issue: GitHub.