cilium/cilium · error

failed to get nodes: %w

Error message

failed to get nodes: %w

What it means

After successfully parsing node metrics, `formatNodeMetricsAsTable` calls c.Client.ListNodes to fetch node capacities (CPU/memory) so usage percentages can be computed. Any Kubernetes API error from the node list call is wrapped as this error.

Source

Thrown at cilium-cli/sysdump/sysdump.go:3634

	cmd.Flags().IntVar(&options.CopyRetryLimit,
		optionPrefix+"copy-retry-limit", DefaultCopyRetryLimit,
		"Retry limit for file copying operations. If set to -1, copying will be retried indefinitely. Useful for collecting sysdump while on unreliable connection.")

	hooks.AddSysdumpFlags(cmd.Flags())
}

// formatMetricsAsTable formats the raw metrics JSON into a table format like kubectl top nodes
func (c *Collector) formatNodeMetricsAsTable(rawMetrics string) (string, error) {
	// Parse the metrics JSON
	var nodeMetrics metricsapi.NodeMetricsList
	if err := json.Unmarshal([]byte(rawMetrics), &nodeMetrics); err != nil {
		return "", fmt.Errorf("failed to parse metrics JSON: %w", err)
	}

	// Get node information to calculate percentages
	nodes, err := c.Client.ListNodes(context.Background(), metav1.ListOptions{})
	if err != nil {
		return "", fmt.Errorf("failed to get nodes: %w", err)
	}

	// Create a map of node names to their capacity
	nodeCapacities := make(map[string]corev1.ResourceList)
	for _, node := range nodes.Items {
		nodeCapacities[node.Name] = node.Status.Capacity
	}

	var sb strings.Builder
	tw := tabwriter.NewWriter(&sb, 0, 0, 2, ' ', 0)
	fmt.Fprintln(tw, "NAME\tCPU(cores)\tCPU(%)\tMEMORY(bytes)\tMEMORY(%)")

	for _, metric := range nodeMetrics.Items {
		name := metric.Name

		// Get current usage
		cpuUsage := metric.Usage[corev1.ResourceCPU]
		memUsage := metric.Usage[corev1.ResourceMemory]

View on GitHub (pinned to ac7b90affa)

Solutions

1. Grant the current identity `list nodes` permission (nodes are cluster-scoped; check with `kubectl auth can-i list nodes`)
2. Verify `kubectl get nodes` works with the same kubeconfig to rule out connectivity/auth issues
3. Renew expired credentials or fix the kubeconfig context
4. Re-run the sysdump after the API server recovers if this was a transient failure

Example fix

// before: failing with restricted RBAC
// after: verify permissions first
kubectl auth can-i list nodes   # must print 'yes'
kubectl get nodes

if allowed, _ := authClient.SelfSubjectAccessReviews().Create(ctx, &authv1.SelfSubjectAccessReview{
	Spec: authv1.SelfSubjectAccessReviewSpec{ResourceAttributes: &authv1.ResourceAttributes{Verb: "list", Resource: "nodes", Group: ""}},
}); !allowed.Status.Allowed {
	return errors.New("identity cannot list nodes; fix RBAC first")
}

func isForbidden(err error) bool { return apierrors.IsForbidden(err) }

nodes, err := c.Client.ListNodes(ctx, metav1.ListOptions{})
if err != nil {
	if apierrors.IsForbidden(err) { return "", fmt.Errorf("missing 'list nodes' RBAC: %w", err) }
	return "", fmt.Errorf("failed to get nodes: %w", err)
}

When it happens

Trigger: Listing Node objects fails during sysdump table formatting: RBAC lacking `list nodes`, API server connection failures, timeouts, or context cancellation.

Common situations: Restricted kubeconfig user without cluster-scoped node read access; API server briefly unavailable; network issues between the CLI host and the cluster; expired credentials/tokens.

Related errors

• failed to collect sysdump: %w
• unable to list K8s Nodes: %w
• failed to list nodes: %w
• could not create exec pod: %w
• failed to collect Kubernetes services: %w

AI-assisted analysis of cilium/cilium@ac7b90affa (2026-08-31). Data as JSON: /api/errors/70d8b4a41a83d791. Report an issue: GitHub.