{"record":{"id":"a4e2dc7cc370f7d3","repo":"kubernetes/kops","slug":"cannot-get-pod-health-for-q-v","errorCode":null,"errorMessage":"cannot get pod health for %q: %v","messagePattern":"cannot get pod health for %q: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/validation/validate_cluster.go","lineNumber":234,"sourceCode":"\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\n\t\tif len(notReadyWorkerNodes) > 0 && len(notReadyWorkerNodes) <= v.maxUnreadyNodes {\n\t\t\ttoleratedNodes = make(map[string]bool)\n\t\t\tfor _, n := range notReadyWorkerNodes {\n\t\t\t\ttoleratedNodes[n] = true\n\t\t\t}\n\t\t\tsort.Strings(notReadyWorkerNodes)\n\t\t\tklog.Warningf(\"Tolerating %d non-ready worker node(s): %s\", len(notReadyWorkerNodes), strings.Join(notReadyWorkerNodes, \", \"))\n\t\t}\n\t}\n\n\treadyNodes, nodeInstanceGroupMapping := validation.validateNodes(cloudGroups, v.allInstanceGroups, v.filterInstanceGroups, toleratedNodes)\n\n\tif err := validation.collectPodFailures(ctx, v.k8sClient, readyNodes, nodeInstanceGroupMapping, v.filterPodsForValidation, toleratedNodes); err != nil {\n\t\treturn nil, fmt.Errorf(\"cannot get pod health for %q: %v\", v.cluster.Name, err)\n\t}\n\n\treturn validation, nil\n}\n\nvar masterStaticPods = []string{\n\t\"kube-apiserver\",\n\t\"kube-controller-manager\",\n\t\"kube-scheduler\",\n}\n\nfunc (v *ValidationCluster) collectPodFailures(ctx context.Context, client kubernetes.Interface, readyNodes []v1.Node, nodeInstanceGroupMapping map[string]*kops.InstanceGroup, podValidationFilter func(pod *v1.Pod) bool, toleratedNodes map[string]bool) error {\n\tlog := klog.FromContext(ctx)\n\n\tmasterWithoutPod := map[string]map[string]bool{}\n\n\tfor _, node := range readyNodes {\n\t\tlabels := node.GetLabels()","sourceCodeStart":216,"sourceCodeEnd":252,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/pkg/validation/validate_cluster.go#L216-L252","documentation":"After nodes validate, Validate calls collectPodFailures to check required pods (including master static pods) are healthy on the right nodes. Any error from that step is wrapped with the cluster name so you know which cluster's pod health check failed. It is a wrapper — the actionable detail is the inner error from collectPodFailures.","triggerScenarios":"validation.collectPodFailures returns a non-nil error, typically because an underlying Pod list call failed (see its own wrapping), or the context passed in is cancelled/timed out mid-check.","commonSituations":"API server connectivity drops while listing pods; RBAC lacks pod list permission in kube-system; validation runs with a short context deadline on a large cluster; cluster name reported helps when validating several clusters in CI.","solutions":["Read the inner %v error to distinguish auth/RBAC/network causes","Run kubectl get pods -A to confirm pod listing works with the same credentials","Retry if the cause was a transient API/timeout issue; increase the command timeout","Ensure the kubeconfig user can list pods in the namespaces being validated"],"exampleFix":"// before\nctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)\nvalidateCluster(ctx) // cannot get pod health for \"prod\": context deadline exceeded\n// after\nctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)\nvalidateCluster(ctx)","handlingStrategy":"try-catch","validationCode":"_, err := k8sClient.CoreV1().Pods(\"kube-system\").List(ctx, metav1.ListOptions{})\nif err != nil {\n    return fmt.Errorf(\"cannot list pods before validation: %v\", err)\n}","typeGuard":null,"tryCatchPattern":"if _, err := v.Validate(ctx); err != nil {\n    if strings.HasPrefix(err.Error(), \"cannot get pod health\") {\n        // inspect inner error: auth/RBAC/network/timeouts\n        return fmt.Errorf(\"pod health check failed for cluster: %w\", err)\n    }\n    return err\n}","preventionTips":["Grant pod list RBAC to the validating identity","Set adequate context deadlines for cluster-wide pod checks","Keep master static pods healthy before running validation","Verify kubeconfig targets the intended cluster (name in the error helps)"],"tags":["go","kubernetes","pods","validation","kops"],"backgroundTag":"kubernetes-api-request-failed","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}