{"record":{"id":"1f45cb87b3837bcf","repo":"GoogleContainerTools/skaffold","slug":"attempting-to-watch-verify-pods-in-cluster","errorCode":null,"errorMessage":"attempting to watch verify pods in cluster","messagePattern":"attempting to watch verify pods in cluster","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/skaffold/verify/k8sjob/verify.go","lineNumber":269,"sourceCode":"\treturn execErr\n}\n\nfunc (v *Verifier) watchJob(ctx context.Context, clientset k8sclient.Interface, job *batchv1.Job, tc latest.VerifyTestCase) error {\n\tw, err := clientset.BatchV1().Jobs(job.Namespace).Watch(ctx,\n\t\tmetav1.ListOptions{FieldSelector: fmt.Sprintf(\"metadata.name=%s\", job.Name)})\n\tif err != nil {\n\t\teventV2.VerifyFailed(tc.Name, err)\n\t\treturn errors.Wrap(err, \"attempting to watch verify job in cluster\")\n\t}\n\tdefer w.Stop()\n\n\tw, err = clientset.CoreV1().Pods(job.Namespace).Watch(ctx,\n\t\tmetav1.ListOptions{\n\t\t\tLabelSelector: labels.Set(map[string]string{\"job-name\": job.Name}).String(),\n\t\t})\n\tif err != nil {\n\t\teventV2.VerifyFailed(tc.Name, err)\n\t\treturn errors.Wrap(err, \"attempting to watch verify pods in cluster\")\n\t}\n\tdefer w.Stop()\n\n\tvar podErr error\n\tfor event := range w.ResultChan() {\n\t\tpod, ok := event.Object.(*corev1.Pod)\n\t\tif ok {\n\t\t\tif pod.Status.Phase == corev1.PodSucceeded {\n\t\t\t\t// TODO(aaron-prindle) add support for jobs w/ multiple pods in the future\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tif pod.Status.Phase == corev1.PodFailed {\n\t\t\t\tfailReason := pod.Status.Reason\n\t\t\t\tif failReason == \"\" {\n\t\t\t\t\tfailReason = \"<empty>\"\n\t\t\t\t}\n\n\t\t\t\tfailMessage := pod.Status.Message","sourceCodeStart":251,"sourceCodeEnd":287,"githubUrl":"https://github.com/GoogleContainerTools/skaffold/blob/a1189de023efc32d4b8e11f395acc678aa555011/pkg/skaffold/verify/k8sjob/verify.go#L251-L287","documentation":"After watching the Job, watchJob also watches the Pods labeled job-name=<job> to capture pod logs/status. If that pod Watch API call fails, the test is marked failed with 'attempting to watch verify pods in cluster' wrapping the kube error.","triggerScenarios":"clientset.CoreV1().Pods(ns).Watch(...) with LabelSelector job-name=<job> returns an error — RBAC denies watching pods, namespace missing, or API server errors, immediately after the job watch succeeded.","commonSituations":"ServiceAccount lacking 'watch pods' permission; pod label selector namespace mismatch; cluster connectivity drops between the two watch calls; restricted environments blocking pod watches.","solutions":["Verify RBAC: `kubectl auth can-i watch pods -n <ns>` and add pods get/list/watch to the role","Confirm pods with label job-name=<job> exist in the namespace","Check API server connectivity/kubeconfig between job and pod watch steps","Rerun `skaffold verify` if the failure was transient"],"exampleFix":"// before\nverbs: [\"create\"]\n// after (include pod watch perms)\n- apiGroups: [\"\"]\n  resources: [\"pods\", \"pods/log\"]\n  verbs: [\"get\", \"list\", \"watch\"]","handlingStrategy":"validation","validationCode":"// Pre-check watch permission on pods\nif err := exec.Command(\"kubectl\", \"auth\", \"can-i\", \"watch\", \"pods\", \"-n\", ns).Run(); err != nil {\n    return fmt.Errorf(\"serviceaccount cannot watch pods in namespace %s\", ns)\n}","typeGuard":null,"tryCatchPattern":"if err := verifier.Verify(ctx, out, tc); err != nil {\n    if strings.Contains(err.Error(), \"attempting to watch verify pods in cluster\") {\n        return fmt.Errorf(\"pod watch failed (add pods/watch RBAC): %w\", err)\n    }\n    return err\n}","preventionTips":["Grant get/list/watch on core pods (and pods/log) to the verify SA","Keep the job's namespace consistent so the label selector matches pods","Retry verify on transient watch failures"],"tags":["kubernetes","verify","watch","pods","rbac"],"backgroundTag":"kubernetes-watch-failed","analyzedSha":"a1189de023efc32d4b8e11f395acc678aa555011","analyzedAt":"2026-09-05T12:09:27.064Z","contentChangedAt":"2026-09-05T12:09:27.064Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}