{"record":{"id":"651b2d1f7fa7d187","repo":"GoogleContainerTools/skaffold","slug":"initializing-pod-watcher-for-q-w","errorCode":null,"errorMessage":"initializing pod watcher for %q: %w","messagePattern":"initializing pod watcher for %q: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/skaffold/kubernetes/watcher.go","lineNumber":95,"sourceCode":"\t\tfor _, w := range watchers {\n\t\t\tw.Stop()\n\t\t}\n\t}\n\n\tkubeclient, err := client.Client(kubeContext)\n\tif err != nil {\n\t\treturn func() {}, fmt.Errorf(\"getting k8s client: %w\", err)\n\t}\n\n\tvar forever int64 = 3600 * 24 * 365 * 100\n\n\tfor _, ns := range namespaces {\n\t\twatcher, err := kubeclient.CoreV1().Pods(ns).Watch(context.Background(), metav1.ListOptions{\n\t\t\tTimeoutSeconds: &forever,\n\t\t})\n\t\tif err != nil {\n\t\t\tstopWatchers()\n\t\t\treturn func() {}, fmt.Errorf(\"initializing pod watcher for %q: %w\", ns, err)\n\t\t}\n\n\t\twatchers = append(watchers, watcher)\n\t\tgo func() {\n\t\t\tl := log.Entry(ctx)\n\t\t\tdefer l.Tracef(\"podWatcher: cease waiting\")\n\t\t\tl.Tracef(\"podWatcher: waiting\")\n\t\t\tfor {\n\t\t\t\tselect {\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t\tl.Tracef(\"podWatcher: context canceled, returning\")\n\t\t\t\t\treturn\n\t\t\t\tcase evt, ok := <-watcher.ResultChan():\n\t\t\t\t\tif !ok {\n\t\t\t\t\t\tl.Tracef(\"podWatcher: channel closed, returning\")\n\t\t\t\t\t\treturn\n\t\t\t\t\t}\n\t\t\t\t\t// If the event's type is \"ERROR\", log and continue.","sourceCodeStart":77,"sourceCodeEnd":113,"githubUrl":"https://github.com/GoogleContainerTools/skaffold/blob/a1189de023efc32d4b8e11f395acc678aa555011/pkg/skaffold/kubernetes/watcher.go#L77-L113","documentation":"Skaffold's pod watcher creates a Kubernetes CoreV1 Pod watch API per namespace (with a long TimeoutSeconds) so log streams can follow pods. When the watch call to the Kubernetes API server fails during setup, it stops any watchers already created and wraps the client error with the namespace name. It indicates the pod watch could not be established at all, not a mid-stream failure.","triggerScenarios":"kubeclient.CoreV1().Pods(ns).Watch() returns an error: API server unreachable, invalid/unresolvable namespace, RBAC denies 'watch' on pods in that namespace, expired/stale kubeconfig credentials, or context timeouts.","commonSituations":"Running skaffold dev against a cluster the kubeconfig points to but whose API server is down; missing ClusterRole watch permission; typo'd namespace or deleted namespace; VPN/proxy blocking the API endpoint; token expired after kubectl login rotated.","solutions":["Verify cluster connectivity with 'kubectl get pods -n <ns>' to isolate auth vs network issues","Check RBAC: ensure the identity has get/list/watch on pods in the namespace (kubectl auth can-i watch pods -n <ns>)","Confirm the namespace exists and is correctly passed (e.g. via --namespace or profiles)","Refresh kubeconfig credentials (kubectl config use-context / re-login / gcloud auth login)","Retry once the API server is reachable; the error is transient when caused by network blips"],"exampleFix":"// before: watching a namespace that does not exist / not permitted\nnamespaces := []string{\"does-not-exist\"}\n// after: resolve/validate namespaces against the cluster first\nns := clientcmd.DefaultNamespace\nif userNs != \"\" {\n    if _, err := kubeclient.CoreV1().Namespaces().Get(ctx, userNs, metav1.GetOptions{}); err != nil {\n        return fmt.Errorf(\"namespace %q not found: %w\", userNs, err)\n    }\n    ns = userNs\n}","handlingStrategy":"try-catch","validationCode":"// before starting watchers\nfor _, ns := range namespaces {\n    if _, err := kubeclient.CoreV1().Namespaces().Get(ctx, ns, metav1.GetOptions{}); err != nil {\n        return fmt.Errorf(\"namespace %q not accessible: %w\", ns, err)\n    }\n}\nok, _ := authClient.Check(ctx, \"watch pods -n \"+ns) // or kubectl auth can-i watch pods -n <ns>","typeGuard":"func canWatchPods(auth authorizationv1.AuthorizationV1Interface, ns string) bool {\n    sar := &authorizationv1.SelfSubjectAccessReview{\n        Spec: authorizationv1.SelfSubjectAccessReviewSpec{\n            ResourceAttributes: &authorizationv1.ResourceAttributes{Verb: \"watch\", Resource: \"pods\", Namespace: ns},\n        },\n    }\n    r, err := auth.SelfSubjectAccessReviews().Create(context.Background(), sar, metav1.CreateOptions{})\n    return err == nil && r.Status.Allowed\n}","tryCatchPattern":"watchFn, err := Start(ctx, namespaces)\nif err != nil {\n    var werr *fmt.WrapError // or inspect message\n    if errors.As(err, &werr) && strings.Contains(err.Error(), \"initializing pod watcher\") {\n        log.Printf(\"kube watch setup failed: %v; check kubeconfig/RBAC, retrying in 30s\", err)\n        time.Sleep(30 * time.Second)\n        return retry()\n    }\n    return err\n}","preventionTips":["Run 'kubectl auth can-i watch pods -n <ns>' before launching skaffold","Keep kubeconfig context and credentials fresh (re-login before long sessions)","Validate namespaces exist before passing them via --namespace/profiles","Check API server reachability (kubectl version) when on VPN or behind a proxy"],"tags":["kubernetes","watch","rbac","network"],"backgroundTag":"kubernetes-watch-failed","analyzedSha":"a1189de023efc32d4b8e11f395acc678aa555011","analyzedAt":"2026-09-05T12:09:27.064Z","contentChangedAt":"2026-09-05T12:09:27.064Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}