GoogleContainerTools/skaffold · error

initializing pod watcher for %q: %w

Error message

initializing pod watcher for %q: %w

What it means

Skaffold's pod watcher creates a Kubernetes CoreV1 Pod watch API per namespace (with a long TimeoutSeconds) so log streams can follow pods. When the watch call to the Kubernetes API server fails during setup, it stops any watchers already created and wraps the client error with the namespace name. It indicates the pod watch could not be established at all, not a mid-stream failure.

Source

Thrown at pkg/skaffold/kubernetes/watcher.go:95

		for _, w := range watchers {
			w.Stop()
		}
	}

	kubeclient, err := client.Client(kubeContext)
	if err != nil {
		return func() {}, fmt.Errorf("getting k8s client: %w", err)
	}

	var forever int64 = 3600 * 24 * 365 * 100

	for _, ns := range namespaces {
		watcher, err := kubeclient.CoreV1().Pods(ns).Watch(context.Background(), metav1.ListOptions{
			TimeoutSeconds: &forever,
		})
		if err != nil {
			stopWatchers()
			return func() {}, fmt.Errorf("initializing pod watcher for %q: %w", ns, err)
		}

		watchers = append(watchers, watcher)
		go func() {
			l := log.Entry(ctx)
			defer l.Tracef("podWatcher: cease waiting")
			l.Tracef("podWatcher: waiting")
			for {
				select {
				case <-ctx.Done():
					l.Tracef("podWatcher: context canceled, returning")
					return
				case evt, ok := <-watcher.ResultChan():
					if !ok {
						l.Tracef("podWatcher: channel closed, returning")
						return
					}
					// If the event's type is "ERROR", log and continue.

View on GitHub (pinned to a1189de023)

Solutions

1. Verify cluster connectivity with 'kubectl get pods -n <ns>' to isolate auth vs network issues
2. Check RBAC: ensure the identity has get/list/watch on pods in the namespace (kubectl auth can-i watch pods -n <ns>)
3. Confirm the namespace exists and is correctly passed (e.g. via --namespace or profiles)
4. Refresh kubeconfig credentials (kubectl config use-context / re-login / gcloud auth login)
5. Retry once the API server is reachable; the error is transient when caused by network blips

Example fix

// before: watching a namespace that does not exist / not permitted
namespaces := []string{"does-not-exist"}
// after: resolve/validate namespaces against the cluster first
ns := clientcmd.DefaultNamespace
if userNs != "" {
    if _, err := kubeclient.CoreV1().Namespaces().Get(ctx, userNs, metav1.GetOptions{}); err != nil {
        return fmt.Errorf("namespace %q not found: %w", userNs, err)
    }
    ns = userNs
}

// before starting watchers
for _, ns := range namespaces {
    if _, err := kubeclient.CoreV1().Namespaces().Get(ctx, ns, metav1.GetOptions{}); err != nil {
        return fmt.Errorf("namespace %q not accessible: %w", ns, err)
    }
}
ok, _ := authClient.Check(ctx, "watch pods -n "+ns) // or kubectl auth can-i watch pods -n <ns>

func canWatchPods(auth authorizationv1.AuthorizationV1Interface, ns string) bool {
    sar := &authorizationv1.SelfSubjectAccessReview{
        Spec: authorizationv1.SelfSubjectAccessReviewSpec{
            ResourceAttributes: &authorizationv1.ResourceAttributes{Verb: "watch", Resource: "pods", Namespace: ns},
        },
    }
    r, err := auth.SelfSubjectAccessReviews().Create(context.Background(), sar, metav1.CreateOptions{})
    return err == nil && r.Status.Allowed
}

watchFn, err := Start(ctx, namespaces)
if err != nil {
    var werr *fmt.WrapError // or inspect message
    if errors.As(err, &werr) && strings.Contains(err.Error(), "initializing pod watcher") {
        log.Printf("kube watch setup failed: %v; check kubeconfig/RBAC, retrying in 30s", err)
        time.Sleep(30 * time.Second)
        return retry()
    }
    return err
}

When it happens

Trigger: kubeclient.CoreV1().Pods(ns).Watch() returns an error: API server unreachable, invalid/unresolvable namespace, RBAC denies 'watch' on pods in that namespace, expired/stale kubeconfig credentials, or context timeouts.

Common situations: Running skaffold dev against a cluster the kubeconfig points to but whose API server is down; missing ClusterRole watch permission; typo'd namespace or deleted namespace; VPN/proxy blocking the API endpoint; token expired after kubectl login rotated.

Related errors

• getting pods for namespace %q: %w
• STATUSCHECK_DEPLOYMENT_FETCH_ERR
• could not fetch deployments: %w
• initializing deployment watcher: %s
• attempting to watch verify job in cluster

AI-assisted analysis of GoogleContainerTools/skaffold@a1189de023 (2026-09-05). Data as JSON: /api/errors/651b2d1f7fa7d187. Report an issue: GitHub.