kubernetes/kops · error

error watching ingresses: %v

Error message

error watching ingresses: %v

What it means

After an initial list, the ingress watcher establishes a Watch on ingresses (NetworkingV1().Ingresses(namespace).Watch) starting from the list's ResourceVersion; if the watch request fails the error is wrapped and the watcher signals the outer loop to retry. This is distinct from errors on the watch's result channel, which are handled in-loop.

Source

Thrown at dns-controller/pkg/watchers/ingress.go:100

			ingress := &ingressList.Items[i]
			klog.V(4).Infof("found ingress: %v", ingress.Name)
			key := c.updateIngressRecords(ingress)
			foundKeys[key] = true
		}
		for _, key := range allKeys {
			if !foundKeys[key] {
				// The ingress previously existed, but no longer exists; delete it from the scope
				klog.V(2).Infof("removing ingress not found in list: %s", key)
				c.scope.Replace(key, nil)
			}
		}
		c.scope.MarkReady()

		listOpts.Watch = true
		listOpts.ResourceVersion = ingressList.ResourceVersion
		watcher, err := c.client.NetworkingV1().Ingresses(c.namespace).Watch(ctx, listOpts)
		if err != nil {
			return false, fmt.Errorf("error watching ingresses: %v", err)
		}
		ch := watcher.ResultChan()
		for {
			select {
			case <-stopCh:
				klog.Infof("Got stop signal")
				return true, nil
			case event, ok := <-ch:
				if !ok {
					klog.Infof("ingress watch channel closed")
					return false, nil
				}

				ingress := event.Object.(*v1.Ingress)
				klog.V(4).Infof("ingress changed: %s %v", event.Type, ingress.Name)

				switch event.Type {
				case watch.Added, watch.Modified:

View on GitHub (pinned to 4c8573c808)

Solutions

1. Check the wrapped %v cause; Forbidden -> add watch verb to RBAC
2. Let the controller retry — the outer loop re-lists and re-watches with a fresh ResourceVersion
3. Ensure the dns-controller SA has list+watch on ingresses: verbs ["list","watch","get"]
4. Verify network stability / API server health if connection errors recur

Example fix

// before
verbs: ["list"]
// after
verbs: ["list","watch"]

// ensure watch permission before starting
authz := client.AuthorizationV1().SelfSubjectAccessReviews()
sar, err := authz.Create(ctx, &authorizationv1.SelfSubjectAccessReview{
    Spec: authorizationv1.SelfSubjectAccessReviewSpec{
        ResourceAttributes: &authorizationv1.ResourceAttributes{
            Group: "networking.k8s.io", Resource: "ingresses", Verb: "watch", Namespace: namespace}}})
if err == nil && !sar.Status.Allowed { klog.Fatalf("SA lacks watch on ingresses in %s", namespace) }

ok, err := runOnce(ctx, c, listOpts)
if err != nil {
    if strings.Contains(err.Error(), "error watching ingresses") {
        klog.Warningf("watch setup failed, will re-list and re-watch: %v", err)
        return false, nil // backoff, then fresh list sets a new ResourceVersion
    }
}

When it happens

Trigger: The Watch API call itself fails: RBAC lacks watch verb, ResourceVersion is stale/expired (410 Gone handling depends on the retry loop), API server connection drops at watch setup, or invalid ListOptions (e.g. bad ResourceVersion).

Common situations: ClusterRole granting list but not watch on ingresses; long-lived controller resuming after API server restart with a too-old ResourceVersion; network interruptions in flaky environments; API server rate limiting.

Related errors

• error listing ingresses: %v
• error watching nodes: %v
• error adding needs-update label: %v
• error applying annotation to record addon installation: %v
• error querying namespace %q: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/0ea375a7d0e341ce. Report an issue: GitHub.