tailscale/tailscale · error

error watching config Secret %q: %v

Error message

error watching config Secret %q: %v

What it means

Returned when the watch stream delivers a watch.Error event for the config Secret. This is the API server reporting an error on the watch itself (as opposed to a Go-level error from the client); ev.Object is formatted with %v.

Source

Thrown at cmd/k8s-proxy/internal/config/config.go:243

				continue
			}

			switch ev.Type {
			case watch.Added, watch.Modified:
				// New config available to load.
				var ok bool
				secret, ok = ev.Object.(*corev1.Secret)
				if !ok {
					return fmt.Errorf("unexpected object type %T in watch event for config Secret %q", ev.Object, secretName)
				}
				if secret == nil || secret.Data == nil {
					continue
				}
				if err := ld.configFromSecret(ctx, secret); err != nil {
					return fmt.Errorf("error reloading config Secret %q: %v", secret.Name, err)
				}
			case watch.Error:
				return fmt.Errorf("error watching config Secret %q: %v", secretName, ev.Object)
			default:
				// Ignore, no action required.
				continue
			}
		}
	}
}

func (ld *configLoader) configFromSecret(ctx context.Context, s *corev1.Secret) error {
	b := s.Data[kubetypes.KubeAPIServerConfigFile]
	if len(b) == 0 {
		return fmt.Errorf("config Secret %q does not contain expected config in key %q", s.Name, kubetypes.KubeAPIServerConfigFile)
	}

	if err := ld.reloadConfig(ctx, b); err != nil {
		return err
	}

View on GitHub (pinned to cfe32b8be6)

Solutions

1. Treat as transient first: restart the proxy pod, which establishes a fresh watch with a current resourceVersion
2. Re-check RBAC (watch verb) and cluster events for API server errors around that time
3. If 410 Gone recurs, reduce churn on the Secret or move to a dedicated config object

for {
	err := cfgLoader.WatchConfig(ctx, path)
	if err == nil || errors.Is(err, context.Canceled) {
		return err
	}
	// watch.Error events (410 Gone, RBAC) are usually recoverable with a fresh watch
	logger.Warnf("secret watch error (%v); restarting watch", err)
	select {
	case <-ctx.Done():
		return ctx.Err()
	case <-time.After(5 * time.Second):
	}
}

When it happens

Trigger: The watch request fails server-side: 410 Gone when the resourceVersion is too old to resume, forbidden responses after an RBAC change mid-watch, or API server internal errors surfaced as watch Error events.

Common situations: The Secret churned heavily so the watch fell behind and the server expired it; RBAC tightened while the proxy was running; etcd/API server instability.

Related errors

• failed to watch config Secret %q: %w
• failed to re-watch config Secret %q: %w
• too many collisions generating new session; please refresh p
• user is not an admin
• authkey found in TS_KUBE_SECRET, but the pod doesn't have pa

AI-assisted analysis of tailscale/tailscale@cfe32b8be6 (2026-08-15). Data as JSON: /api/errors/e521009f410b9d52. Report an issue: GitHub.