tailscale/tailscale · error

failed to watch IPN bus for auth health: %w

Error message

failed to watch IPN bus for auth health: %w

What it means

In the Tailscale Kubernetes operator's proxy pod, monitorAuthHealth opens a subscription to the embedded tailscaled's IPN notification bus via local.Client.WatchIPNBus with the ipn.NotifyInitialHealthState filter, so terminal login failures can trigger an auth key reissue. This error wraps the failure to open that watch through the tsnet local API. It almost always means the embedded tailscaled in the same container was not reachable or not ready when the watch was attempted.

Source

Thrown at cmd/k8s-proxy/kube.go:94

	return false
}

// checkInitialAuthState checks if the tsnet server is in an auth failure state
// immediately after coming up. Returns true if auth key reissue is needed.
func checkInitialAuthState(ctx context.Context, lc *local.Client) (bool, error) {
	status, err := lc.Status(ctx)
	if err != nil {
		return false, fmt.Errorf("error getting status: %w", err)
	}
	return needsAuthKeyReissue(status.BackendState, status.Health), nil
}

// monitorAuthHealth watches the IPN bus for auth failures and triggers reissue
// when needed. Runs until context is cancelled or auth failure is detected.
func monitorAuthHealth(ctx context.Context, lc *local.Client, reissueCh chan<- struct{}, logger *zap.SugaredLogger) error {
	w, err := lc.WatchIPNBus(ctx, ipn.NotifyInitialHealthState)
	if err != nil {
		return fmt.Errorf("failed to watch IPN bus for auth health: %w", err)
	}
	defer w.Close()

	for {
		if ctx.Err() != nil {
			return ctx.Err()
		}
		n, err := w.Next()
		if err != nil {
			return err
		}
		if n.Health != nil {
			if _, ok := n.Health.Warnings[health.LoginStateWarnable.Code]; ok {
				logger.Info("Auth key failed to authenticate (may be expired or single-use), requesting new key from operator")
				select {
				case reissueCh <- struct{}{}:
				case <-ctx.Done():
				}

View on GitHub (pinned to cfe32b8be6)

Solutions

1. Check the container's earlier log lines for tsnet/localapi startup errors or state-dir permission failures
2. Verify the state volume is mounted writable by the container user and not corrupted
3. Confirm an auth key was available on first boot (TS_AUTHKEY / operator-provided Secret)
4. Delete the pod so it restarts cleanly; transient startup races resolve on retry

// Ping the local API before starting the monitor, so a hard failure is
// distinguishable from a startup race.
if _, err := lc.Status(ctx); err != nil {
    logger.Errorf("local API not ready, retrying: %v", err)
}

err := monitorAuthHealth(ctx, lc, reissueCh, logger)
switch {
case errors.Is(err, context.Canceled), errors.Is(err, context.DeadlineExceeded):
    return // pod shutdown, not a fault
case err != nil:
    logger.Errorf("auth health monitor failed: %v", err)
    // re-run with backoff; tsnet may still be booting
}

When it happens

Trigger: lc.WatchIPNBus(ctx, ipn.NotifyInitialHealthState) returning a non-nil error: the tsnet server has not finished starting (local API not listening yet), the tsnet state directory is unreadable/unwritable so tailscaled failed to boot, or the ctx was already cancelled (pod shutting down) at call time.

Common situations: Proxy container starts the monitor goroutine before tsnet finishes initialization; corrupted or permission-broken --state-dir (volume mount issues); missing TS_AUTHKEY on first boot leaving tsnet unauthenticated; context cancellation racing pod termination.

Related errors

• failed to get local client: %w
• failed to get status: %w
• error disconnecting from control: %w
• expected at least one argument
• expected at least one argument after method

AI-assisted analysis of tailscale/tailscale@cfe32b8be6 (2026-08-15). Data as JSON: /api/errors/352f9305912a55ba. Report an issue: GitHub.