{"record":{"id":"352f9305912a55ba","repo":"tailscale/tailscale","slug":"failed-to-watch-ipn-bus-for-auth-health-w","errorCode":null,"errorMessage":"failed to watch IPN bus for auth health: %w","messagePattern":"failed to watch IPN bus for auth health: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/k8s-proxy/kube.go","lineNumber":94,"sourceCode":"\treturn false\n}\n\n// checkInitialAuthState checks if the tsnet server is in an auth failure state\n// immediately after coming up. Returns true if auth key reissue is needed.\nfunc checkInitialAuthState(ctx context.Context, lc *local.Client) (bool, error) {\n\tstatus, err := lc.Status(ctx)\n\tif err != nil {\n\t\treturn false, fmt.Errorf(\"error getting status: %w\", err)\n\t}\n\treturn needsAuthKeyReissue(status.BackendState, status.Health), nil\n}\n\n// monitorAuthHealth watches the IPN bus for auth failures and triggers reissue\n// when needed. Runs until context is cancelled or auth failure is detected.\nfunc monitorAuthHealth(ctx context.Context, lc *local.Client, reissueCh chan<- struct{}, logger *zap.SugaredLogger) error {\n\tw, err := lc.WatchIPNBus(ctx, ipn.NotifyInitialHealthState)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"failed to watch IPN bus for auth health: %w\", err)\n\t}\n\tdefer w.Close()\n\n\tfor {\n\t\tif ctx.Err() != nil {\n\t\t\treturn ctx.Err()\n\t\t}\n\t\tn, err := w.Next()\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t\tif n.Health != nil {\n\t\t\tif _, ok := n.Health.Warnings[health.LoginStateWarnable.Code]; ok {\n\t\t\t\tlogger.Info(\"Auth key failed to authenticate (may be expired or single-use), requesting new key from operator\")\n\t\t\t\tselect {\n\t\t\t\tcase reissueCh <- struct{}{}:\n\t\t\t\tcase <-ctx.Done():\n\t\t\t\t}","sourceCodeStart":76,"sourceCodeEnd":112,"githubUrl":"https://github.com/tailscale/tailscale/blob/cfe32b8be6a33f8e24fbc369cbfbf7c729d9e042/cmd/k8s-proxy/kube.go#L76-L112","documentation":"In the Tailscale Kubernetes operator's proxy pod, monitorAuthHealth opens a subscription to the embedded tailscaled's IPN notification bus via local.Client.WatchIPNBus with the ipn.NotifyInitialHealthState filter, so terminal login failures can trigger an auth key reissue. This error wraps the failure to open that watch through the tsnet local API. It almost always means the embedded tailscaled in the same container was not reachable or not ready when the watch was attempted.","triggerScenarios":"lc.WatchIPNBus(ctx, ipn.NotifyInitialHealthState) returning a non-nil error: the tsnet server has not finished starting (local API not listening yet), the tsnet state directory is unreadable/unwritable so tailscaled failed to boot, or the ctx was already cancelled (pod shutting down) at call time.","commonSituations":"Proxy container starts the monitor goroutine before tsnet finishes initialization; corrupted or permission-broken --state-dir (volume mount issues); missing TS_AUTHKEY on first boot leaving tsnet unauthenticated; context cancellation racing pod termination.","solutions":["Check the container's earlier log lines for tsnet/localapi startup errors or state-dir permission failures","Verify the state volume is mounted writable by the container user and not corrupted","Confirm an auth key was available on first boot (TS_AUTHKEY / operator-provided Secret)","Delete the pod so it restarts cleanly; transient startup races resolve on retry"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Ping the local API before starting the monitor, so a hard failure is\n// distinguishable from a startup race.\nif _, err := lc.Status(ctx); err != nil {\n    logger.Errorf(\"local API not ready, retrying: %v\", err)\n}","typeGuard":null,"tryCatchPattern":"err := monitorAuthHealth(ctx, lc, reissueCh, logger)\nswitch {\ncase errors.Is(err, context.Canceled), errors.Is(err, context.DeadlineExceeded):\n    return // pod shutdown, not a fault\ncase err != nil:\n    logger.Errorf(\"auth health monitor failed: %v\", err)\n    // re-run with backoff; tsnet may still be booting\n}","preventionTips":["Give tsnet time to finish startup before starting health monitors; retry with backoff on first failure","Keep the tsnet state directory writable and backed by a healthy volume","Provide TS_AUTHKEY on first boot so tailscaled reaches a queryable state"],"tags":["kubernetes","tailscale","tsnet","ipnbus","localapi","auth"],"backgroundTag":null,"analyzedSha":"cfe32b8be6a33f8e24fbc369cbfbf7c729d9e042","analyzedAt":"2026-08-15T19:58:31.583Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}