{"record":{"id":"0cae6d6db7dc441a","repo":"tailscale/tailscale","slug":"failed-to-clean-up-cert-resources-w-0cae6d","errorCode":null,"errorMessage":"failed to clean up cert resources: %w","messagePattern":"failed to clean up cert resources: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/k8s-operator/ingress-for-pg.go","lineNumber":526,"sourceCode":"\t\t// 3. Delete the Tailscale Service from the control plane.\n\t\ttsService, err := tsClient.VIPServices().Get(ctx, tsSvcName.String())\n\t\tswitch {\n\t\tcase tailscale.IsNotFound(err):\n\t\t\t// Already gone at the control plane; continue with cluster\n\t\t\t// cleanup rather than aborting the sweep.\n\t\tcase err != nil:\n\t\t\treturn svcsChanged, fmt.Errorf(\"getting Tailscale Service %q: %w\", tsSvcName, err)\n\t\tdefault:\n\t\t\tupdated, err := r.cleanupTailscaleService(ctx, tsService, logger, tsClient)\n\t\t\tif err != nil {\n\t\t\t\treturn svcsChanged, fmt.Errorf(\"deleting Tailscale Service %q: %w\", tsSvcName, err)\n\t\t\t}\n\t\t\tsvcsChanged = svcsChanged || updated\n\t\t}\n\n\t\t// 4. Clean up cluster cert resources.\n\t\tif err := cleanupCertResources(ctx, r.Client, r.tsNamespace, tsSvcName, pg); err != nil {\n\t\t\treturn svcsChanged, fmt.Errorf(\"failed to clean up cert resources: %w\", err)\n\t\t}\n\t}\n\n\treturn svcsChanged, nil\n}\n\n// maybeCleanup ensures that any resources, such as a Tailscale Service created for this Ingress, are cleaned up when the\n// Ingress is being deleted or is unexposed. The cleanup is safe for a multi-cluster setup- the Tailscale Service is only\n// deleted if it does not contain any other owner references. If it does the cleanup only removes the owner reference\n// corresponding to this Ingress.\n//\n// Steps are ordered so the proxy cancels its cert loop (via serve config\n// removal) before the VIPService is deleted; otherwise the loop retries\n// against a domain the control plane no longer recognises.\nfunc (r *HAIngressReconciler) maybeCleanup(ctx context.Context, hostname string, ing *networkingv1.Ingress, logger *zap.SugaredLogger, tsClient tsclient.Client, pg *tsapi.ProxyGroup) (svcChanged bool, err error) {\n\tlogger.Debugf(\"Ensuring any resources for Ingress are cleaned up\")\n\tix := slices.Index(ing.Finalizers, FinalizerNamePG)\n\tif ix < 0 {","sourceCodeStart":508,"sourceCodeEnd":544,"githubUrl":"https://github.com/tailscale/tailscale/blob/cfe32b8be6a33f8e24fbc369cbfbf7c729d9e042/cmd/k8s-operator/ingress-for-pg.go#L508-L544","documentation":"Thrown by HAIngressReconciler.maybeCleanupProxyGroup when cleanupCertResources fails to delete the per-domain TLS Secret and its Role/RoleBinding (label-selected on ProxyGroup and domain) for an orphaned Tailscale Service. It wraps dnsNameForService failing because no ProxyGroup node metadata exists (Pods already deleted: deriving the MagicDNS domain needs a live proxy's DNS name), or DeleteAllOf on RoleBinding/Role/Secret failing (RBAC, apiserver).","triggerScenarios":"ProxyGroup or its Pods deleted before its Ingresses finished cleanup, so the DNS name cannot be derived; operator RBAC missing delete on roles/rolebindings/secrets; apiserver errors during teardown.","commonSituations":"Uninstalling the operator and ProxyGroup in one shot, leaving Ingress finalizers stuck; RBAC pruned during teardown; version skew between operators.","solutions":["Order teardown: delete Ingresses and wait for finalizers to clear before deleting the ProxyGroup or operator.","Reapply full RBAC if 'forbidden' appears on the DeleteAllOf calls.","If the ProxyGroup is already gone: verify no control-plane VIPService remains, then manually remove the operator finalizer from the stuck Ingress and delete leftover cert Secrets, Roles, and RoleBindings by label."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// teardown precondition: DNS name must be derivable, which needs live proxy state\nif _, err := dnsNameForService(ctx, cl, serviceName, pg, tsNamespace); err != nil {\n\t// postpone ProxyGroup deletion until Ingress finalizers have cleared\n}","typeGuard":null,"tryCatchPattern":"if err := cleanupCertResources(ctx, r.Client, r.tsNamespace, tsSvcName, pg); err != nil {\n\tif strings.Contains(err.Error(), \"dns name\") {\n\t\tlogger.Error(\"ProxyGroup metadata is gone; remove cert Secrets, Roles, RoleBindings by label and the Ingress finalizer manually\")\n\t}\n\treturn svcsChanged, fmt.Errorf(\"failed to clean up cert resources: %w\", err)\n}","preventionTips":["Delete Ingresses before the ProxyGroup; cleanup derives DNS names from live proxy state.","Keep operator RBAC (roles, rolebindings, secrets delete) intact until teardown completes.","After manual cleanup, delete leftover resources by the operator's labels, not by guesswork."],"tags":["kubernetes","rbac","secrets","proxygroup","cleanup"],"backgroundTag":null,"analyzedSha":"cfe32b8be6a33f8e24fbc369cbfbf7c729d9e042","analyzedAt":"2026-08-15T19:58:31.583Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}