{"record":{"id":"138422cee21dc8d5","repo":"tailscale/tailscale","slug":"failed-to-remove-finalizer-w-138422","errorCode":null,"errorMessage":"failed to remove finalizer: %w","messagePattern":"failed to remove finalizer: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/k8s-operator/ingress.go","lineNumber":115,"sourceCode":"\tif ix < 0 {\n\t\tlogger.Debugf(\"no finalizer, nothing to do\")\n\t\ta.mu.Lock()\n\t\tdefer a.mu.Unlock()\n\t\ta.managedIngresses.Remove(ing.UID)\n\t\tgaugeIngressResources.Set(int64(a.managedIngresses.Len()))\n\t\treturn nil\n\t}\n\n\tif done, err := a.ssr.Cleanup(ctx, operatorTailnet, logger, childResourceLabels(ing.Name, ing.Namespace, \"ingress\"), proxyTypeIngressResource); err != nil {\n\t\treturn fmt.Errorf(\"failed to cleanup: %w\", err)\n\t} else if !done {\n\t\tlogger.Debugf(\"cleanup not done yet, waiting for next reconcile\")\n\t\treturn nil\n\t}\n\n\ting.Finalizers = append(ing.Finalizers[:ix], ing.Finalizers[ix+1:]...)\n\tif err := a.Update(ctx, ing); err != nil {\n\t\treturn fmt.Errorf(\"failed to remove finalizer: %w\", err)\n\t}\n\n\t// Unlike most log entries in the reconcile loop, this will get printed\n\t// exactly once at the very end of cleanup, because the final step of\n\t// cleanup removes the tailscale finalizer, which will make all future\n\t// reconciles exit early.\n\tlogger.Infof(\"unexposed ingress from tailnet\")\n\ta.mu.Lock()\n\tdefer a.mu.Unlock()\n\ta.managedIngresses.Remove(ing.UID)\n\tgaugeIngressResources.Set(int64(a.managedIngresses.Len()))\n\treturn nil\n}\n\n// maybeProvision ensures that ing is exposed over tailscale, taking any actions\n// necessary to reach that state.\n//\n// This function adds a finalizer to ing, ensuring that we can handle orderly","sourceCodeStart":97,"sourceCodeEnd":133,"githubUrl":"https://github.com/tailscale/tailscale/blob/cfe32b8be6a33f8e24fbc369cbfbf7c729d9e042/cmd/k8s-operator/ingress.go#L97-L133","documentation":"After cleanup reported done, the reconciler strips the tailscale finalizer from the Ingress and calls a.Update. If that update fails — most commonly a 409 optimistic-lock conflict because the Ingress was modified concurrently (by a controller or kubectl apply), or an RBAC update denial — the finalizer stays and the Ingress cannot be garbage-collected.","triggerScenarios":"a.Update(ctx, ing) right after ing.Finalizers is sliced. Fires on concurrent Ingress writes between the Get at reconcile start and this Update (conflict), missing update permission on ingresses, or apiserver errors. The parent Reconcile already special-cases optimistic-lock errors and retries them.","commonSituations":"Another controller (cert-manager, Istio, Argo) writing the Ingress at the same moment; kubectl apply racing the deletion; RBAC without ingresses/update.","solutions":["If conflict: no action — the reconciler logs 'optimistic lock error, retrying' and the requeue retries the finalizer removal","Verify update permission on ingresses.networking.k8s.io for the operator service account","Reduce churn from other controllers writing the Ingress metadata during deletion","If stuck, manually strip the finalizer: kubectl patch ingress <name> -n <ns> --type=json -p='[{\"op\":\"remove\",\"path\":\"/metadata/finalizers\"}]' after confirming proxy resources are gone"],"exampleFix":"// before\nif err := a.Update(ctx, ing); err != nil {\n    return fmt.Errorf(\"failed to remove finalizer: %w\", err)\n}\n\n// after (retry once on conflict before surfacing)\nif err := a.Update(ctx, ing); apierrors.IsConflict(err) {\n    // requeue immediately; someone else wrote the Ingress\n    return reconcile.Result{Requeue: true}, nil\n} else if err != nil {\n    return fmt.Errorf(\"failed to remove finalizer: %w\", err)\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"if err := a.Update(ctx, ing); err != nil {\n    if apierrors.IsConflict(err) {\n        // Ingress changed concurrently; requeue rebuilds from a fresh Get\n        return reconcile.Result{Requeue: true}, nil\n    }\n    return fmt.Errorf(\"failed to remove finalizer: %w\", err)\n}","preventionTips":["Limit concurrent writers to Ingress metadata (one controller owns finalizers)","Keep ingresses update permission in the operator RBAC","Verify finalizer removal completed (kubectl get ingress -o jsonpath='{.metadata.finalizers}') before assuming cleanup finished"],"tags":["kubernetes","ingress","finalizer","conflict","operator"],"backgroundTag":null,"analyzedSha":"cfe32b8be6a33f8e24fbc369cbfbf7c729d9e042","analyzedAt":"2026-08-15T19:58:31.583Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}