{"record":{"id":"c9ee65cc3a2b1de4","repo":"kubernetes/kops","slug":"failed-to-taint-node-q-v","errorCode":null,"errorMessage":"failed to taint node %q: %v","messagePattern":"failed to taint node %q: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/instancegroups/instancegroups.go","lineNumber":368,"sourceCode":"\t\t\t\tif taint.Key == rollingUpdateTaintKey {\n\t\t\t\t\tfoundTaint = true\n\t\t\t\t}\n\t\t\t}\n\t\t\tif !foundTaint {\n\t\t\t\ttoTaint = append(toTaint, u.Node)\n\t\t\t}\n\t\t}\n\t}\n\tif len(toTaint) > 0 {\n\t\tnoun := \"nodes\"\n\t\tif len(toTaint) == 1 {\n\t\t\tnoun = \"node\"\n\t\t}\n\t\tklog.Infof(\"Tainting %d %s in %q instancegroup.\", len(toTaint), noun, group.InstanceGroup.Name)\n\t\tfor _, n := range toTaint {\n\t\t\tif err := c.patchTaint(ctx, n); err != nil {\n\t\t\t\tif c.FailOnDrainError {\n\t\t\t\t\treturn fmt.Errorf(\"failed to taint node %q: %v\", n, err)\n\t\t\t\t}\n\t\t\t\tklog.Infof(\"Ignoring error tainting node %q: %v\", n, err)\n\t\t\t}\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc (c *RollingUpdateCluster) patchTaint(ctx context.Context, node *corev1.Node) error {\n\toldData, err := json.Marshal(node)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tnode.Spec.Taints = append(node.Spec.Taints, corev1.Taint{\n\t\tKey:    rollingUpdateTaintKey,\n\t\tEffect: corev1.TaintEffectPreferNoSchedule,\n\t})","sourceCodeStart":350,"sourceCodeEnd":386,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/pkg/instancegroups/instancegroups.go#L350-L386","documentation":"During a rolling update, kOps taints Kubernetes nodes that need updating so no new workloads are scheduled there. This error wraps a failure from patching the node's taint via the Kubernetes API when FailOnDrainError is enabled; otherwise the error is only logged and rolling update continues.","triggerScenarios":"c.patchTaint(ctx, n) returns an error for a node in the toTaint list and c.FailOnDrainError is true. Typical causes: node is already deleted/NotReady, RBAC forbids patching nodes, or the API server is unreachable.","commonSituations":"API server briefly unavailable mid-roll; node object stale after cloud instance was replaced out-of-band; kOps service account lacking nodes patch permission; K8s version skew causing patch failures.","solutions":["Re-run the rolling update; tainting is transient and often succeeds on retry","Verify the node exists and is Ready via kubectl get nodes","Check RBAC permissions to patch nodes for the kops controller identity","Inspect connectivity to the API server (kubeconfig, VPN, security groups)"],"exampleFix":"// before\nklog.Infof(\"Ignoring error tainting node %q: %v\", n, err) // silently continuing\n// after\n// run with --fail-on-drain-error (FailOnDrainError=true) so taint failures abort the roll and can be retried cleanly","handlingStrategy":"retry","validationCode":"// before rolling update\nnodes, _ := clientset.CoreV1().Nodes().List(ctx, metav1.ListOptions{})\nfor _, n := range nodes.Items {\n    if n.DeletionTimestamp != nil || !isNodeReady(n) { /* exclude from taint or wait */ }\n}\n// also verify RBAC: auth can-i patch nodes","typeGuard":"func isTaintable(n *corev1.Node) bool {\n    return n != nil && n.DeletionTimestamp == nil && hasReadyConditionTrue(n)\n}","tryCatchPattern":"err := c.patchTaint(ctx, n)\nif err != nil {\n    if k8serrors.IsNotFound(err) || k8serrors.IsConflict(err) { continue } // node gone; safe to proceed\n    return fmt.Errorf(\"failed to taint node %q: %v\", n, err)\n}","preventionTips":["Check node Ready/DeletionTimestamp before rolling","Confirm RBAC (can-i patch nodes) for the operator identity","Run with FailOnDrainError in staging first to surface issues","Retry rolls; taint errors are often transient"],"tags":["kubernetes","rolling-update","taint","api-server"],"backgroundTag":"kubernetes-api-request-failed","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T12:17:11.808Z"}