{"record":{"id":"a1e089d5c1dd154c","repo":"kubernetes/kops","slug":"error-deregistering-instance-q-node-q-w","errorCode":null,"errorMessage":"error deregistering instance %q, node %q: %w","messagePattern":"error deregistering instance %q, node %q: %w","errorType":"exception","errorClass":"DeregisterError","httpStatus":null,"severity":"error","filePath":"pkg/instancegroups/instancegroups.go","lineNumber":729,"sourceCode":"\t\treturn fmt.Errorf(\"error excluding node from load balancer: %v\", err)\n\t}\n\n\tshouldDeregister := true\n\tif !c.Options.DeregisterControlPlaneNodes {\n\t\tif u.CloudInstanceGroup != nil && u.CloudInstanceGroup.InstanceGroup != nil {\n\t\t\trole := u.CloudInstanceGroup.InstanceGroup.Spec.Role\n\t\t\tswitch role {\n\t\t\tcase api.InstanceGroupRoleAPIServer, api.InstanceGroupRoleControlPlane:\n\t\t\t\tklog.Infof(\"skipping deregistration of instance %q, as part of instancegroup with role %q\", u.ID, role)\n\t\t\t\tshouldDeregister = false\n\t\t\t}\n\t\t}\n\t}\n\n\tif shouldDeregister {\n\t\tif err := c.Cloud.DeregisterInstance(u); err != nil {\n\t\t\treturn &DeregisterError{\n\t\t\t\terr: fmt.Errorf(\"error deregistering instance %q, node %q: %w\", u.ID, u.Node.Name, err),\n\t\t\t}\n\t\t}\n\t}\n\n\tif err := drain.RunNodeDrain(helper, u.Node.Name); err != nil {\n\t\tif apierrors.IsNotFound(err) {\n\t\t\treturn nil\n\t\t}\n\t\treturn fmt.Errorf(\"error draining node: %v\", err)\n\t}\n\n\tif c.PostDrainDelay > 0 {\n\t\tklog.Infof(\"Waiting for %s for pods to stabilize after draining.\", c.PostDrainDelay)\n\t\ttime.Sleep(c.PostDrainDelay)\n\t}\n\n\treturn nil\n}","sourceCodeStart":711,"sourceCodeEnd":747,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/pkg/instancegroups/instancegroups.go#L711-L747","documentation":"Before draining pods, drainNode may deregister the instance from cloud load balancers via c.Cloud.DeregisterInstance(u). Any failure is wrapped in a DeregisterError carrying 'error deregistering instance %q, node %q: %w'. This prevents the node from being safely removed from backend target pools before it disappears.","triggerScenarios":"DeregisterInstance fails on the cloud provider side: instance already terminated (AWS InvalidInstanceID), target group / backend service not found, cloud API throttling or credential issues.","commonSituations":"Stale kOps state where the instance was deleted externally; IAM policy missing elbv2/autoscaling deregister permissions; API rate limits during large rolling updates; provider SDK errors after region/credential changes.","solutions":["Verify IAM permissions for deregistering instances from load balancers (e.g. elasticloadbalancing:* on the target groups)","Confirm the instance still exists: `aws ec2 describe-instances --instance-ids <id>`; if already gone, retry the rolling update (NotFound-like cases often proceed)","Check cloud API rate limits/errors in kOps logs and reduce MaxSurge/parallelism or increase intervals","Refresh credentials/config (`kops export kubecfg`) if auth errors appear"],"exampleFix":"// before: policy without ELB permissions\n{\"Effect\":\"Deny\",\"Action\":\"elasticloadbalancing:DeregisterTargets\",\"Resource\":\"*\"}\n// after: allow deregistration\n{\"Effect\":\"Allow\",\"Action\":\"elasticloadbalancing:DeregisterTargets\",\"Resource\":\"*\"}","handlingStrategy":"retry","validationCode":"if _, err := elbSvc.DescribeTargetHealth(tgArn); err != nil {\n    return fmt.Errorf(\"cannot reach load balancer API: %w\", err)\n}","typeGuard":null,"tryCatchPattern":"var de *DeregisterError\nif err := c.drainNode(ctx, u); err != nil {\n    if errors.As(err, &de) {\n        time.Sleep(backoff)\n        err = c.drainNode(ctx, u) // cloud API errors are often transient\n    }\n    return err\n}","preventionTips":["Grant elasticloadbalancing deregister IAM permissions","Throttle rolling-update parallelism to avoid API rate limits","Keep kOps state refreshed so instance IDs are current"],"tags":["aws","load-balancer","deregister","rolling-update"],"backgroundTag":"cloud-api-deregister-failed","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}