{"record":{"id":"ec696fafba096bb5","repo":"kubernetes/kops","slug":"failed-to-detach-instance-v","errorCode":null,"errorMessage":"failed to detach instance: %v","messagePattern":"failed to detach instance: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/instancegroups/instancegroups.go","lineNumber":772,"sourceCode":"\t\tif apierrors.IsNotFound(err) {\n\t\t\treturn nil\n\t\t}\n\n\t\treturn fmt.Errorf(\"error deleting node: %v\", err)\n\t}\n\n\treturn nil\n}\n\n// UpdateSingleInstance performs a rolling update on a single instance\nfunc (c *RollingUpdateCluster) UpdateSingleInstance(ctx context.Context, cloudMember *cloudinstances.CloudInstance, detach bool) error {\n\tif detach {\n\t\tif cloudMember.CloudInstanceGroup.InstanceGroup.IsControlPlane() {\n\t\t\tklog.Warning(\"cannot detach control-plane instances. Assuming --surge=false\")\n\t\t} else if cloudMember.CloudInstanceGroup.InstanceGroup.Spec.Manager != api.InstanceManagerKarpenter {\n\t\t\terr := c.detachInstance(cloudMember)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to detach instance: %v\", err)\n\t\t\t}\n\t\t\tif err := c.maybeValidate(\" after detaching instance\", c.ValidateCount, cloudMember.CloudInstanceGroup); err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t}\n\t}\n\n\treturn c.drainTerminateAndWait(ctx, cloudMember, 0)\n}\n","sourceCodeStart":754,"sourceCodeEnd":782,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/pkg/instancegroups/instancegroups.go#L754-L782","documentation":"UpdateSingleInstance detaches the instance from its cloud autoscaling/instance group before replacing it (surge updates). When c.detachInstance fails, the error is wrapped as 'failed to detach instance'. Control-plane instances are skipped by design and Karpenter-managed groups are skipped entirely.","triggerScenarios":"The provider detach call fails: AWS AutoScalingGroup DetachInstances errors (instance not in group, ASG not found), permission denied, or the group is managed by a controller that also mutates membership concurrently.","commonSituations":"Instance was already removed/replaced by the autoscaler so the ID is no longer a member; IAM missing autoscaling:DetachInstances; kOps state out of sync after a partially failed previous rolling update; using Karpenter-managed groups incorrectly flagged (Manager not set).","solutions":["Confirm instance membership in the ASG (`aws autoscaling describe-auto-scaling-instances`) and retry once state is consistent","Grant IAM permission autoscaling:DetachInstances on the ASG","For Karpenter-managed groups, set spec.Manager: karpenter in the InstanceGroup so detach is skipped","Re-run `kops rolling-update cluster` — it re-syncs group membership before detaching"],"exampleFix":"// before\nkind: InstanceGroup\nmetadata:\n  name: nodes\n// after\nkind: InstanceGroup\nmetadata:\n  name: nodes\nspec:\n  manager: karpenter","handlingStrategy":"try-catch","validationCode":"members, err := asgSvc.DescribeAutoScalingInstances(&autoscaling.DescribeAutoScalingInstancesInput{InstanceIds: []string{instanceID}})\nif err != nil || len(members.AutoScalingInstances) == 0 {\n    return fmt.Errorf(\"instance %s not in any ASG; detach will fail\", instanceID)\n}","typeGuard":null,"tryCatchPattern":"if err := c.UpdateSingleInstance(ctx, cloudMember, detach, true); err != nil {\n    if strings.Contains(err.Error(), \"failed to detach instance\") {\n        klog.Warningf(\"detach failed: %v; will retry after resync\", err)\n        time.Sleep(backoff)\n        return c.UpdateSingleInstance(ctx, cloudMember, detach, true)\n    }\n    return err\n}","preventionTips":["Set spec.Manager: karpenter for Karpenter-managed groups","Grant autoscaling:DetachInstances IAM permission","Avoid concurrent autoscaler activity during rolling updates"],"tags":["aws","autoscaling","detach","rolling-update"],"backgroundTag":"cloud-api-detach-failed","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}