kubernetes/kops · error

error detaching instance %q, node %q: %v

Error message

error detaching instance %q, node %q: %v

What it means

kOps detaches an instance from its cloud instance group (ASG/mig) before terminating or updating it. This error wraps a failure of the cloud provider's DetachInstance call for an instance that has an associated Kubernetes node name.

Source

Thrown at pkg/instancegroups/instancegroups.go:638

	return false
}

// detachInstance detaches a Cloud Instance
func (c *RollingUpdateCluster) detachInstance(u *cloudinstances.CloudInstance) error {
	id := u.ID
	nodeName := ""
	if u.Node != nil {
		nodeName = u.Node.Name
	}
	if nodeName != "" {
		klog.Infof("Detaching instance %q, node %q, in group %q.", id, nodeName, u.CloudInstanceGroup.HumanName)
	} else {
		klog.Infof("Detaching instance %q, in group %q.", id, u.CloudInstanceGroup.HumanName)
	}

	if err := c.Cloud.DetachInstance(u); err != nil {
		if nodeName != "" {
			return fmt.Errorf("error detaching instance %q, node %q: %v", id, nodeName, err)
		}
		return fmt.Errorf("error detaching instance %q: %v", id, err)
	}

	return nil
}

// deleteInstance deletes an Cloud Instance.
func (c *RollingUpdateCluster) deleteInstance(u *cloudinstances.CloudInstance) error {
	id := u.ID
	nodeName := ""
	if u.Node != nil {
		nodeName = u.Node.Name
	}
	if nodeName != "" {
		klog.Infof("Stopping instance %q, node %q, in group %q (this may take a while).", id, nodeName, u.CloudInstanceGroup.HumanName)
	} else {
		klog.Infof("Stopping instance %q, in group %q (this may take a while).", id, u.CloudInstanceGroup.HumanName)

View on GitHub (pinned to 4c8573c808)

Solutions

1. Re-run the rolling update; group membership state is re-discovered
2. Check cloud IAM permissions for detach operations
3. Look for external automation (ASG edits, cluster-autoscaler) conflicting with the roll
4. Retry after cloud API throttling subsides

Example fix

// before
// IAM policy missing autoscaling detach
// after
{ "Effect": "Allow", "Action": ["autoscaling:DetachInstances"], "Resource": "*" }

// pre-flight: confirm instance still in group via cloud API before detach
asg, err := autoscalingClient.DescribeAutoScalingGroups(...)
// verify instanceIDs contains u.ID before calling DetachInstance

func inGroup(u *cloudinstances.CloudInstance, memberIDs []string) bool {
    return u != nil && u.ID != "" && contains(memberIDs, u.ID)
}

if err := c.Cloud.DetachInstance(u); err != nil {
    if isNotFound(err) || isAlreadyDetached(err) { return nil }
    if isThrottling(err) { /* backoff and retry */ }
    return fmt.Errorf("error detaching instance %q, node %q: %v", id, nodeName, err)
}

When it happens

Trigger: c.Cloud.DetachInstance(u) errors (AWS: autoscaling detach_instances; GCP: mig removeInstances) while nodeName != "". Causes include the instance no longer belonging to the group, API throttling, or IAM permission failures.

Common situations: ASG modified externally between discovery and detach; AWS API rate limits during large rolls; missing autoscaling:DetachInstances IAM permission; instance in a transitional state.

Related errors

• error detaching instance %q: %v
• failed to detach instance: %v
• the image for the hook exec action not set
• errShutdown
• error deleting instance %q, node %q: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/afe5d475b9c6f87b. Report an issue: GitHub.